JP2003330463A - Composition support system and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a composition support system and method which permit creation of new phrases and musical pieces. <P>SOLUTION: An operator 16 lines up note cards 18 on a table 14. A camera 12 disposed above the table 14 photographs the table and the cards and imparts camera video signals to a computer 24. The computer 24 acquires the two-dimensional coordinate information indicating the positions and arrangement information of the note cards 18 from the camera video signals and forms note events (MIDI data) according thereto. The MIDI data is imparted to an electronic musical instrument 26. The instrument 26 produces the tones of the musical expressions (timbre, interval, loudness, etc.), meeting the MIDI data, i.e., the positions and arrangement of the note cards. A series of the note events are stored as phrases into the computer 24 and are displayed on a screen 34. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a music composition support system and method, and more particularly, for example, a card arranged on a table is photographed by a camera, and position and arrangement information of each card is obtained based on a camera video signal from the camera. The present invention relates to a novel composition support system and method that can proceed with composition by capturing and storing a note event corresponding to the position and arrangement of cards.

[0002]

2. Description of the Related Art An example of a musical expression element changing means (musical instrument interface) which is considered to be relatively related to the present invention is disclosed in Japanese Patent Application Laid-Open No. 2002-32076 [G10H1 / 00, 1/05
3, 1/32 H04N7 / 18].

According to this prior art, according to the three-dimensional position information of a handheld object such as a card, an effect, a filter, a pitch, etc. are applied to a phrase (sound frame) of a musical instrument previously associated with the handheld object. It is a musical instrument interface that enables various musical expressions by changing the musical expression elements.

[0004]

By using this conventional technique, it is possible to newly create rich musical expressions while having fun by giving various changes to existing phrases. However, even with this technique, it is not possible to freely create new phrases that should be played or want to be played. This is because this conventional technique can only be applied to existing frames.

Therefore, a main object of the present invention is to provide a composition support system and method that enables the creation of new phrases and music.

[0006]

The present invention comprises: (a) acquiring at least two-dimensional information of a note card placed on a table; (b) creating a note event based on the two-dimensional information; And (c) including the step of pronouncing the musical expression according to the note event, step (a)
This is a composition support method in which ~ (c) is repeated to generate a series of note events, and (d) the series of note events is stored as a phrase.

The present invention is a composition support method for creating a series of music data in accordance with at least the positions of a plurality of note cards arranged on a table.

The present invention is based on a two-dimensional information acquisition means for acquiring at least two-dimensional information of a musical note card placed on a table, a musical note event generating means for generating a musical note event based on the two-dimensional information, and a musical note event. The composition support system includes a pronunciation unit that produces a musical expression and a phrase storage unit that stores a series of note events created by the note event creation unit as phrases.

[0009]

In the embodiment, a substantial music card is used, and the music card is placed on the substantial table,
By photographing the state with a camera, the camera image signal is taken into the computer, and the two-dimensional information (for example, two-dimensional coordinates and rotation angle) of the music card is acquired based on the camera image signal. However, when using a virtual music card or virtual table, such a camera is not necessary and can be obtained by the computer by referring to the contents of the display memory.

According to the thus obtained two-dimensional information, the computer sends the two-dimensional information to the MID, for example.
A note event (music data) such as I data is generated. For example, MIDI data is given to an electronic musical instrument having a MIDI sound source. Therefore, the electronic musical instrument has a musical expression (timbre, pitch, duration,
Pronunciation). The operator can correct the position and arrangement of the note card as necessary by confirming the pronunciation from the electronic musical instrument.

Then, a series of note events is stored as phrase data by the computer.

By visually displaying the position of the note card on the table, the operator can confirm the note event created by himself not only by his ear but also by his eyes.

When a phrase card created for each phrase is placed on a table, a musical expression according to a series of note events included in the phrase card is pronounced, and the series of note events and, if necessary, Visually display changed elements. Then, when the change element acts on the displayed note event in a predetermined manner, the musical expression of the note event can be changed. In the embodiment, the change element is a computer graphics (hereinafter, CG) spanner displayed based on the note change card and the phrase change card. Therefore, the operator can arbitrarily change the created phrase, that is, the phrase card. Then, the changed content may be stored as a new phrase card.

[0014]

According to the present invention, it is possible to compose music by arranging a music card on a table using a substantial music card or table or a virtual music card or virtual table.

The above-mentioned objects, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Referring to FIG. 1, the composition assisting system 10 of this embodiment includes a camera 12, which is installed above the table 14, for example, so that the entire upper surface of the table 14 can be photographed. It And operator 1
6 can hold a card 18, which is an example of a hand-held object, by hand, and place the card 18 on the table 14 or change its position.

FIG. 2 shows an example of the card 18. The card 18 includes, for example, a rectangular marker 20 formed on its surface and a symbol 22 displayed in the marker 20. The marker 20 and the symbol 22 are the computer 2
4 and the operator 16 has a meaning readable. That is, the marker 20 means that the symbol 22 to be identified is described therein, and the symbol 22 is a predetermined musical expression or a predetermined musical expression depending on its pattern. It has the meaning of operation. However, the shapes of the marker 20 and the symbol 22 in FIG. 2 are merely examples, and other arbitrary shapes may be selected. The back surface of the card 18 has no pattern, for example, is entirely white.

The camera 12 photographs the table 14 and the card 18 placed by the operator 16 on the table 14, and supplies a camera video signal to the computer 24 shown in FIG. As will be described in detail later, the computer 24 takes in the position, orientation, or arrangement information of the card 18 based on the camera video signal input from the camera 12, and also the symbol 22 written on the card 18.
Specify the meaning of and determine or specify the musical expression.

When the symbol 22 on the card 18 means a note, the card 18 will be particularly referred to as a note card below. When the card 18 is a note card, the note symbol 22 on the card is a tone color, pitch (pitch) of a predetermined musical instrument, which is predetermined according to the type of the symbol (pattern) that means the note. A sound that has loudness and duration. FIG. 2 shows the case where the card 18 is a note card. The operator creates a musical phrase by placing such a note card 18 on the table 14.

The operator 16 presets a coordinate system as shown in FIG. That is, the operator 16 sets, for example, one point on the table 14 facing the operator on the left side of the operator and on the side close to the operator as the origin. Further, the operator 16 sets the X axis in the horizontal direction of the operator passing through this origin in the plane of the table 14 and designates the right hand side of the operator as the positive direction of the X axis. Further, the operator 16 sets an axis that is perpendicular to the X axis and passes through the origin in the plane of the table 14 as the Y axis, and
Specify the positive direction of the Y-axis in the direction away from the operator. Further, the operator 16 takes the Z axis in the direction perpendicular to the plane of the table 14 through the origin and designates the upper direction as the positive direction.

When the operator 16 places the note card 18 on the table 14 in which such a coordinate system is preset, the computer 24 causes the marker on the card 18 to be based on the camera video signal input from the camera 12. Identify 20 and symbol 22 to know that it is a note symbol (and thus card 18 is a note card).
In addition, 2 on the coordinate system of the table 14 of the card 18
Detects dimensional information. The two-dimensional information of the card 18 is, for example, the X coordinate and the Y coordinate of the center of the card 18 in the coordinate system of FIG. 3, and an axis (not shown here) parallel to the Z axis passing through the center of the card ( Hereinafter, this axis will be referred to as the Z'axis) and is the rotation angle of the card. And X
It is given in advance that the axis represents the time axis and the larger the X coordinate, the later time. Similarly, it is given in advance that the Y-axis represents the pitch of the sound, and the larger the Y coordinate, the higher the sound. Further, it is given in advance that the rotation angle around the Z ′ axis represents the loudness of the sound, and the larger the rotation angle, the louder the sound.

Based on these pieces of information, the computer 24 produces a tone corresponding to the Y-coordinate of the detected note card with a tone color of a predetermined musical instrument previously associated with the identified note symbol 22 and having a predetermined duration. Then, the music data is converted into music data such as MIDI data that can be played at a size corresponding to the detected rotation angle about the Z ′ axis, and the music data (MIDI data) is output. As music data,
Although MIDI data is used in the embodiment, other similar music data may be used when an electronic musical instrument different from that in the embodiment is used.

Then, the computer 24 uses the MID
The I data is repeatedly output with a silent period of, for example, 2 seconds interposed. MID output from computer 24
The I data is given to the electronic musical instrument 26. Electronic musical instrument 26
In this embodiment, includes a MIDI sound source (a sound source that receives MIDI data and outputs a sound, and there are a keyboard type with a keyboard, a module type without a keyboard, a board type or a card type to be installed in a computer slot). . That is, the electronic musical instrument 26 is, for example, a synthesizer (an instrument capable of freely synthesizing timbres), a sampler (an instrument capable of playing a scale based on an actual sound sampled), a scale, a tempo, a timbre, etc. Is a musical instrument that can electronically control the musical expression elements of.
The electronic musical instrument 26, when enabled by the computer 24, has a designated musical instrument sound (timbre), a designated pitch, and a designated loudness according to the received MIDI data.
The music signal corresponding to the sound that lasts for the specified time is output. The music signal from the electronic musical instrument 26 is output as sound from the speaker 30 via the amplifier 28. Computer 2
Since the MIDI data from 4 is repeatedly output with a silent period of, for example, 2 seconds, the sound based on this is also repeated at regular intervals.

Although only one amplifier 28 and one speaker 30 are shown in FIG. 1, it is possible to use two sets of them for a stereo arrangement, and further, to arrange three speakers.
Of course, it is also possible to have surround output in groups or more.

The computer 24 further converts the camera video signal (camera video signal indicating the arrangement of the cards on the table 14) received from the camera 12 into a video signal which can be displayed by the video projector 32, and will be described later. As described above, the computer graphics image generated by the computer 24 is also superimposed on the camera image signal as needed, and is supplied to the video projector 32. The video projector 32 displays, on the screen 34, an image showing the card arrangement state on the table 14, and in some cases, an image on which CG is superimposed. The video projector 32 and the screen 34 constitute a large-screen display, and as such a large-screen display, an arbitrary one such as a cathode ray tube, a plasma display, a liquid crystal display can be used.

Next, the operator 16 holds another musical note card of the same kind as the musical note card 18, for example.
Place another note card along the axis, for example, somewhere to the right of the previous note card. At this time, the computer 24 detects the Y coordinate and the rotation angle around the Z ′ axis of the note card (the center of the note card) placed this time based on the camera image signal received from the camera 12, and plays. Determine the pitch and loudness of the sound. Further, the duration of the sound to be played corresponding to the previously placed note card is determined by the difference between the X coordinate of the note card placed this time and the X coordinate of the previously placed note card. However, if there is no succeeding note card, the duration of the sound corresponding to the note card placed this time is the (default) duration previously assigned to the note symbol. If Figure 4
As shown in, for example, if the note card of this time is placed at the same X coordinate position as the previously placed note card but at a different Y coordinate position, these two note cards cause two notes of different heights to be placed. The sounds will be played at the same time. Again, if there is no following card behind (to the right of) these cards, the duration of these two notes will default.

In this way, the operator 16 visually confirms the arrangement of the note cards arranged on the table, and at the same time, is played by the electronic musical instrument 26 on the basis of the camera image signal indicating the arrangement of the note cards. You can hear the sound repeatedly. If the operator does not like the flow of these sounds, he can change the position of any of the cards. Then, the sound to be played is also changed accordingly. Further, when the operator 16 successively arranges a plurality of note cards 18 on the table 14,
The computer 24 identifies each note card and their arrangement from the camera image signal, and plays the corresponding melody in real time. This performance is also repeated with a silent period of 2 seconds, for example. Operator 16
Is changing the position and arrangement of the note cards on the table, while listening to the playing sound of the electronic musical instrument 26 corresponding to the changed arrangement at the same time, until the sequence of sounds becomes a favorite phrase. Card placement work can be continued. An array pattern composed of such note cards is considered to have event information of a series of notes. In that sense, such an arrangement of note cards may be referred to as a note event pattern below. The card arrangement of FIG. 4 is an example of such a note event pattern.

The operator 16 can save the phrase thus created as his own work in the computer 24 and can also create a phrase card 36 meaning this phrase. An example of a phrase card corresponding to the note event pattern of FIG. 4 is shown in FIG. This phrase card 36, like the note card 18, also includes a marker 38 and a symbol 40 described in the marker 38. However, the symbol 40 described on the phrase card 36 does not necessarily have to represent the arrangement of the corresponding note cards, as in the example of FIG. 5, and the operator 16 and the computer 24 correspond to the note cards. Any symbol may be used as long as it can identify the arrangement (arrangement) of. To physically create the phrase card 36, for example, the computer 2
4 and the printer 42 (FIG. 1) connected to it,
For example, the marker 38 and the corresponding phrase symbol 40 may be printed on a paper such as a postcard. Of course, the note card 18 may also be created in advance by this method.

Once the note event pattern as shown in FIG. 4 is once recorded in the phrase card as shown in FIG. 5, the note event pattern, that is, the phrase, is fixed. The card 18 can be removed. Thus, the operator can move on to the creation of the next phrase card 36, ie the composition of the next phrase.

When some of his or her own phrases are created in this way, the operator 16 shifts from the phrase composition stage so far to the phrase operation stage. Phrase operations include intra-phrase editing, whole phrase operations, and inter-phrase operations. The following will be described in order.

First, the case of editing within a phrase will be described. The operator 16 first activates the phrase playing mode on the menu screen. Next, the operator 16 places one phrase card 36 to be operated on the table 14, and the computer 24 graphically displays the note event pattern held by the phrase card 36 on the screen 34 as a score. FIG. 6 shows an example of the musical score graphically displayed. On the screen 34, the upper surface of the table 14 and the phrase card 36 placed on the table 14 are also displayed as a video image, and this musical score graphic is displayed by being superimposed on the image of the corresponding phrase card on the table. To be done. The computer 24 also repeatedly plays this note event pattern.

Here, when the operator 16 wants to give a change to the inside of this phrase, that is, when he or she wants to perform in-phrase editing, the operator 16 activates the in-phrase editing mode from the menu screen. Then, the operator 16 brings out a card called a note change card 44 onto the table 14. As the symbol 48 in the marker 46, for example, a note and a spanner are drawn side by side on the note change card 44. An example of the note change card 48 is shown in FIG.

When the computer 24 identifies the unique symbol 48 of the note changing card 44 from the camera video signal, first, on the screen 34, the corresponding image is displayed instead of the phrase card 36 placed on the table 14. Graphically display the note event pattern (FIG. 4: Arrangement of a series of note cards). Further, the graphic of the score as shown in FIG. 6 is also displayed. Computer 24
Displays a spanner on the screen in computer graphics (hereinafter, CG) in accordance with the current position of the note change card 44. Further, this display is displayed by being superimposed on the graphic display of the note event.

Subsequently, the operator 16 changes the musical note changing card 44 to any one musical note card (hereinafter, this musical note card is referred to as a CG musical note card because it is displayed in graphics). When you bring it over, a small dot called a handle appears in the center and upper right corner of the CG note card. If the operator 16 wants to change the pitch of this note card, the position of the note change card 44 is adjusted so that the hexagonal portion of the CG-displayed wrench (hereinafter referred to as CG spanner) is changed to the CG note card. Align it with the handle in the center of and hold it there for, for example, 2 seconds (and so on). As a result, the CG spanner can grasp the center of the CG note card by a predetermined process of the computer 24. Then, the operator 16
Moves the note change card in the direction of the height axis by the required amount, and keeps it at that position for 2 seconds again. As a result, the handle is released from the CG spanner. In this way
The operator 16 can change the position of the height axis of the CG note card. The same applies when the CG note card is moved in the time axis direction.

When it is desired to change the loudness of the sound, CG
Align the hexagonal part of the wrench with the handle at the upper right corner of the CG note card, hold it for 2 seconds, and then grab the handle, then rotate the note change card 44 by the required angle, By resting for a second, the handle is released from the CG spanner. In this way, the operator 16 can rotate the CG note card by the required angle. The operator 16 runs the CG phrase definition mode from the menu screen here,
Then, when the CG phrase performance is activated, the melody or rhythm of the CG phrase just defined can be repeatedly heard as a sound for the time being. The above operation can be repeated as many times as desired until a CG phrase is created.

When the operator 16 completes the necessary modification of the note cards in the phrase in this way, the operator removes the note modification card 44 from the table 14 and then selects the phrase symbol generation and the card save mode on the menu screen. To start.

This causes the computer 24 to request the operator 16 to enter a symbol for a new phrase card representing the last note event pattern. In this case, the computer 24 may display an unused unused symbol for the phrase card on the computer screen as a candidate and let the operator 16 select it. When the operator 16 specifies the symbol for the new phrase card, the computer 24 stores the new phrase card and prints it out as a phrase card 36. Also, screen 3
Instead of the graphic display of the note event pattern, the newly defined phrase card is displayed graphically on FIG.

When the operator 16 places a new phrase card on the table 14 and activates the phrase playing mode from the menu screen, the operator 16 can hear the melody and rhythm of the new phrase card.

Next, the entire phrase operation will be described. The operations for the entire phrase are as follows.
For example, for a phrase card that graphically displays a score as shown in Fig. 6, invert (reversal on the height axis. However, here for the explanation of the principle, on the height axis within one octave. 7), the musical score as shown in FIG. 7 is displayed graphically. When reverse (reversal on the time axis) is applied to the same phrase card, a score as shown in FIG. 8 is displayed graphically. When transposing (parallel movement on the height axis) is applied to the same phrase card, the result shown in FIG.
The score as shown in is displayed graphically. The operator 16 can also provide stretching or compression on the height axis or on the time axis.

When the operator 16 places one phrase card 36 to be operated on the table 14 and activates the phrase playing mode, the computer 24 superimposes it on the video image of the phrase card on the screen 34. , The musical note event pattern held by the phrase card 36 is graphically displayed as a musical score.

Next, when the operator 16 intends to perform a phrase invert operation, the operator 16 brings out a card called the phrase change card 50 on the table 14 and then activates the phrase invert mode on the menu screen. . For example, a spanner is drawn as a marker 52 and a symbol 54 in the marker 52 on the phrase changing card 50. Phrase change card 50
An example is shown in FIG.

Next, the operator 16 holds one phrase changing card 50 on the table 14 in parallel with the table 14. Then, on the screen 34, the phrase change card 50 held by the hand on the upper surface of the table 14 is displayed as a video image in a form of being overlaid on the phrase card 36 already displayed.

At this time, the computer 24 superimposes it on the phrase changing card 50 and also displays the CG spanner. The operator 16 adjusts the position of the phrase change card 50 so that the hexagonal part of the CG spanner is located at the upper end of the phrase card 36 while watching the image including this CG spanner on the screen. The virtual CG spanner can grab the top edge of the CG phrase card by holding the phrase change card stationary for 2 seconds at that position.

Subsequently, the operator 16 linearly moves the hexagonal portion of the CG spanner to the lower end of the phrase card, and keeps it there for 2 seconds. The computer 24 uses the phrase change card 50 and the phrase card 3 at this time.
The operator 16 recognizes the change in the relative positional relationship with 6 and identifies that the operator 16 has instructed the inversion effect, and graphically displays the inverted score shown in FIG. 7. Subsequently, the computer 24 repeatedly plays the inverted phrase.

Next, when the reverse effect is applied, the operator 16 makes 1
Hold the phrase change card 50 parallel to the table 14 and hold the phrase change card 50 so that the hexagonal part of the CG spanner is at the right end of the CG phrase card while watching the image including the CG spanner on the screen. Adjust the position of. The CG spanner can grab the right edge of the CG phrase card by holding the phrase change card still for 2 seconds at that position. Next, operator 16
Moves the hexagonal part of this CG spanner linearly to the left edge of the CG phrase card and rests there for 2 seconds.
The computer 24 uses the phrase change card 50 at this time.
By recognizing the change in the relative positional relationship between the phrase card 36 and the phrase card 36, it is identified that the operator 16 has instructed the reverse effect, and the reversed score shown in FIG. 8 is displayed graphically. Then, play the reversed phrase repeatedly.

Next, when giving a transpose, the operator 16 operates the table 14 in the same manner as above.
In the above, put one phrase change card 50 on the table 14
Hold it parallel to and hold the phrase change card 50 so that the hexagonal part of the CG spanner is near the center of the CG phrase card while watching the image including the CG of the spanner on the screen. The virtual CG spanner can grab the center of the CG phrase card by keeping the phrase change card 50 stationary at that position for 2 seconds.
Subsequently, the operator 16 moves the hexagonal portion of the CG spanner by a necessary amount in a direction parallel to the height axis,
Then, let it rest for 2 seconds. The computer 24 recognizes the change in the relative positional relationship between the phrase changing card 50 and the phrase card 36 and the change in the position of the phrase changing card 50 at this time, and confirms that the operator 16 has instructed the transpose effect. Identify and graphically display the score shown in FIG. 9 transposed by the specified amount. Then, the phrase transposed by the specified amount is played repeatedly.

Next, when compression is applied in the height direction, the operator 16 holds two phrase changing cards 50 on the table 14 in parallel with the table 14 in the left and right hands, respectively. Then, two CGs of spanners appear on the screen. CG of the hexagonal part of one of these CG spanners
Align with the top of the phrase card. Align the hexagonal part of the other CG spanner with the bottom edge of the CG phrase card. Keep it still for 2 seconds. First, the CG spanner fitted to the upper end is moved by a necessary amount in a direction parallel to the height axis in a direction closer to the user, and is left there for 2 seconds. At this time, the changed phrase becomes a phrase compressed to a lower height, and when the changed score is displayed graphically, the phrase compressed in the height direction is repeatedly played. To be done. At this time, it is necessary to display two CG spanners in order to distinguish it from the above invert.

In order to compress the phrase to the high-pitched side, this time, the CG spanner aligned with the lower end of the CG phrase card may be moved in a direction away from oneself in parallel with the height axis.

If it is desired to compress the phrase from both the low side and the high side, the above operation may be performed continuously in time, but the lower CG panner is on the higher side and the upper CG is CG.
The panners may be moved to the lower side in parallel with the height axis by the required amount.

Next, the same applies to the case of expanding in the height direction, and the virtual CG spanner by two phrase change cards is displayed at the same time. However, in the case of compression, CG
Reverse the moving direction of the spanner.

Next, the same applies to the case of compressing and expanding in the time direction, and the CG spanner by the two phrase change cards 50 is also displayed at the same time. In this case, one CG spanner may be aligned with the left edge of the CG phrase card, the other CG spanner may be aligned with the right edge of the CG phrase card, and each CG spanner may be moved in parallel with the time axis by the required amount.

In any case, these effects are reflected in the score of the graphics in real time, and at the same time, the corresponding phrase is repeatedly played.

When the operator 16 completes the necessary whole phrase operation in this way, the phrase change card 5
After removing 0 from the table 14, the mode for generating the phrase card symbol and saving the card is activated from the menu screen. This causes the computer 24 to request the operator 16 to enter the symbol for the new phrase card. When the operator 16 specifies a new symbol, the computer 24 stores the new phrase card and creates a new phrase card by printing it on a paper such as a postcard with the printer 42. Further, on the screen 34, a newly defined phrase card is graphically displayed instead of the image of the phrase card before the change.

Next, the operation between phrases will be described. The phrase card created as described above can be copied to another phrase card. This copy is the only operation between phrases. Such an operation can be easily performed using the menu screen displayed on the screen of the computer 24. Of course, after this, again, it is possible to define a new phrase card by performing the above-mentioned various operations on the copied phrase card.

The above is the stage of phrase operation. At the end of the phrase operation stage, each of the phrase cards 36 (FIG. 5) completed as described above is prepared as a physical entity (for example, a card printed on a paper such as a postcard as described above). There is. The operator 16 then proceeds to the phrase playing stage. For that purpose, the operator 1
6 has at least one phrase card on the table 14
After placing it on, start the phrase playing mode on the menu screen.

In the phrase playing mode, the operator 16 places two phrase cards 3 on the table 14.
Putting 6 next to each other along the time axis, the computer 24
Play these phrase cards sequentially in time. At this time, the duration of the last sound of the first phrase card is determined by the position of the second phrase card on the time axis. The performance in this case is also repeated, for example, with a silent period of about 2 seconds interposed. The operator 16 can judge the quality of the melody and rhythm by combining these phrase cards in real time while judging whether the phrase is good or bad. It can be the card 36. Such an operation is also performed by activating a phrase card symbol generation and card saving mode on the menu screen displayed on the screen of the computer 24. When the phrase cards are placed at the same position in the time axis direction and next to each other in the height axis direction, these phrases are sounded at the same time. These phrases can be played by one musical instrument or as different tracks (parts) of different musical instruments according to an instruction from the operator 16 to the computer 24. Such an instruction is also given by activating a mode of symbol generation and card saving of the phrase card on the menu screen displayed on the screen of the computer 24.

In this way, by using this system, a larger phrase can be composed, and finally one music composition can be composed.

The operation of the computer 24 in the FIG. 1 embodiment outlined above will now be described in detail by reference to the respective flow diagrams.

FIG. 12 shows each mode activated from the menu screen, and after displaying the menu screen, the operator requests the desired mode by, for example, clicking the mouse. However, it operates in the previous mode unless the next request is input. Then, when a new request is input, the previous mode is reset and the new mode is set.

The "coordinate system generation" routine shown in FIG. 12 is executed in the coordinate system (X,
Y, Z) is a mode for setting or forming
Detailed explanation is omitted. Similarly, detailed description of the creation and storage of the note card 18, the creation and storage of the phrase card 36, and the definition of the CG phrase will be omitted.

Details of the phrase composition mode are shown in FIG. In the first step S1 of FIG. 13, the computer 24 (FIG. 1) captures the camera video signal from the camera 14. Specifically, the camera video signal is digitally converted and stored in the computer 24 as color video data. Then, the computer 24 processes the color image data and sets the previously set coordinate system in the table 1.
4 is placed on the top image.

Then, in the next step S2, the computer 24 makes the two-dimensional coordinate system X1, Y1 and X2.
Y2 is respectively X1 = 0, Y1 = 0, X2 = ΔX, Y
Set as 2 = ΔY. However, ΔX and ΔY are
Each indicates the amount of increment when changing the range, and for example, a value about the width operator height of each note card 18 is used. At the next step S3, the computer 24 determines whether the musical note card 18 as shown in FIG. 2 is placed within the range of the X axis (X = X1 to X2) and within the range of the Y axis (Y = Y1 to Y2). Make a decision.

If "YES" is determined in the step S3, in the next step S4, the computer 24
MIDI data is created according to the position and / or arrangement information of the note card 18 and set or added to the output data area. That is, the tone color (type of musical instrument) is set by identifying the symbol 22 on the note card 18, the X coordinate is detected, and the duration is set.
The coordinates are detected to set the pitch (pitch), and the rotation angle around the Z'axis is detected to set the size. However, for the duration, as described above, the difference from the immediately preceding measured value is set as the duration of the immediately preceding note, but if there is no immediately preceding note, silent is selected as the immediately preceding note and the X coordinate is set. After setting the silent duration according to the above, this duration will be the default.

After determining "NO" in the previous step S3 or after the step S4, the computer 24
In step S5, the Y coordinate position is updated (Y1 +
ΔY → Y1, Y2 + ΔY → Y2). And step S
At 6, the computer 24 determines whether the coordinate position Y2 is within the (predetermined) range of the entire table, for example.

If Y2 is within the predetermined range, "YES" is determined in the step S6, and the process proceeds to the next step S7. In step S7, the computer 24 is within the range of the X axis (X = X1 to X2) and within the range of the Y axis (Y = Y1 to Y1).
It is determined whether the note card 18 is placed in Y2). If "NO", the process returns to the previous step S5. "YES"
Then, similar to step S4, MIDI data is created according to the position and / or arrangement information of the note card 18 and set or added to the output data area.

However, if "NO" is determined in the step S6, the computer 24 proceeds to the following step S9.
, The X coordinate position is updated (X1 + ΔX → X1,
X2 + ΔX → X2). Then, in step S10, the computer 24 determines whether the coordinate position X2 is within a (predetermined) range of the entire table, for example. "YE
If S ", the process returns to step S3, where" N "
If it is "O", the computer 24 proceeds to step S11 to set or add silent MIDI data for 2 seconds to the output data area.

In this way, steps S5 and S9
In step S4, S8 or S11, the computer 24 executes MIDI in the phrase composition mode by updating the coordinates in accordance with the flow chart shown in FIG.
Set the data in the output data area, step S12
To output those MIDI data, and the electronic musical instrument 2
6 (FIG. 1). The MIDI data is shown in Fig. 1.
It is sent to the electronic musical instrument 26 every cycle of the routine of No. 3, but the electronic (MIDI) musical instrument 26 accumulates a fixed number of MIDI data and then discards the overflowed MIDI data.

Details of the phrase playing mode are shown in FIG. In the first step S21 of FIG. 14, the computer 24 takes in the camera video signal from the camera 14 in the same manner as in the previous step S1. Then, in the next step S22, the computer 24 determines the two-dimensional coordinates X1, Y1 and X2, Y as in the previous step S2.
Set 2 respectively. In the next step S23,
The computer 24 is within the range of the X axis (X = X1 to X2)
And it is determined whether or not the phrase card 36 as shown in FIG. 5 is placed within the range of the Y axis (Y = Y1 to Y2).

If "YES" is determined in the step S23, in the next step S24, the computer 24
Analyzes the note event pattern shown on the phrase card 36 and displays the note event pattern as a graphic on the screen 34 (FIG. 1). However, the CG display is displayed by being superimposed on the actual image of the phrase card by the camera image signal. At the same time, in step S25, the computer 24 sets the MIDI corresponding to the note event included in the note event pattern in the output data area.

After determining "NO" in the previous step S23 or after the step S25, the computer 24
Updates the Y coordinate position in step S26 as in step S5 of FIG. And step S2
At 7, the computer 24 determines whether the coordinate position Y2 is within a predetermined range.

If Y2 is within the predetermined range, "YES" is determined in the step S27, and in the next step S28, the computer 24 is in the range of the X axis (X = X1 to X).
2) and whether or not the phrase card 36 is placed within the range of the Y axis (Y = Y1 to Y2). "NO"
Then, the process returns to the previous step S26. If “YES”, the same as step S24 and S25, the step S
At 29 and S30, the note event pattern shown on the phrase card 36 is displayed graphically on the screen 34, and the MIDI corresponding to the note event included in the note event pattern is set in the output data area.

However, when "NO" is determined in the step S27, the computer 24 proceeds to the subsequent step S27.
In step 31, the X coordinate position is updated as in step S9 of FIG. Then, in step S32, the computer 24 determines whether the coordinate position X2 is within a predetermined range. If "YES", the process returns to step S23, and if "NO", the process proceeds to step S33, where the computer 24 adds 2 seconds of silent MIDI data to the output data area.

In this way, steps S26 and S
By updating the coordinates at 31, the "phrase play" mode is executed according to the flow chart shown in FIG. The computer 24 executes steps S25, S30 or S33.
Sets the MIDI data in the output data area, and outputs the MIDI data in step S34,
It is given to the electronic musical instrument 26 (FIG. 1).

The "CG phrase playing" mode shown in FIG. 15 is executed to listen to a melody before the phrase card is changed by the phrase editing mode or the entire phrase operation and saved. First step S51
Then, the computer 24 determines whether the CG phrase card is already designated on the screen 34.
If “YES”, in step S52, the computer 24 CG-displays the note event pattern included in the CG phrase card, and in step S53.
Then, the MIDI data corresponding to the note event is set in the output data area, and the MIDI data is output to the electronic musical instrument 26 in step S55. Therefore, this FIG.
By executing the flow chart of FIG. 5, when the phrase card is changed, the change can be confirmed as a sound each time.

The in-phrase edit mode using the note change card 44 (FIG. 10) is shown in detail in FIGS. 16-18. In the first step S61 of the in-phrase edit mode, the camera video signal from the camera 12 is captured in the same manner as in step S1 of FIG. And computer 2
In the next step S62, the CPU 4 determines whether or not one phrase card 36 is placed on the table 14 based on the camera video signal. Similarly, based on the camera image signal, it is determined in step S63 whether the note change card 44 is placed on the table. Step S62
And / or if "NO" is determined in S63, the phrase cannot be edited. Therefore, after the warning is displayed, the process returns to the "requested" check in FIG. 12, and the process is started by determining the request again. Restart.

In both steps S62 and S63, "Y"
If "ES" is determined, in the next step S64,
The computer 24 determines whether or not there is only one note change card on the table 14. Two or more note change cards cannot be used at the same time, so if there are two or more note change cards on the table at the same time, after a warning, return to the "Requested" check in Fig. 12 and request again. Restart the process that starts by re-determining.

If "YES" is determined in step S64, that is, if one phrase card 36 and one note change card 44 are simultaneously present on the table 14, the computer 24 continues. In step S65, the note event pattern of the phrase card is displayed in CG, and the score corresponding to the note event pattern as shown in FIG. 6, for example, is displayed in CG. Then, in step S66, the computer 24 detects the position X, Y on the coordinate of the note change card, and C at the position where the note change card exists.
Display the G spanner. In a succeeding step S67, the computer 24 determines whether or not the CG spanner is displayed overlaid on the CG note card.

If "NO" is determined in the step S67, the process returns to the step S65, but if "YES", in the next step S68, the computer 24 is operated.
Displays the handle on the CG note card, and step S6
At 9, it is determined whether the handle is sandwiched by the hexagonal parts of the CG spanner. If "NO", step S
Return to 68. If "YES", in the next step S70, the computer 24 determines whether the handle on the CG note card is the center of the CG note card. "NO"
Then, in step S71, it is determined whether the handle on the CG note card is the upper right corner of the CG note card. If "NO" in the step S71, the process returns to the step S68.

If "YES" is determined in the step S70, the operator 16 wants to change the pitch (pitch) of the note card, so that the process proceeds to the step S72 of FIG. Then, in step S72, the computer 24 initializes the counter i to 0. This step S72 to step S78 is a routine for determining whether or not the hexagonal portion of the CG spanner has been stationary for 10 consecutive 0.2 seconds, and the counter i counts the 10 times. To use. After step S72, step S7
3, the computer 24 measures the Y position Y (i) of the hexagonal part of the CG spanner when the counter value is i. Specifically, the Y position Y (i) of the note change card 44
Is measured (the same applies hereinafter). Next, in step S74, the computer 24 waits for 0.2 seconds. After 0.2 seconds, in step S75,
The computer 24 increments the counter i by "1". Next, in step S76, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner at the new count value. Then, in step S77, the current measurement value Y (i) and the previous measurement value Y
It is checked whether or not the absolute value of the difference from (i-1) is equal to or smaller than a small value "Δ1" that can be regarded as being almost stationary. If "NO", the CG spanner is operator 1
According to 6, it is determined that the robot is being moved, the process returns to step S72, the counter i indicating the number of continuous stationary state detections is initialized to 0, and the above determination is repeated again.

If "YES" in the step S77, it is determined that the computer 24 has been stationary for 0.2 seconds, and the computer 24 determines whether the counter i reaches "10" in a next step S78. Find out. If "N
If it is "O", the process returns to step S74, and the operation for checking the next 0.2 second is started.

If "YES" in the step S78, it is determined that the hexagonal portion of the CG spanner is stationary for 0.2 times continuously 10 times, that is, a total of 2 seconds, and the process proceeds to the next step S79. . In step S79, the computer 24 measures the Y position of the hexagonal part of the CG spanner at that time, and displays the Y position of the CG note card in accordance therewith.

This step S79 to step S86
Determines whether or not there was a second rest for 2 seconds by checking if the rest for 0.2 seconds continued 10 times again, and if it was determined that there was no rest for 0.2 seconds. , A routine for displaying a CG note card in accordance with the Y position of the CG spanner measured every 0.2 seconds. With this routine, the note change card 44
The display position of the CG note card moves along the Y coordinate in accordance with the movement of the CG note card. Therefore, the operator 16 moves the CG note card by holding the center handle of the CG note card with a CG spanner. It is possible to obtain such an operation feeling.

Returning to the explanation of the operation subsequent to step S79, in the next step S80, the computer 24
Initializes the counter i to 0. Subsequently, in step S81, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner when the counter value is i. Next, in step S82, the computer 24
Waits for 0.2 seconds. After 0.2 seconds has passed, in step S83, the computer 24 increments the counter i by "1". Next, step S84.
At 24, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner at the new count value. Then, in step S85, the current measured value Y
The absolute value of the difference between (i) and the previous measured value Y (i-1) is
It is checked whether or not the value is smaller than a small value “Δ1” that can be regarded as almost stationary.

If "NO", the CG spanner judges that it is being moved by the operator 16, returns to step S79, and the computer 24 makes the Y of the hexagonal portion of the CG spanner at that time. The position is measured, the Y position of the moving CG note card is displayed in accordance therewith, and subsequently, in step S80, a counter i indicating the number of continuous stationary state detections is initialized to 0, and the above determination is repeated again.

If "YES" in the step S85, it is determined that the vehicle has been stationary for 0.2 seconds this time, and it is checked in the next step S86 whether the counter i has reached "10". . If “NO”, step S8
Returning to step 2, the operation for checking the next 0.2 seconds is started.
If "YES" in step S78, 0.2
It is determined that the hexagonal portion of the CG spanner is stationary for 10 consecutive times per second, that is, for a total of 2 seconds, and the next step S8
Proceed to 6-1. In step S86-1, the computer 24 measures the Y position of the hexagonal portion of the CG spanner at that time, displays the Y position of the CG note card in accordance therewith, and then checks "requested" in FIG. return,
Restart the process that starts by re-determining the request.

If "YES" is determined in the step S71, the operator 16 wants to change the volume of the note event, so the process proceeds to the step S87 in FIG. Then, in step S87, the computer 24
The counter i is initialized to 0. From step S87 to step S93, the hexagonal part of the CG spanner is 0.2
This is a routine for determining whether or not the vehicle has been substantially stationary for 10 consecutive times per second, and the counter i is used for counting the 10 times. After step S87, step S
At 88, the computer 24 causes the X, Y position X (i) of the hexagonal portion of the CG spanner when the counter value is i,
Measure Y (i). Next, in step S89, the computer 24 waits for 0.2 seconds. If 0.2 seconds has passed, in step S90, the computer 24
Counts up the counter i by "1". Next, in step S91, the computer 24 measures the X, Y positions X (i), Y (i) of the hexagonal portion of the CG spanner at the new count value. And step S
At 92, the current measured value X (i) and the previous measured value X (i
The absolute value of the difference between the measured value Y (i) and the previous measured value Y (i-) is smaller than the small value "Δ2".
It is also checked whether the absolute value of the difference from 1) is less than the small value “Δ3”.

If "NO", the CG spanner judges by the operator 16 that the operator is moving and returns to step S87 to initialize the counter i indicating the number of continuous stationary state detections to 0. Then, the above judgment is repeated again.

If "YES" in the step S92, it is determined that the vehicle has been stationary for 0.2 seconds this time, and it is checked in a next step S93 whether the counter i has reached "10". . If “NO”, step S8
Returning to step 9, the operation for checking the next 0.2 second is started.
If “YES” in the step S92, it is determined that the hexagonal part of the CG spanner is stationary for 10 consecutive times every 0.2 seconds, that is, a total of 2 seconds, and the next step S94.
Proceed to. In step S94, the computer 24 measures the X and Y positions of the hexagonal portion of the CG spanner at that time, calculates the rotation angle of the hexagonal portion of the CG spanner, and further uses the CG musical note card to generate the CG musical note. The card is displayed by rotating it around the Z'axis by the rotation angle.

From step S94 to step S101
Determines whether or not there was a second rest for 2 seconds by checking if the rest for 0.2 seconds continued 10 times again, and if it was determined that there was no rest for 0.2 seconds. , CG according to the rotation angle calculated from the X and Y positions of the CG spanner measured every 0.2 seconds
This is a routine for displaying a note card. This routine causes the display of the CG note card to rotate around the Z'axis in accordance with the movement of the note change card, so that the operator 16 grasps the handle at the upper right corner of the CG note card with a CG spanner. It is possible to get the feeling of rotating a musical note card.

Returning to the explanation of the operation subsequent to step S94, in the next step S95, the computer 24
Initializes the counter i to 0. Subsequently, in step S96, the computer 24 causes the computer 24 to determine the X, Y position X (i), Y of the hexagonal portion of the CG spanner when the counter value is i.
Measure (i). Next, in step S97, the computer 24 waits for 0.2 seconds. If 0.2 seconds have elapsed, the computer 24 increments the counter i by "1" in step S98. Next, in step S99, the computer 24, at the new count value, the X, Y position X (i), of the hexagonal portion of the CG spanner.
Measure Y (i). Then, in step S100,
The absolute value of the difference between the current measurement value X (i) and the previous measurement value X (i) is less than a small value “Δ2”, and the current measurement value Y (i) and the previous measurement value Y ( It is also checked whether the absolute value of the difference from i-1) is less than or equal to the small value “Δ3”. If "NO", the CG spanner is operator 1
6, it is determined that the CG spanner is being moved, the process returns to step S94, and the computer 24 measures the X and Y positions of the hexagonal portion of the (moving) CG spanner at that time, and then the CG The rotation angle of the hexagonal part of the spanner is calculated, and the CG note card is displayed by rotating the CG note card around the Z ′ axis by the rotation angle. Continuing,
In step S95, a counter i indicating the number of continuous stationary state detections is initialized to 0, and the above determination is repeated again.

If "YES" in the step S100, it is determined that the vehicle has been stationary for 0.2 seconds at this time,
In the next step S101, the computer 24
Counter i checks whether it has reached "10". If “NO”, the process returns to step S97 and the next 0.2
Enter the operation to check for seconds. If step S101
If “YES”, it is determined that the hexagonal part of the CG spanner has been stationary for 0.2 times continuously for 10 times, that is, for a total of 2 seconds, and the process proceeds to the next step S101-1. In step S101-1, the computer 24 measures the X and Y positions of the hexagonal portion of the CG spanner at that time,
Calculate the rotation angle of the hexagonal part of the CG spanner, and then
After the note card is displayed by rotating it about the Z'axis of the CG note card by that rotation angle, the process returns to the "requested" check in Fig. 12, and the process starts again by re-determining the request. To start.

The phrase invert mode, which is an example of the entire phrase operation using the phrase change card 50 (FIG. 11), is shown in detail in FIGS. As described above, the phrase entire operation includes phrase reverse, phrase reverse, and phrase transpose. Here, the phrase invert mode will be described.

At the first step S111 in the phrase invert mode, as in step S1 of FIG.
A camera video signal from the camera 14 is captured. And
In the next step S112, the computer 24 determines whether or not one phrase card 36 is placed on the table 14 based on the camera video signal. Similarly, based on the camera image signal, it is determined in step S113 whether the phrase change card 50 is placed on the table. When "NO" is determined in step S112 and / or S113, the phrase inversion cannot be performed. Therefore, after the warning is displayed, the process returns to the "requested" check in FIG. 12 and starts again from the request determination. To restart.

When "YES" is determined in both steps S112 and S113, in the next step S114, the computer 24 determines whether or not there is only one phrase change card on the table 14. In the phrase invert mode, two or more phrase change cards cannot be used at the same time, so if two or more phrase change cards are on the table at the same time, after a warning, “Requested” in FIG. 12 is displayed. Return to check
Restart the process that starts by re-determining the request.

When "YES" is determined also in step S114, that is, one phrase card 36 and one phrase change card 50 are placed on the table 14.
When and exist at the same time, the computer 24 CG-displays the musical score corresponding to the note event pattern of the phrase card in the subsequent step S115. Then, in step S116, the computer 24
The position X, Y on the coordinate of the phrase change card is detected, and CG is detected at the position where the phrase change card exists.
Display spanner. In the following step S117, the computer 24 determines whether or not the hexagonal part of the CG spanner is at the upper end of the CG phrase card.

If "NO" is determined in the step S117, the process returns to the steps S116 and S117 to detect the X and Y positions of the phrase change card again and display the CG spanner in accordance with the position. ,
The phrase changing card is repeated until it comes to the upper end of the phrase card. If “YES” in the step S111, the process proceeds to the next step S118.

Then, in step S118, the computer 24 initializes the counter i to 0. This step S118 to step S124 is a routine for determining whether or not the hexagonal portion of the CG spanner has been substantially stationary for 0.2 consecutive times every 0.2 seconds, and the counter i counts the 10 times. To use.

After step S118, step S119
At 24, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner when the counter value is i. Next, in step S120, the computer 2
4 waits for 0.2 seconds. If 0.2 seconds has elapsed, in step S121, the computer 24 sets the counter i
Is incremented by "1". Next, step S1
At 22, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner at the new count value.

Then, in step S123, the absolute value of the difference between the measured value Y (i) of this time and the measured value Y (i-1) of the previous time is smaller than a small value "Δ4" which can be regarded as being almost stationary. To see if it was. If “NO”
If so, the CG spanner determines that it is being moved by the operator 16, and returns to step S118,
A counter i indicating the number of continuous stationary state detections is initialized to 0, and the above determination is repeated.

If "YES" in the step S123, it is determined that the vehicle has been stationary for 0.2 seconds at this time,
In the next step S124, it is checked whether or not the counter i has reached 10. If “NO”, step S1
Returning to step 20, the operation for checking the next 0.2 second is started.

If "YES" in the step S124, it is determined that the hexagonal part of the CG spanner is stationary for 0.2 times continuously 10 times, that is, a total of 2 seconds, and the following step S124-1. Proceed to. Step S124-1
Then, the computer 24 uses 6 of the CG spanner at that time.
The Y position of the rectangular portion is measured, and then step S125.
Then, it is determined whether the hexagonal part of the CG spanner is located at the lower end of the phrase card based on the measured value. If "NO" is determined, the process returns to step S124-1, the Y position of the hexagonal portion of the CG spanner at that time is measured again, and in the subsequent step S125, the hexagonal portion of the CG spanner is measured based on the measured value. The operation of determining whether is located at the lower end of the phrase card is repeated. However, if "YES" is determined in the step S125, the computer 24 proceeds to the next step S126.

Steps S126 to S132
Is a routine for determining whether or not there is a second rest for 2 seconds by checking again whether the rest for 0.2 seconds continues 10 times.

Returning to the description of the operation from step S126, in step S126, the computer 24
The counter i is initialized to 0. Then, step S1
At 27, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner when the counter value is i. Next, in step S128, the computer 24
Waits for 0.2 seconds. If 0.2 seconds have elapsed, in step S129, the computer 24 increments the counter i by "1". Next in step S130
At 24, the computer 24 measures the Y position Y (i) of the hexagonal portion of the CG spanner at the new count value. Then, in step S131, the current measured value Y
The absolute value of the difference between (i) and the previous measured value Y (i-1) is
It is checked whether or not the value is smaller than a small value "Δ4" that can be regarded as almost stationary.

If "NO", the CG spanner judges that the operator 16 is in the middle of moving, returns to step S126, initializes the counter i to 0, and repeats the above judgment again. In step S131,
If "YES", it is determined that the vehicle has been stationary for 0.2 seconds this time, and it is checked in the next step S132 whether the counter i has reached "10". If “NO”
If so, the process returns to step S128 to start the operation for checking the next 0.2 second. If “YES” in the step S132, it is determined that the hexagonal part of the CG spanner is stationary for 0.2 times continuously 10 times, that is, a total of 2 seconds, and the process proceeds to the next step S133 (FIG. 21). move on.

In step S133, the computer 24
Displays the graphics (FIG. 7) converted in this way, and displays CG in step S134.
Define a phrase card, and in step S135,
After requesting the CG phrase playing shown in FIG. 15 by designating the CG phrase card immediately above, the process returns to the check of “requested” in FIG. 12 and restarts the process which starts from re-determining the request. .

In the above-described first embodiment, when a note card or phrase card is to be changed, a substantial note change card or phrase change card is prepared, and the note card is operated by operating those change cards. Although various changes were made to the phrase cards and phrase cards, the operator 16 more directly picked a part of the note cards and phrase cards with the fingers of the hand without using the note change cards and phrase change cards. Various operations may be performed.

For example, when giving an invert to the phrase card 36, the operator 16 uses the table 1
On 4, pick the top edge of the phrase card you are targeting,
Then, by lifting the picked hand vertically upward, the upper end of the card is lifted vertically upward, and then the hand picked the upper end of the card is brought to the front side of the card while the card is faced down. As a result, the phrase card is placed upside down in its original position on the table 14 in an inverted state. The computer 24 recognizes the positional relationship between the finger and the phrase card that has detected this series of operations from the camera video signal, how the marker 20 is deformed, and how the back of the card is displayed. Is displayed, and the inverted score (FIG. 7) is displayed graphically. At the same time, repeat the inverted phrase. Reverse and transpose can be realized in almost the same way.

By doing so, there is a merit that it becomes possible to perform an operation which is more routine and easier to understand than the previous embodiment.

Further, in the case of giving an effect to a note card or a phrase card, unlike any of the above-mentioned embodiments,
It is also possible to make the computer 24 recognize the finger of the operator and give the finger the role of the note change card or the phrase change card in the previous embodiment. For example, the thumb can correspond to the note change card and the index finger can correspond to the phrase change card. In this case, the virtual spanner is displayed as a CG by being superimposed on the image of the operator's finger. Therefore, the operator does not need to operate the change card and can use his / her finger as a spanner, which is advantageous in that a more direct and easy operation can be performed.

Furthermore, in each of the above-described embodiments, a substantial table and various actual cards printed on paper such as postcards (note cards, phrase cards, note change cards, phrase change cards). Was used, but without using such a substance,
A virtual table and card displayed as G may be used. In this case, the CG of the virtual table is displayed on the computer screen or screen 34, and the operator 16 operates the menu screen or the like to create various necessary cards as the CG,
While viewing the CG image on the screen, these virtual cards can be grabbed and placed on the virtual table by using, for example, a mouse. In this way, the composition can be performed in the same way as the actual card is handled in the previous embodiment. In this case, since the camera image signal is not used to acquire the two-dimensional information (two-dimensional coordinates or rotation angle), the computer 24 refers to the contents of the display memory (not shown) to do so. Obtains two-dimensional information.

Then, at the stage of phrase composition, that is, at the stage of determining the note event pattern, a CG for displaying the state of notes corresponding to each note card in a more understandable manner is displayed on the note card on the screen 34. You may make it superimpose and display. For example, as such a CG, a toy snake is displayed, the color of the snake is determined according to the tone color of the musical instrument, and the brightness of the color is changed according to the loudness of the sound, and the duration of the sound is further shortened. It is possible to think of a display that changes the length of the snake according to. By doing this, rather than recognizing the state of the sound from the position, rotation angle and arrangement of the note card,
The advantage is that it can be recognized more intuitively and realistically.

Further, in place of the camera 12, the video projector 32 and the screen 34 in the embodiment of FIG. 1, the operator 16 mounts a head mounted display (HMD: not shown) on the head and the HM thereof.
You may make it attach a small camera integrally with D.
In this way, it is possible to construct a smaller device and to construct a music composition support system in which the operator 16 can concentrate more easily.

Further, by storing the note event pattern data of the phrase card in a memory (not shown) of the computer 24 and making it possible for a plurality of people to access the data, for example, via the Internet, Multiple people can work together to perform composition activities.

[Brief description of drawings]

FIG. 1 is an illustrative view showing one embodiment of the present invention.

FIG. 2 is an illustrative view showing a card (note card) which is an example of a hand-held object used in the embodiment in FIG. 1;

FIG. 3 is an illustrative view showing one example of two-dimensional information of a note card on three-dimensional coordinates in the embodiment of FIG.

FIG. 4 is an illustrative view showing one example of a state of arrangement of note event patterns on a table in the embodiment of FIG. 1;

5 is an illustration showing an example of a phrase card used in the embodiment in FIG. 1. FIG.

FIG. 6 is an illustrative view showing one example of a phrase created in the embodiment in FIG. 1;

FIG. 7 is an illustrative view showing one example of editing the phrase of FIG.

FIG. 8 is an illustrative view showing another example of editing the phrase of FIG. 6;

FIG. 9 is an illustrative view showing still another example of editing the phrase of FIG. 6.

FIG. 10 is an illustration showing an example of a note change card used in the embodiment in FIG. 1;

FIG. 11 is an illustration showing an example of a phrase change card used in the embodiment in FIG. 1;

FIG. 12 is a flowchart showing the overall operation of the embodiment shown in FIG.

FIG. 13 is a flowchart showing a phrase composition operation in the embodiment of FIG.

FIG. 14 is a flowchart showing a phrase playing operation in the embodiment of FIG.

FIG. 15 is a flowchart showing a CG phrase playing operation in the embodiment of FIG. 1;

FIG. 16 is a flowchart showing a part of an in-phrase editing operation in the embodiment in FIG. 1;

FIG. 17 is a flowchart showing another part of the intra-phrase editing operation.

FIG. 18 is a flowchart showing another portion of the intra-phrase editing operation.

19 is a flowchart showing a part of the phrase invert operation in the embodiment in FIG.

FIG. 20 is a flowchart showing another portion of the phrase invert operation.

FIG. 21 is a flowchart showing another portion of the phrase invert operation.

[Explanation of symbols]

10… Composition support system 12 ... Camera 18 ... Note card 24 ... Computer 36… Phrase card 44 ... Note change card 50… Phrase change card

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] May 21, 2002 (2002.5.2)
1)

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0062

[Correction method] Change

[Correction content]

Then, in the next step S2, the computer 24 makes the two-dimensional coordinate system X1, Y1 and X2.
Y2 is respectively X1 = 0, Y1 = 0, X2 = ΔX, Y
Set as 2 = ΔY. However, ΔX and ΔY are
Each indicates the amount of increment when changing the range, and for example, a value about the width and height of each note card 18 is used. In the next step S3, the computer 24
Is within the range of the X axis (X = X1 to X2) and within the range of the Y axis (Y = Y1 to Y2), as shown in FIG.
Determine if 8 is placed.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0096

[Correction method] Change

[Correction content]

When "NO" is determined in the step S117, the process returns to the steps S116 and S117 to detect the X and Y positions of the phrase change card again and display the CG spanner in accordance with the position. ,
Repeat until the phrase change card reaches the top of the phrase card. If "YES" in step S117 ,
Then, the process proceeds to next step S118.

Claims (18)

[Claims] 1. A step of: (a) obtaining at least two-dimensional information of a note card placed on a table; (b) a step of creating a note event based on the two-dimensional information; and (c) the note event. According to the method, the steps (a) to (c) are repeated to generate a series of note events, and (d) the series of note events are stored as a phrase. , Music composition support method. 2. The composition support method according to claim 1, wherein the two-dimensional information includes two-dimensional coordinates. 3. The composition support method according to claim 1, wherein the two-dimensional information includes a rotation angle. 4. The composition support method according to claim 1, further comprising the step of (e) visually displaying the position of the note card on the table prior to the step (d). 5. The composition support method according to claim 1, further comprising the step of: (f) creating a phrase card for each phrase. 6. (g) When the phrase card is placed on the table, a musical expression according to a series of note events included in the phrase card is pronounced, and the series of note events is visually displayed. The composition support method according to claim 5, further comprising a step of performing. 7. (h) a note change element is displayed together with the note event displayed in step (g), and (i) the note change element acts on the displayed note event in a predetermined manner. 7. The composition support method according to claim 6, wherein the musical expression of the note event is changed, and (j) a new phrase card is stored. 8. The method according to claim 5, further comprising the step of: (k) visually displaying a musical score according to a series of note events included in the phrase card when the phrase card is placed on the table. The composition support method described in any of the above. 9. A composition assisting method, wherein a series of music data is created in accordance with at least positions of a plurality of note cards arranged on a table. 10. A two-dimensional information acquisition means for acquiring at least two-dimensional information of a note card placed on a table, a note event creation means for creating a note event based on the two-dimensional information, and according to the note event. A composition support system comprising a sounding means for producing a musical expression and a phrase storing means for storing a series of note events created by the note event creating means as phrases. 11. The composition support system according to claim 10, wherein said two-dimensional information acquisition means includes a camera for photographing said table and said note card and outputting a camera video signal. 12. The composition support system according to claim 10, further comprising display means for visually displaying the position of the note card on the table. 13. A phrase card creating means for creating a phrase card for each of the phrases is further provided.
The composition support system described in 2. 14. The sounding means produces a musical expression according to a series of note events included in the phrase card when the phrase card is placed on the table, and the display means displays the series of note events. The composition support system according to claim 13, wherein the composition is visually displayed. 15. A change element display means for displaying a change element together with the note event by the display means, and a musical expression of the note event when the change element acts on the displayed note event in a predetermined manner. 15. The composition support method according to claim 14, further comprising changing means for changing the. 16. The musical score display means for visually displaying a musical score according to a series of musical note events included in the phrase card when the phrase card is placed on the table, further comprising: score display means. The composition support system described in Crab. 17. The composition support system according to claim 10, wherein the two-dimensional information includes two-dimensional coordinates. 18. The composition support system according to claim 10, wherein the two-dimensional information includes a rotation angle.