JP3528051B2 - Performance operation display device and recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for displaying movements of a keyboard or the like, which is a target for performance operations, when performing operations in accordance with the progress of a performance.

[0002]

2. Description of the Related Art Nowadays, many keyboard musical instruments are sequentially informed of the keys to be pressed (operated) according to the progress of the performance. The function is installed. However, many users who use the navigation function are technically inexperienced, and even if they can know the key to be pressed next by the navigation function, they do not know which finger should press the key. Is the actual situation. For this reason, keyboard musical instruments equipped with a device (playing action display device) for displaying a finger to be used for pressing a key have been commercialized. By using the performance operation display device, it is possible to perform a performance in consideration of how to carry the finger, and therefore, the user can practice (learn) more efficiently.

Most of the performance operation display devices simply display the finger to be used for the next key depression (operation). But,
The way to carry a finger to a key to be pressed (operated) is not only to carry the finger to the key. It may be carried by passing under another finger, or by straddling over another finger. This means that when only a finger is displayed, the user may not immediately understand how to carry the finger and press (operate) the key. Therefore, as disclosed in, for example, Japanese Unexamined Patent Publication No. 10-39739, a performance operation display device has been devised which displays not only the finger to be used but also how to carry the finger in a more understandable form.

In the conventional performance operation display device disclosed in Japanese Unexamined Patent Publication No. 10-39739, it is possible to determine which finger should be used for key pressing by displaying the finger (the part from the wrist) when the key is pressed. To let the user know what form to use. However, if the next key to be pressed is separated from the key currently being pressed, it is usually necessary to move the hand or finger largely. This means that even if the finger is displayed when the key is pressed, the user cannot always immediately understand how the shape of the displayed finger is changed. Therefore, the conventional performance operation display device has a problem that it is difficult to understand the overall motion of the fingers.

In the above conventional performance operation display device,
Computer graphics (hereinafter abbreviated as CG)
Using technology, we are moving the finger used to press the keys. The user can know the finger to be used for key depression by the movement of the finger. However, the movement of the finger is the movement after forming the shape of the finger when pressing the key. For this reason, it is not possible to inform the user how to move the finger shape when the key is pressed by the movement of the finger. Further, in the actual movement of the fingers during the performance, the timing of starting the key pressing operation differs depending on the position and the state of the finger pressed just before that. In other words, if the previous key pressing position is far from the next key pressing position, or if the state of the fingers is very different from the state of the fingers when the previous key was pressed, the timing to start the key pressing operation should be advanced. I am doing this. The timing of the start of such finger movement cannot be recognized by the user with the conventional performance operation display device that merely displays the state of the finger after key depression.

An object of the present invention is to provide a technique for displaying the movements of fingers during a performance in a form that the user can easily understand.

[0007]

According to one embodiment of the present invention, a performance operation display device includes event data representing the contents of an event to be generated on a performance operator group, and a time indicating the occurrence timing of the event. Music and fingering data acquisition means for acquiring tone and fingering data composed of data and finger data for designating a finger to be used to generate the event, and acquired by the tone and fingering data acquisition means Based on the musical tone and fingering data, the operation of the performance operators in the performance operators group
When the work is completed or when the release of the operation is completed
An operation control data generating means for generating an operation control data representing the state of the finger, the operation of the performance operators to be represented by the operation control data that is generated by the operation control data generating means
The operation start timing that should start the transition to the finger state when the operation is completed is required to complete the operation of the performance operator.
An operation timing determining means for determining based on a predetermined reference time representing time, an operation control data from the operation control data generating means, and an operation opening from the operation timing determining means.
An operation display unit for displaying a moving image of the operation of the finger operating the performance operator group based on the start timing.

A performance operation according to another aspect of the present invention.
The display is an event that should occur on the performance control group.
Data indicating the contents of the event, the event occurrence timing
Used to generate the event and time data
Musical tone / fingering composed of finger data that specifies the finger to be used
Tones and fingering data acquisition means to acquire data
Tones and fingering data acquired by the fingering data acquisition means
Complete the operation of the performance controls in the performance control group.
State of the finger when it is released or when the release of the operation is completed
Motion control data generation for generating motion control data
Means and the operation generated by the operation control data generating means
Completed the operation of the performance operator indicated by the control data
Timing to start the transition to the finger state
The same finger as the finger used when operating the performance operator.
Time data in the musical tone / fingering data immediately before
And the time required to complete the operation of the performance controller.
Completed releasing the first reference time and the operation of the performance control
Based on a second reference time that represents the time required to do
Operation timing determination means and operation control data
Operation control data from operation means and operation timing determination
Based on the operation start timing from the means, the performance operator
Action display means for displaying a movie of the actions of fingers operating the group
And .

[0009]

A recording medium according to an aspect of the present invention is a performance medium.
Events that should be generated on the performance control group that is the target of the operation.
Event data indicating the contents of the event, occurrence of the event
For time data indicating timing and occurrence of the event
A musical sound composed of finger data that specifies the finger to be used.
The acquisition procedure for acquiring fingering data and the acquisition procedure
Based on the recorded musical tone and fingering data.
When the operation of the playing operator is completed, or when the operation is canceled.
Generates motion control data that represents the state of the finger when completed
And the motion control data generated by the generation procedure.
Data when the operation of the performance operator
Demonstrate the operation start timing that should start the transition to the finger state.
A predetermined value that represents the time required to complete the operation of the performance operator
In the decision procedure that determines based on the reference time of
The motion control data generated and the motion opening determined by the determination procedure
Operate the performance controls based on the start timing
The display procedure for displaying the motion of your finger as a movie
The program to be read and executed by is recorded .

A recording medium according to another aspect of the present invention includes event data representing the content of an event to be generated on a group of performance operators to be subjected to a performance operation, time data indicating the timing of occurrence of the event, and an acquisition procedure for acquiring a tone-fingering data composed of finger data that specifies the finger to be used to generate the event, based on the tone-fingering data acquired by acquisition procedure, the performance operation When the operation of the performance operator in the child group is completed or
Was completed and generation procedure for generating an operation control data representing the state of the finger when the completion of the release, the operation of the performance operator represented by Ru generated by the generation procedure kinematic <br/> operation control data
The operation start timing that should start the transition to the state of the finger when the finger is pressed is the same as the finger used when operating the performance operator.
The time data in the musical tone / fingering data immediately before having the data
Data and the time required to complete the operation of the performance controller.
First reference time to represent and release of operation of the performance operator
And determination steps of determining based on a second reference time represents the time required to complete, based on the operation start timing determined by the generated operation control data and determining steps on generation procedure, the performance operation actual read display procedure and which Ru is moving image display operation of the finger to continue to operate the element group, to a computer
It has recorded a program for row.

According to the present invention, based on the musical tone / fingering data, a group of performance operators , for example, a key press or key release operation on the keyboard is performed.
This motion is generated by generating motion control data that represents the state of the human finger.
Video is displayed based on the work control data . In this way, by expressing a series of flow of the playing action with a moving image, it becomes possible for the user to easily understand the action of the finger during the playing. In addition, the video display when pressing the key
Event occurrence (pronunciation) in musical tone and fingering data
Start moving your finger earlier than Ming
To control the actual performance in a natural way.
A close video display is possible .

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a circuit configuration diagram of a performance operation display device according to the present embodiment.
This performance operation display device is realized as a device that acquires musical sound / fingering data described later from the outside and displays the motion of a finger during performance as a moving image based on the acquired musical sound / fingering data.

As shown in FIG. 1, the performance operation display device includes a CPU 101 for controlling the entire device, and a CPU 101 for controlling the entire device.
101 storing such profiles grams and various control data to perform (including various data for image display) ROM 102
And a RAM 103 used by the CPU 101 for work
A data input unit 104 for acquiring various data including the above-mentioned musical sound / fingering data, a display unit 105 for displaying the data sent from the CPU 101 on the screen, and an operation unit 106 to be operated by the user. A sound source 107 that generates waveform data of musical tones according to instructions from the CPU 101, a sound system 108 that converts the waveform data output from the sound source 107 into sound, and emits the sound, and a bus 109 that connects the CPU 101 to the units 102 to 107. And are provided.

The data input unit 104 is, for example, a floppy (registered trademark) disk drive (FDD).
The display unit 105 is a display device such as a CRT or a liquid crystal display device. The operation unit 106 is an input device including a plurality of operators such as various switches and keys, and a detection circuit that detects an operation on the operators.
The sound system 108 is, for example, a D / A converter,
It is a system consisting of an amplifier and a speaker. In addition,
The data input unit 104 may be an interface capable of receiving at least a signal output by an external device, or a communication control device that communicates with a predetermined network.

As shown in FIG. 2, the musical tone / fingering data is formed by adding the fingering data to the musical tone data. The musical tone data is performance data (sequence data) that is usually used for playing a musical performance, and the musical scale (pitch) data, whether the musical tone starts to sound (note-on event) or silence (note-off event). The type data indicating the type of the event including the, and the pitch data of the musical tone and the time data indicating the timing at which the event specified by the type data occurs are defined as one unit of data (hereinafter referred to as unit musical tone data for convenience). , The unit musical tone data are arranged in the order of event occurrence. The fingering data is data consisting of finger number data added to each unit tone data and indicating a finger to be used for generating an event represented by the unit tone data. As a finger number, 0 is assigned to the thumb, 1 to the index finger, 2 to the middle finger, 3 to the third finger, and 4 to the little finger.

In the fingering data, usually, in addition to the finger number data, at least hand data indicating whether the finger to be moved is the right hand or the left hand exists, and these data are added to each unit tone data. However, here, in order to avoid confusion and facilitate understanding, only the right hand is focused and hand data and the like are not considered. The time data is data representing the occurrence timing of the event in the time interval (delta time) between the events, and the type data and the pitch data are, for example, data forming MIDI data.

In the tone data (performance data), the tone generation starts (ON) and the tone disappears (OFF).
Are treated as separate events, and the tone generation period of the musical sound is specified by those events, but the musical sound data may be a collection of those events. That is, the unit musical tone data is, for example, pitch data, time data designating the sounding start timing of the musical tone designated by the pitch data, and time length data designating the time length (gate time) during which the musical tone is sounded. It may be composed of As is clear from this, the format of the musical sound data is not particularly limited.

The operation of the above configuration will be described. When the power is turned on, the CPU 101 turns on the ROM 10
By reading out and executing the program stored in 3, the control of the entire apparatus is started and the initial screen is displayed on the display unit 105. After that, the operation unit 106 of the user
Control according to the operation.

The CPU 101 generates image data for displaying the initial screen using a plurality of image data read from the ROM 102, using the RAM 103 for work, for example. The display unit 105, to which the image data of the initial screen generated in this way is sent, stores it in a memory (for example, a dual port RAM) not particularly shown. For the whole or partial change of the display content after that, for example, the CPU 101 reads from the ROM 102 the image data of the portion of the image data stored in the memory in which the data should be rewritten, or is generated using it. Then, the image data obtained in this way is
It is performed by overwriting the image data in the memory on the portion to be rewritten. In that way, the CPU 101
Causes the display unit 105 to display a screen or a message to be displayed, a finger or the like expressing a performance operation described later, as necessary.

When the user operates the operation unit 106 to instruct to read the data of the file stored in the floppy (registered trademark) disk (FD), the data indicating that is transmitted from the operation unit 106 via the bus 109. CPU 101
Sent to. The CPU 101 that received the data
A command is sent to the data input unit (FDD) 104 to read it, and then the data input unit 104 is read.
The data of the file sent from the RAM is stored in the RAM 103. In this way, the data specified by the user including the musical tone / fingering data is obtained from the FD in file units.

The specification of the file from which the CPU 101 reads data is not described in detail, but the operation unit 1
The file name is input by operating 06, or a list of files stored in the FD is displayed on the display unit 105 and a file is selected from the list. The file name displayed in the list is CP
The U 101 causes the data input unit 104 to read and receive the file name stored in the FD.

As in the conventional performance display device disclosed in Japanese Unexamined Patent Publication No. 10-39739, basically, by displaying only the shape of a finger when a key is pressed, the finger before the transition to that shape is displayed. It is difficult for the user to understand how to move.
For this reason, in the present embodiment, the motion from the state of the finger until then to the state of the finger that causes the event is displayed as a moving image for each event. That either move from where the hand to where, the operator displays video that what fingers how to change, such as. In this way, when a series of finger movements during performance is displayed in detail, the user can be made to understand without having to think about how to move the fingers.

In order to display the motions of the fingers during the performance as a moving image, it is necessary to generate data for that (hereinafter referred to as motion control data). The data is
It is generated from the musical tone / fingering data, but depending on the target musical tone / fingering data, a heavy load is required to generate it. This means that if it is attempted to reproduce the musical tone / fingering data (musical tone data) while generating the motion control data, there may be a problem in the pronunciation of the musical tone or the display of the image. That is, there is a possibility that a musical sound cannot be sounded at the timing at which it should be sounded, or that the motion of a finger cannot be displayed smoothly. Therefore, in the present embodiment, the motion control data is generated separately from the reproduction of the musical tone / fingering data.

The operation control data is generated by, for example, the RAM 1
The musical tone / fingering data stored in 03 in the file format is targeted. When the user operates the operation unit 106 to specify (select) one of the musical sound / fingering data stored in the RAM 103 and instruct to generate the motion control data, the CPU 101 performs the following steps. The operation control data of the musical sound / fingering data designated by the user is generated. Note that the method of designating the musical sound / fingering data is basically the same as the case of designating the file to be read from the FD.

FIG. 3 is a diagram for explaining the structure of the fingers for display. Each part of the body can move by bending a joint. This is the same with fingers. Therefore, in the present embodiment, as shown in FIG. 3, the wrist and the tip are divided into a hand and five fingers. Each finger is further divided into three parts because it can perform bending motion of three joints.
Hereinafter, the fingers to be displayed will be described with the “part” added at the end to distinguish them from those of the actual fingers.

Reference numerals 301 to 306 in FIG. 3 are base points for moving each part. Specifically, the base point 301 is the base point of the hand part and corresponds to (the joint of) the wrist. Base point 3
The reference numeral 02 is the base point of the thumb part and corresponds to the first joint thereof. The base points 303 to 306 are the base points of the index finger part, the middle finger part, the ring finger part, and the little finger part, respectively, and correspond to the first joint of the corresponding finger.

The states of the hand parts are x, y, and
In addition to the positions hx, hy, and hz on the z axis, the rotation angle hrz around the z axis is used for management. The values indicating them are retained by substituting the prepared variables. As a result, the movement of the hand is expressed in three dimensions. The rotation angle hrz about the z-axis is the base point from the position on the x-axis where the base point 304 is projected when the length La from the base point 301 of the hand part to the base point 304 of the middle finger part is projected on the x-axis. The reference (0 degree) is a state in which the length obtained by subtracting the projected position on the x-axis of 301 matches the length La. For example, the counterclockwise direction is positive and the clockwise direction is negative as viewed in FIG. In this way, the relative positional relationship with the keyboard 300 in FIG. 3 can be arbitrarily changed. The rotation angle around the x-axis and / or the y-axis may be further considered. For the variables that hold the above hx, hy, hz, and hrz, the symbol will be added to the end of the description (variable hx, etc.).

Each finger has three joints as described above. From this, not only the first joint but also the second and third joints
The joint is also used as a base point, and the parts between the first joint and the second joint, between the second joint and the third joint, and beyond the third joint are one part. Here, each of those parts,
They will be referred to as the finger first part, the finger second part, and the finger third part. Since the lengths L1, L2, and L3 are different for each finger, they are determined for each finger part.

The first joint of the finger moves in two directions (bending and extending directions and opening and closing directions), and the other second and third joints move only in one direction. From this, the state of the first part of the finger is managed by the rotation angle frz around the z axis on the base point corresponding to the first joint and the angle frx1 obtained by bending it from the back of the hand toward the palm. The finger second part and the finger third part are managed at angles frx2 and frx3 obtained by bending them from the back of the hand toward the palm on the base points corresponding to the second joint and the third joint, respectively. In this way, it is possible to imitate actions such as expanding or narrowing the space between fingers, or extending or bending the fingers. An angle frx1 for managing the states of the finger first to third parts
.About.frx3 is represented by, for example, a state where two parts connected via a base point are straight lines as a reference (0 degree). Therefore, for example, when the back of the hand and the second joint of the finger are linear, the angle frx1 is 0. Hereinafter, the movement that changes any of the angles frx1 to 3 will be referred to as the vertical movement of the finger (finger part), and the movement that changes the rotation angle frz will also be referred to as the horizontal movement of the finger (finger part).

The values of frz and frx1 to frx3 are retained by substituting them into prepared variables. Since these values must be held for each finger part, the variable is an array variable and the corresponding value is assigned to the element specified by the finger number. After that, fr
An array variable that holds z and frx1 to 3 is also described by adding the symbol to the end (array variable frz,
Etc.)

The base points 302 to 306 of each finger part move in accordance with the change in the position of the base point (wrist) 301 of the hand part. As described above, each finger part can express a movement imitating a human finger. Therefore, the keyboard 300
The action of moving your hand to press any key above,
A motion of a finger pressing a key or releasing a key, and a motion of a combination thereof can be represented by a moving image. The expression is
Since the operation of each part of the fingers operating the keyboard 300 is displayed, the data such as the size of the white and black keys on the keyboard 300, their arrangement, and the positional relationship between the keys are variable in advance. It is prepared in the form of.

FIG. 4 is a diagram for explaining the range of movement of the hand. The hand part manages its display state by the base point 301. FIG. 4 shows a range that can be taken as the position of the base point 301.

The distance on the y-axis between the keyboard and the wrist differs depending on whether the key is pressed is the white key or the black key. As a matter of course, when the black key is pressed, the distance becomes smaller than when the white key is pressed. Further, even if the same white key or the black key is pressed, the distance changes depending on whether a plurality of keys are pressed simultaneously or only one key is pressed. When a plurality of keys are pressed at the same time, the space between the fingers is widened, so that the keys are closer to the keyboard than when only one key is pressed. From this, in the present embodiment, the y of the base point 301 when separated from the keyboard 300
YPOS1 is defined as a position on the axis, and YPOS2 is defined as a position when the keyboard 300 is approached.

On the other hand, the distance on the z-axis between the keyboard and the wrist (palm) also changes depending on whether a plurality of keys are pressed simultaneously or only one key is pressed. When pressing a plurality of keys at the same time, since the whole hand is moved toward the keyboard, the distance becomes smaller.
Therefore, in the present embodiment, the ZPO position is set as the position on the z-axis of the base point 301 when it is separated from the keyboard 300.
S1, ZP as the position when approaching the keyboard 300
OS2 is defined. By setting such a position, the white key,
Or, whether to press the black key or multiple keys 1
I try to express the difference in behavior, such as whether to press a key.

As shown in FIG. 3, the moving image display of fingers divided into a plurality of parts is performed by sequentially changing the state of each part according to the progress of the performance. The moving image control data for realizing the moving image display is generated in two steps in this embodiment. Here, for convenience, the data generated in the first stage will be referred to as moving image control data A, and the data generated in the next stage will be referred to as moving image control data B.

The moving picture control data A is stored in the CPU 101.
However, in addition to the musical tone / fingering data as shown in FIG. 2, the length of each finger part (the length of the finger first to third parts), the arrangement in the hand part, the arrangement of keys on the keyboard 300, and the like. It is generated by referring to data (parameters) prepared in advance such as various key shapes. The CPU 101 uses the unit tone data in the tone / fingering data and the finger number data added to the unit tone data.
Furthermore, by analyzing the flow of the performance specified by them, the state of the finger that is to be moved to generate the next event from the state of the finger of the previous event is obtained, and the state of the finger of the latter is specified. The data to be specified, that is, the data designating which part state and how should be changed is stored in the RAM 103. The speed with which you move your hand or finger depends on the length of the time interval between events. The shorter the time interval, the faster you must move your hands and fingers. Therefore, based on the time interval between events, the timing to start changing the state of the part and the time during which the change continues are determined, and the data indicating them is also stored in the RAM 103. The data generated and stored in this way is the moving image control data A. The above analysis is performed by the algorithm used to generate the fingering data added to the musical sound data, for example. As a result, the condition of the obtained finger does not become an unnatural shape.

By the generation of the moving image control data A, in the key release event, the state of the finger part to be released after the key release is determined, and in the key press event, in addition to the state of the finger part when the key is pressed, if necessary. Then, the states of other parts are determined in accordance with the transition to that state. What is the determination of the state of other parts?
For example, the position of the key to be pressed and its type (white key or black key)
If there is a key that is being pressed or if there is a key being pressed, the finger part that is pressing the key cannot be released from that key. To widen or narrow. CPU1
01 divides into several stages until it shifts to the determined state, and generates data indicating the state for each stage,
The operation control data A is stored separately in the RAM 103. The data generated and stored in this way is the motion auxiliary data B.
Is. In this embodiment, as will be described later, the operation control data B is also generated in association with the generation of the operation control data A for each event.

FIG. 5 is a diagram for explaining the structure of the operation control data generated / saved as described above. The figure (a) shows the operation control data A, and the figure (b) shows the operation control data B.

The operation control data A is, as shown in FIG. 5A, data for designating a variable whose value is to be changed or an element of an array variable (hereinafter referred to as variable designation data), a variable,
Alternatively, the value data which is the value after the change of the element of the array variable, the operation time data indicating the time during which the part corresponding to the variable or the element of the array variable is operated, and the operation end or operation start timing of the part. The time data for designating "1" is configured as data corresponding to one event in the performance (these are collectively referred to as unit operation data A). Value data and operation time data exist for each variable designation data. Thereby, the state of the part designated by the variable designated by the variable designation data or the value of the element of the array variable can be changed independently. The time data in the unit motion data A matches the time data in the corresponding unit tone data in the tone / fingering data. Since a set of variable designation data, value data, and operation time data represents how to change a certain state quantity of a part, they will be collectively referred to as part operation data hereinafter. To do.

The operation control data A shown in FIG. 5A is generated on the basis of the musical tone / fingering data shown in FIG. 2, and causes the following events, that is, the state change of the part. Is represented. The description will be given in chronological order with reference to FIG.

As shown in FIG. 2, the event designated by the first unit tone data in the tone / fingering data is the start of sounding the tone having the note name "C3". The key number (note number) 64 corresponds to the "C3". For this reason,
The event represents the pressing of the key of keyboard number 64.

The occurrence of the event is the timing designated by the value 100 of the time data, and the event does not exist before that. An initial state (corresponding variable or initial value of an element of an array variable) is defined for each part. Therefore, to display how the event occurs, move the hand part to a position where you can press the key on the keyboard number 64 by the index finger part specified by the finger number data. I have to keep it.
Here, the hand part must be moved to a position where the value of the variable hx is obtained as 100 of the value data. The first unit operation data A in the operation control data A is generated for that purpose. In the unit operation data A,
The value of the time data is 0 and the operation time data is also 0. Therefore, as shown in FIG. 6A, this means that the hand part is displayed at the position on the x axis with the value of the variable hx set to 100 from the beginning.

The hand part does not have to be displayed from the beginning at the position where the first key depression can be performed. For example, after being displayed at a predetermined position, it may be moved from that position to a position where the key can be pressed by the time when the key should be pressed. The values of time data and operation time data are
Since it is expressed in a predetermined unit time, the unit of time designated by the value will be called unit time.

In the next unit operation data A, the time data is 100, the variable designation data designates the array element frx1 [1] and the array element note [1], and the value data is the array element f.
The value of rx1 [1] is 10, and the array variable note
64 is designated as the value of [1], and the operation time data is -5 in the array element frx1 [1], and the array element not.
It is 0 at e [1]. This means that, after 100 unit time has passed from the event represented by the immediately preceding (first) unit motion data A, an event of bending the index finger part at the first joint and pressing the key of the keyboard number 64 is generated. There is. A negative value of the operation time data indicates that the bending of the first joint is started 5 unit times earlier than the timing specified by the time data. As a result, after 95 unit time has passed from the last event,
The index finger part starts the key pressing operation for the key of the keyboard number 64, and the key pressing is completed when 100 unit time has elapsed. The 0 in the operation time data of the array variable note [1] indicates that the musical tone starts to be sounded at the timing indicated by the time data.

In the next unit operation data A, the time data is 50, the variable designation data designates the array element frx1 [1] and the array element note [1], and the value data is the value of the array element frx1 [1]. 0 is the array variable note
0 is designated as the value of [1], and the operation time data is 2 in the array element frx1 [1], and the array element note.
It is 0 in [1]. This is after 50 unit time has passed from the event represented by the immediately preceding unit operation data A,
This indicates that the index finger part that has been bent at the first joint is extended to generate an event of releasing the key of the keyboard number 64. The value of the operating time data is positive. As a result, after 50 units of time have passed from the previous event, the index finger part begins to extend, releases the key of the keyboard number 64, and then returns to the original position after 2 units of time have elapsed. 0 of the value data of the array variable note [1]
Represents a key release, and 0 of the operation time data represents that the musical sound is muted at the timing indicated by the time data.

As described above, the unit operation data A in the operation control data A not only specifies which part is transferred to which state when an event occurs, but also specifies the time taken for the transfer by the operation time data. It is supposed to do.

On the other hand, the operation control data B is, as shown in FIG. 5B, after the change of the variable designating data for designating the variable whose value is to be changed or the element of the array variable, the variable, or the element of the array variable. Value data that is the value of, and time data that designates the timing to change to the state designated by the variable designation data and its value data, the data that represents one stage of the operation (collectively these as stage operation data Is called). As a matter of course, the value of the time data in the step operation data is less than or equal to the value of the time data in the corresponding unit operation data A.
The number of part operation data constituting the, that is, the number of variables designated by the variable designation data or the number of elements of the array variable differs depending on the state and the content of the event. If you only need to bend the finger part, you only need to change the value of the element of one array variable. However, if a certain finger part is bent by widening the space between the finger part and another finger part, it is necessary to change the values of the two elements for that finger part. For example, if the finger part is the middle finger part and the distance from the forefinger part is widened, the ring finger part and the little finger part located outside the middle finger part are also adjusted to one by each side movement of the middle finger part. You have to change the value of the element. From such a point, FIG.
In (b), for each part, variables that can be subject to change of values and elements of array variables are shown. Frz and frx1 to 3 of the thumb part are all array variable elements,
Although not explicitly shown, they are also present in the index finger part, middle finger part, ring finger part, and little finger part.
The element of each array variable is designated with the finger number of the finger part as an argument.

The operation control data B shown in FIG. 5B is shown in FIG.
It is generated based on the operation control data A shown in (a). The value of the operation time data in the unit operation data A located first in the operation control data A is 0, the value of the time data in the second unit operation data A is 100, and the first in the unit operation data A value of the operating time data in the part operation data located at the - 5. The values of the time data in the second and third stage operation data are 95 and 2, respectively.
Has become. As is apparent from this, the stepwise motion data is generated based on the second unit motion data A, and is not particularly shown in order to express the key pressing motion by the index finger part. The array element frx1 [1] is designated by the variable designation data, and the value to be assigned to it is set as the value data.

FIG. 6 is a diagram for explaining an example of finger motion display. FIG. 6 shows how the state of each part of the finger is changed according to the operation control data shown in FIG. 5. In FIG. 6A, in the musical tone / fingering data, the sound generation start timing (key pressing timing) of the musical tone specified by the musical tone / fingering data and its mute timing (key releasing timing) are indicated by arrows on the time axis. It is shown. The note names of musical tones (such as "C" and "F") are shown near the arrows. In the finger operation, the key press operation start timing and the key release operation start timing are indicated by arrows on the time axis. Numbers such as "1" and "2" written near the arrow are finger numbers of finger parts to be pressed or released. In the hand x position,
The time change of the position of the hand part on the x-axis is shown. Hand y
In the position and the hand z position, time changes of the position on the y axis and the z axis of the hand part are shown, respectively. The 0 indicated on the time axis of the hand z position indicates the time when the reproduction of the musical sound / fingering data is started. It should be noted that the hand x position, the hand y position, and the hand z position respectively indicate movements from the initial position.

As shown in FIG. 6 (a), the key pressing operation is started before the timing of actually producing the musical tone, and the key releasing operation is started at the timing of muting the musical tone. There is. The position of the hand part on the x-axis starts to move to a position where the key having the note name “F” can be pressed next by the index finger part after the key releasing operation of the key having the note name “C” is started. , The movement is completed before the key depression start timing. Regarding the position of the hand part on the y-axis, the note names of the musical tones produced by the thumb part, the index finger part, the middle finger part, and the ring finger part are “D”.
In order to express that the keys of “#”, “F”, “G”, and “B” are pressed at the same time, the movement toward the keyboard 300 is started after the key of “G” is released. That is, the movement starts from the position YPOS1 shown in FIG. 4 toward the position YPOS2, and the position YPOS2 continues until the keys are released.
After the key release, the movement from the position YPOS2 to the position YPOS1 is started. Hand part z
The position on the axis is lowered toward the keyboard 300 from the start of the key pressing operation of those keys to the end of the key pressing of the last key. That is, in the meantime, it moves from the position ZPOS1 shown in FIG. 4 to the position ZPOS2. The movement from the position ZPOS2 to the position ZPOS1 is performed from the start of the key releasing operation of those keys to the end of the key releasing of the last key.

FIG. 6B shows the time points in FIG.
5 is a diagram showing the position of a base point (wrist) 301 of the hand part in FIG. The positions on the xy plane at those time points are indicated by.

As shown in FIG. 6B, the hand part moves from the position shown at the beginning to the position shown at. Next, in order to press a plurality of keys at the same time, move to the position indicated by. As is clear from the fact that the position indicated by and the position indicated by, are the same while pressing a plurality of keys. The position indicated by is the same as the position indicated by. Therefore, after releasing the keys, the key moves from the position indicated by to the position indicated by.

As described above, the motion of the fingers is represented by a moving image by moving the hand part or changing the state of each finger part. FIG. 7 is a diagram showing an actual display example.

As shown in FIG. 7, not only the right hand but also the left hand is actually displayed. In order for the user to easily recognize whether or not the key is being pressed, the key being pressed is expressed in a different display color from the other keys. The display color can be changed when a key is pressed (at the start of sounding)
It may be performed only when the key is released (when the sound is muted), but multiple display colors are used to notify the key to be pressed in advance or the key to be released in advance. The color may be changed to. Although detailed description is omitted, in the present embodiment, the size of each part of the fingers is changed, that is, enlarged or reduced, the transparent display of the display of each part, or the change or display to the wire frame display. You can also change the angle. When the keyboard actually operated by the user is provided, the reproduction speed of the musical tone / fingering data (including the display speed of the performance operation) may be changed according to the performance speed of the user.

The operation display of the finger part as shown in FIG. 6 is performed when the user operates the operation unit 106 to instruct the reproduction of the musical sound / fingering data stored in the RAM 103. When the user gives the instruction, the CPU 101
Musical tone / fingering data (here, only the musical tone data, which is the portion excluding the fingering data), and the motion control data B generated based on it are processed in parallel. As a result, the operation of each part of the finger synchronized with the pronunciation of the musical sound is displayed on the display unit 105. In addition to the above, in the present embodiment, a musical score indicating musical performance data is displayed from the musical sound data.

FIG. 8 is a functional block diagram of the performance operation display device according to this embodiment. As shown in FIG. 8, the performance operation display device is basically functionally
It is composed of a data generation unit 801 that generates operation control data from fingering data, and a real-time processing unit 802 that reproduces tone / fingering data and operation control data. In the data generation unit 801, the motion control data generation unit 803 generates the motion control data from the musical tone / fingering data.
Done by The data generation unit 801 includes
In, for example, CPU 101, ROM 102, RAM
103, the data input unit 104, the display unit 105, and the operation unit 106 correspond to each other.

On the other hand, the real-time processing unit 802 is a data transmission unit (sequence processing unit) 804 for processing the operation control data and the musical tone data generated by the data generation unit 801.
The sound source section 8 to sound a musical tone in accordance with the operation display unit 805 for displaying the operation of the finger in accordance with the operation control data sent from the data sending unit 804 tone data then sent,
06, and a musical score display unit 807 for displaying a musical score in accordance with the musical sound data sent from the musical instrument 06. In FIG. 1, for example, CPU 101, ROM 102, RAM 1
03, the display unit 105, the operation unit 106, the sound source 107, and the sound system 108 correspond.

Next, the operation of the CPU 101 for realizing the above-mentioned generation of the operation control data and the reproduction thereof will be described in detail with reference to various operation flowcharts shown in FIGS. 9 to 21.

FIG. 9 is an operation flowchart of the operation control data generation processing. This generation processing is performed by the CPU 101 when the user operates the operation unit 106 to specify (select) one of the musical sound / fingering data stored in the RAM 103 and to instruct generation of motion control data. , ROM
The process is realized by executing the program stored in 102. The data generation unit 801 shown in FIG. 8 is realized by executing the generation process. First, the generation process will be described in detail with reference to FIG.

In this generation process, the finger number data designates the event represented by the unit tone data while sequentially reading out the unit tone data and the finger number data added thereto from the tone / fingering data designated by the user. The operation control data for displaying the state of being generated by the finger is generated. Here, the flow of processing is shown with emphasis on note-on events and note-off events as the types of events represented by the unit musical sound data.

First, in step 901, initial setting is performed. By the initial setting, initial values are assigned to various variables, and an area for storing operation control data is secured in the RAM 103. The variable pe for managing the unit tone data read from the tone / fingering data is assigned the address value at which the unit tone data at the head is stored.

In step 902 following step 901, it is determined whether or not the musical tone / fingering data has been completed. For example, at the end of the musical tone / fingering data, unique data indicating the end of the data (hereinafter referred to as end data) is added. Therefore, when the data designated by the value of the variable pe is the end data, the determination is YES, and the series of processing is ended here. Otherwise, the determination is no and the process moves to step 903.

In step 903, the value of the variable tktime is assigned to the variable tkold, and then the variable tktim.
The value obtained by adding the value of the time data of the unit musical sound data specified by the variable pe to the value up to that point is substituted into e. In the following step 904, it is determined whether the event represented by the unit musical sound data designated by the value of the variable pe is a note-on event. If the unit tone data represents a note-on event, the determination is yes and step 90.
If not, the determination is NO and the process proceeds to step 915. It should be noted that, as is clear from the fact that the values of the time data of each unit musical sound data are accumulated, the value of the variable tktime is a value indicated by the time (absolute time) elapsed after the reproduction of the musical sound data is started. .

In steps 905 to 909, the processing for dealing with the unit tone data representing the note-on event is performed. First, at step 905, various settings are made based on the unit musical tone data designated by the value of the variable pe.
As a result, the value of the finger number data is assigned to the variable fig,
Pitch data (keyboard number) is substituted for the variable note, and unit operation data A is generated and stored from those values. After such substitution is completed, the process proceeds to step 906. The unit operation data A generated here
Is time data, variable designation data that designates the array element note [fig] designated by the value of the variable fig, and value data of the array element note [fig] (variable fig).
Is the note number of the key to be pressed by the finger part specified by the value (see FIG. 5A).

At step 906, when a note-on event represented by the unit tone data designated by the value of the variable pe occurs, whether or not another note-on event is occurring, that is, whether or not another key is depressed. Determine whether or not. If no other key is pressed, the determination is yes and step 9
If not, the determination is NO, and the process proceeds to step 909 described later. The determination of the key depression is made on the assumption of a virtual performance represented by the musical tone / fingering data. For the sake of convenience, the explanation will be given on the assumption that it is used unless otherwise specified.

In step 907, the y position processing of the hand for determining the position of the hand part on the y axis is executed. In the following step 908, the z position processing of the hand for determining the position of the hand part on the z axis is executed. Step 90 to move to thereafter
In 9, the note-on main processing is executed to determine the content of operating each part of the finger in accordance with the note-on event represented by the unit musical sound data while considering the relationship with other events. After executing the note-on-main process, the process proceeds to step 910.

At step 910, it is determined whether or not the value representing "TRUE" is substituted for the variable bchord. When it is determined that the note event represented by the unit musical tone data is for producing a chord during execution of the immediately preceding note-on-main processing, a value representing “TRUE” is substituted into the variable bchord. Therefore, if it is determined that the unit tone data is for producing a chord when the note-on-main processing is executed, the determination is YES.
Then, the process proceeds to step 913. Otherwise, the determination is no and the process moves to step 911.

At step 911, x position processing of the hand for determining the position of the hand part on the x axis is executed. In the following step 912, the y position reprocessing for determining the position of the hand part on the y axis is executed again. After that processing is executed, other processing is executed in step 913, and in the following step 914, the address of the unit musical sound data to be the next target is substituted into the variable pe, and then the processing returns to step 902.

Steps 907 to 909, 911, 9
In step 12 and step 917 described later, the part action data focusing on the parameter (variable or the element of the array variable) accompanied by the specific action is generated and added to the unit action data A. As a result, at the time of shifting to step 913, the state in which each part of the finger is to be shifted is determined. Therefore, in step 913, stepwise operation data is generated from the unit operation data A and stored in the RAM 103. In the present embodiment, the generation is performed by dividing the time interval specified by the operation time data into a plurality of intervals and interpolating a straight line between the state up to that point and the state after the transition. In step 913, unit tone data representing an event other than the note-on event and the note-off event is also processed.

On the other hand, in step 915, in which the determination in step 904 is NO and the process shifts, it is determined whether or not the unit musical tone data designated by the value of the variable pe represents a note-off event. If the unit tone data represents a note-off event, the determination is yes and the process moves to step 916. If not, the process proceeds to step 913.

At step 916, various settings are made based on the unit musical tone data designated by the value of the variable pe. As a result, the value of the finger number data is substituted for the variable fig, the pitch data (keyboard number) is substituted for the variable note, and the unit operation data A is generated and stored from those values. After such assignment is completed, the process proceeds to step 9 17. The unit operation data A generated here is
Array element no specified by the time data and the value of the variable fig
It is variable designation data that designates te [fig], and the value data of its array element note [fig] (it is 0 because it is a key release here). In the subsequent step 917, a note-off main process for generating operation control data (part operation data to be added to the unit operation data A) for operating each part of the finger at the note-off event is executed. After that, the processing shifts to the above step 910.

As described above, in the motion control data generation process, the motion control data is generated while sequentially changing the unit tone data to be the target of motion control data generation and the finger number data added thereto. As a result, the operation control data is generated in units of performance events.

Next, various subroutine processes executed in the operation control data generation process will be described with reference to FIGS.
This will be described in detail with reference to the operation flowchart shown in FIG.

FIG. 10 is an operation flowchart of the y position processing of the hand executed as step 907. In the subroutine processing, first referring to FIG.
The position processing will be described in detail.

The value indicating the position of the base point 301 on the y-axis is held in the variable hynew. In the y position processing of this hand, a position where the base point 301 should be arranged next is obtained, and a value indicating the position is substituted into the variable hynew.

First, in step 1001, initialization is performed and the value of the variable hynew is substituted into the variable hynow. At the following step 1002, the unit musical tone data of interest and the data after the finger number data are currently read out, and whether the thumb or the little finger presses the black key until all the keys are released. Determine whether or not. Next, if there is unit tone data of a black key pitch having 0 or 4 as finger number data by the time all keys are released , the determination becomes YES and the process proceeds to step 1005. Otherwise, the determination is no and step 1003.
Move to.

In step 1003, it is determined whether or not a value indicating the position YPOS1 (see FIG. 4) is assigned to the variable hynow. If the value has been assigned to the variable hynow, the determination is yes, and the series of processes ends here. Otherwise, the determination is no, in step 1004 a value indicating the position YPOS 1 is assigned to the variable hynew, and the position Y is changed from the processing timing of the immediately preceding unit tone data to the processing timing of the unit tone data this time.
After generating the part operation data for completing the movement to the POS 1 and adding it to the unit operation data A, the series of processes is ended.

On the other hand, in step 1005, which is judged as YES in step 1002, the variable hynow is also used.
It is determined whether or not a value indicating the position YPOS1 (see FIG. 4) has been substituted into. If the value is not assigned to the variable hynow, the determination is NO, and the series of processing is terminated here. Otherwise, the determination is yes and in step 1006 the position YPOS is set in the variable hynew.
After substituting the value indicating 2 and generating the part operation data for completing the movement to the position YPOS2 from the processing timing of the last unit musical sound data to the processing timing of the current unit musical sound data, a series of processing is ended.

As described above, in the y-position processing of the hand, when the user has to press the black key with the thumb or the little finger, the hand part is moved to the position YPOS2 so as to approach the keyboard 300 in advance. I have to. Including this time
When all the keys are released without using the thumb and little finger, the hand part is located at the position YPOS1.

In addition to pressing the black key with the thumb or the little finger, if a plurality of keys are pressed with the fingers wide open, the distance between the hand and the keyboard on the y axis is reduced. Get smaller.
Base part 301 of the hand part for dealing with such a thing
The position adjustment on the y axis is performed by the y position reprocessing of the hand executed as step 912.

FIG. 11 is an operation flowchart of the hand z position processing executed as step 908 in the operation control data generation processing shown in FIG. Next, the processing will be described in detail with reference to FIG.

Z at which the base point 301 should be present at the present time
The value indicating the position on the axis is held in the variable hznow. Variable tkti when processing the immediately preceding unit tone data
The value of me is held in the variable tkold. As shown in FIG. 4, the base point 301 of the hand part is the position Z on the z axis.
The position is either POS1 or ZPOS2. At the time of executing the z position processing of the hand, there is no key being pressed.

First, in step 1101, the variable hzn
It is determined whether or not the value indicating the position ZPOS1 is substituted for ow. If the value has been assigned to the variable hznow, the determination is yes, and the series of processes ends here. If not, that is, if the base point 301 of the hand part is located at the position ZPOS2, the determination is NO, the process proceeds to step 1102, and the value of the variable tktime is indicated from the timing indicated by the value of the variable tkold. By the timing, part operation data for moving the base point 301 from the position ZPOS2 to the position ZPOS1 is created, and the part operation data is added to the unit operation data A and stored in the RAM 103. After that, the series of processing is ended.

FIG. 12 is an operation flowchart of the note-on main processing executed as step 909 in the operation control data generation processing shown in FIG. Next, the processing will be described in detail with reference to FIG.

A value ("TRUE" or "FALS" indicating whether or not a chord (chord) is being sounded when the unit musical tone data to be processed is actually processed, that is, reproduced.
E ”) is held in the variable bchord. If a chord is being sounded, the address of the unit tone data of the tone that is the last one of the tones that compose the chord is held in the variable pchorend. In the note-on-main processing, the values of various variables including such variables are referenced.

First, in step 1201, the variable bch
It is determined whether the value of ord is “TRUE”. If it is assumed that a chord is being sounded in the situation where the unit tone data that is the processing target is reproduced, “TRUE” is assigned to the variable bchord, and therefore the determination is YE.
If S, the process proceeds to step 1207, and if not, the determination becomes NO and the process proceeds to step 1202.

In step 1202, the variable ballof
It is determined whether the value of f is “TRUE”. If there is a depressed key in the situation of reproducing the unit tone data to be processed, the determination is NO and step 12
Move to 06. Otherwise, the determination is yes and the process moves to step 1203.

At step 1203, a chord judging process for judging whether or not a chord is sounded by the unit musical tone data to be processed and the following unit musical tone data is executed. In the chord judging process, when it is judged that a chord is produced by the unit tone data, the variable b
“TRUE” is substituted for chord.
After the execution, the process moves to step 1204.

At step 1204, the variable bchord
Is determined to be "TRUE". As a result of executing the chord determination process, when it is considered that a new chord is to be pronounced, the determination is YES, and in step 1205, a series of chord operation processes for determining the content of the action for producing the chord is executed, and then a series of The process ends. If not,
The determination is NO, and after the note-on operation processing for determining the content of the key pressing operation is executed in step 1206, the series of processing ends.

On the other hand, the determination in step 1201 is YES.
In step 1207, in which the process proceeds to S, the variable pch
It is determined whether the value of “endend” is equal to the value of the variable “pe”. The unit musical tone data designated by the value of the variable pe, that is, the unit musical tone data currently being processed, instructs the start of the pronunciation of the final musical tone among the musical tones that are considered to constitute a chord. If YES, the determination is YES and the variable bchord is set to “FAL in step 1208.
SE ”and NULL (null value) are assigned to the variable pchorend, respectively, and then a series of processing is ended. Otherwise, the determination is no and the series of processes ends here.

Thus, in the note-on-main processing,
The processing is branched depending on whether or not the musical sound instructed to start the sound generation by the unit musical sound data constitutes a chord. As a result, each part of the finger is operated depending on whether to produce a chord or simply a musical tone.

Here, various subroutine processes executed in the note-on-main process will be described with reference to FIGS.
This will be described in detail with reference to FIG. FIG. 13 is an operation flowchart of the chord determination process executed as step 1203. In the subroutine process executed in the note-on-main process, the chord determination process will be first described in detail with reference to FIG.

First, in step 1301, initial setting is performed. By performing the initial setting, the variable tknex
In t, a value indicating the processing timing of the unit musical sound data that first instructs the start of musical sound generation after the unit musical sound data specified by the value of the variable pe in absolute time is substituted, and the variable tk
In nextoff, a value indicating the processing timing of the unit tone data for instructing the mute of the tone first after the unit tone data designated by the value of the variable pe in absolute time is substituted.

Step 1302 following step 1301
Then, it is determined whether or not the value of the variable tknext is smaller than the value obtained by adding the value of the variable tkchord to the value of the variable tktime. In the variable tkchord, a value indicating a tone generation start interval between musical tones, which is a reference for determining whether or not a chord is formed, is substituted. From this,
If the next musical tone starts to be sounded within the interval, the determination is YES and the process proceeds to step 1303. Otherwise, the determination is NO and the series of processes is ended here.

At step 1303, the variable tknext is set.
The value of off is changed to the value of variable tktime by variable tkcho.
It is determined whether the value is larger than the value obtained by adding the value of rd. If there is no tone to be muted before the time indicated by the value of the variable tkchord has elapsed, the determination is YES,
After substituting "TRUE" into the variable bchord in step 1304, the series of processes is ended. If not, the determination is no, and the series of processes ends here.

As described above, in the present embodiment, the variable tk
If there are multiple key presses (note-on events) within the time indicated by the chord value, and no key-release (note-off event) exists within that time, the musical tone that starts to be sounded within that time is a chord. Is considered to constitute.

FIG. 14 is an operation flowchart of the chord operation process executed as step 1205 in the note-on main process shown in FIG. Next, the chord operation process will be described in detail with reference to FIG.

First, in step 1401, initialization processing is executed. By executing the initialization process, the value of the variable tktime is assigned to the variable tktime0, the value of the variable pe is assigned to the variable pe0, and the variable pc.
NULL is substituted into the holdend. Step 1
From step 1402 subsequent to step 401, the value of the variable pe0 is sequentially incremented, and processing for specifying the range of musical tones regarded as chords and determining the operation content for producing the chord is performed.

At step 1402, it is determined whether the musical tone / fingering data has been completed. If the data designated by the value of the variable pe0 is the end data located at the end of the musical tone / fingering data, the determination is yes and step 141
If not, the determination is no and the process proceeds to step 1403.

In step 1403, it is determined whether the value of the variable pe0 is equal to the value of the variable pe. If the values are equal, the determination is yes and the process moves to step 1405. Otherwise, the determination is no, and in step 1404, a value obtained by adding the value of the time data of the unit musical sound data specified by the value of the variable pe0 to the value up to that point is assigned to the variable tktime0, and then step 1405.
Move to. In this way, the variable tktime0 is substituted with the value indicating the timing, in absolute time, at which the unit tone data subsequent to the unit tone data designated by the value of the variable pe should be processed.

At step 1405, the variable tktime is set.
The value of 0 is changed to the value of the variable tktime by the variable tkchord.
It is determined whether it is smaller than the value obtained by adding the value of. That is, if the difference between the processing timings of the unit musical tone data designated by the values of the variables pe0 and pe is not within the time designated by the value of the variable tkchord, the determination is no and the above step 1410 is performed. Move to. Otherwise, the determination is yes and the process moves to step 1406.

[0103] At step 1406, the unit tone data value of the variable pe0 specifies the key depression (note-on event)
Is determined. If the unit tone data represents a note-on event, the determination is YES.
Then, in step 1407, the value of the variable pe0 is substituted for the variable pchorend, the note-on operation process is executed in the following step 1408, and then the process proceeds to step 1409. Otherwise, the determination is no and the process moves to step 1409.

At step 1409, the address of the next unit tone data is assigned to the variable pe0. After that, the process returns to step 1402. As described above, the processing loop formed in steps 1402 to 1409 is repeatedly executed until the determination in step 1402 is YES or the determination in step 1405 is NO, so that the variable pchordend is set to the tone that is regarded as a chord. Among them, the address value of the unit musical tone data of the musical tone which finally starts to be generated is substituted.

On the other hand, the determination in step 1402 is YES,
Alternatively, in step 1410 where the determination in step 1405 is NO and the process shifts, the hand part moves on the z axis (here, from position ZPOS1 to position ZPOS shown in FIG. 4).
Part movement data expressing the movement to POS2) is created. By creating the motion data, the base point 301 of the hand part is changed to the variable tkstart.
The movement starts at the timing indicated by the value of 0, and the variable tkti
The movement to the position Z POS2 is completed at the timing indicated by the value of me. After adding such part operation data to the unit operation data A, a series of processing is ended.

The timing indicated by the value of the variable tkstart0 is the timing at which the operation of the finger part for producing the first tone to be sounded among the tones constituting the chord is started, or the key release from each finger part is performed. If the time until shifting to is relatively long, it is the timing indicated by the value obtained by subtracting the value indicating the reference time from the start of the key pressing operation to the key pressing from the value of the variable tktime. As a result, the operation of each part of the finger for playing the chord is made different according to the situation of playing the chord. Figure 6
In the example shown in (1), as a result of the former value being assigned to the variable tkstart0, the movement of the hand part on the z-axis starts at the start of the key pressing operation of the thumb part, and the movement on the z-axis is changed to "D # It means that it has ended at the start of the pronunciation of the musical tone. Note that the value that indicates the criteria time is assigned to the variable Tkcost. The value indicating the reference time from the start of the key release operation to the completion of the operation is the variable tkoffcos.
It is assigned to t.

Next, in step 1408, or in FIG.
The note-on operation process process executed as step 1206 in the note-on main process shown in FIG.
This will be described in detail with reference to the operation flowchart shown in FIG. In this process, the finger number data added to the unit musical tone data is substituted into the variable fig, and the pitch data in the unit musical tone data is substituted into the variable note.

First, in step 1501, initial setting is performed. In the initial setting, a value indicating the absolute timing of the processing timing of the unit musical sound data immediately before the note-off event in which the value of the variable fig is added as finger number data is calculated, and the calculated value is substituted into the variable tkfoff. Is done.

Step 1502 following step 1501
Then, the value obtained by subtracting the value of the variable tkfoff from the value of the variable tktime is the value of the variable tkcost to the value of the variable tkcost.
It is determined whether or not the cost value is smaller than the added value.
That is, the time from the start of the key release operation of the finger part specified by the variable fig to the key press of the finger part is shorter than the time obtained by adding the reference time of the key release operation and the reference time of the key press operation. It is determined whether or not. If the former time is longer than the latter time, the determination is no, and in step 1503, the variable tkstart is changed to the variable t from the value of the variable tktime.
After substituting the value obtained by subtracting the value of kc ost, step 15
Move to 04. If not, i.e. key release action,
Alternatively, when there is no room to allow the key depression operation to be performed within the reference time, the determination is yes and the process moves to step 1506.

In step 1504, the variable tkend is set.
The value of the variable tktime is substituted for the value indicating the timing at which the key pressing operation ends. In the following step 1505, the finger part specified by the value of the variable fig is changed to the variable tk.
The key pressing operation for the key indicated by the value of the variable note is started at the timing indicated by the value of start, and the operation is completed at the timing indicated by the value of the variable tkend, that is, the part operation data for causing the key depression operation is created. Then, it is added to the unit operation data A. The time indicated by the value of the action time data in the part action data is the time indicated by the value of the variable t kc ost. After the creation / addition, the series of processing is ended.

In some cases, the finger part specified by the value of the variable fig cannot be pressed unless the hand part is moved. The rotation angle frz of each finger part cannot be fixed unless the position of the hand part is fixed. For this reason, only the part motion data for performing the key pressing motion is created here. The part motion data for laterally moving the finger part is the step 9 in the motion control data generation process shown in FIG.
It is created as necessary when the x position processing of 11 hands is executed. The part motion data created here is also modified as necessary.

On the other hand, the determination in step 1502 is YES.
In step 1506, where the variable S is changed to S, the variable tkt
The value obtained by subtracting the value of the variable tkfoff from the value of image is 0.
It is determined whether the value multiplied by 2 is larger than the value of the variable tkoffcost. If the time obtained by multiplying the time from the start of the key release to the next key depression by 0.2 is longer than the reference time of the key release operation, the determination is YES, and step 150
After substituting the value obtained by adding the value of the variable tkoffcost to the value of the variable tkfoff to the variable tkstart in step 7,
The procedure moves to step 1504. Otherwise, the determination is no and in step 1508 the variable tks
The value obtained by adding the value of the variable tkfoff to the value obtained by multiplying the value of the variable tkoffcost by 0.8 is substituted for the start value, and then the process proceeds to step 1504. Then step 1
If the value of the operation time data set when executing step 505 changes from step 1507 to step 1504, the value of the variable tktime is changed to the value of the variable tkfof.
The value of f is subtracted, and the variable tkoff is further subtracted from the subtraction result.
This is a value obtained by subtracting the cost value. If the process has proceeded from step 1508 to step 1504, the variable tkt
subtracting the value of the variable tkfoff from the value of ime, further from the subtraction result, 0.8 the value of the variable Tkoffco st
It is the value obtained by subtracting the value multiplied by.

As described above, when the key release and the key press by the same finger part are performed within a short time, at least one of the time required for the key release operation and the time required for the key press operation is adjusted to perform those operations. I am trying to express.

In the note-on-main processing shown in FIG. 12,
The various subroutine processes described above are executed. As a result, part motion data for expressing the motion of pressing the finger part is generated in subroutine processing units and added to the unit motion data A. In those Parts operation data, variables h z, is any value to be changed of the array variable f rx1~3.

FIG. 16 is an operation flowchart of the note-off main processing executed as step 917 in the operation control data generation processing shown in FIG. Next, the note-off main process will be described in detail with reference to FIG.

A value ("TRUE" or "F") indicating whether or not there is a chord (chord) to be muted when the unit musical tone data to be processed is actually processed, that is, reproduced.
ALSE ") is held in the variable bchordoff. If there is the chord, the address of the unit tone data of the tone to be silenced last among the tones constituting the chord is held in the variable pchordendoff. In the note-off main process, the values of various variables including such variables are referenced.

First, in step 1601, the variable bch
It is determined whether the value of ordoff is “TRUE”. If it is assumed that there is a chord to be silenced in the situation where the unit tone data to be processed is reproduced, the variable bcho
Since “TRUE” is substituted for rdoff, the determination is yes and the process moves to step 1606,
Otherwise, the determination is no and step 16
Move to 02.

In step 1602, a chord off judgment process for judging whether or not a chord is muted, that is, all the constituent sounds of the chord are muted by the unit musical tone data to be processed and the subsequent unit musical tone data is executed. In the chord off determination process, when it is determined that the chord is muted by the unit tone data, the variable bchord is used.
"TRUE" is substituted for off. After that, the process moves to step 1603.

At step 1603, the variable bchord
It is determined whether the off value is “TRUE”. As a result of performing the chord off determination process, when it is determined that the chord is to be silenced, the determination is YES, and after performing the chord off operation process that determines the content of the operation to silence the chord in step 1604, a series of processes is performed. To finish. Otherwise, the determination is no, and after executing the note-off operation process that determines the content of the key release operation in step 1605,
A series of processing ends.

On the other hand, the determination in step 1601 is YES.
In step 1606 where the processing shifts to S, the variable pch
It is determined whether the value of "orderoff" is equal to the value of the variable pe. If the unit musical tone data designated by the value of the variable pe, that is, the unit musical tone data currently being processed is an instruction to mute the musical tone to be muted last among the musical tones constituting the chord, the determination is If YES, then in step 1607 the variable bchordoff is set to “FALS
E ”and NULL are substituted into the variable pchordendoff, respectively, and the series of processing is terminated. Otherwise, the determination is no and the series of processes ends here.

Thus, in the note-off main process,
The processing is branched depending on whether or not the musical sound instructed to be muted by the unit musical sound data constitutes a chord. Thereby,
The content of each part of the finger to be operated is determined depending on whether to mute the musical tones that compose the chord or simply to mute the musical tones.

Here, various subroutine processes executed in the note-off main process will be described with reference to FIGS.
This will be described in detail with reference to FIG. FIG. 17 is an operation flowchart of the chord off determination process executed as step 1602. In the subroutine processing executed in the note-off main processing, the chord off determination processing will be first described in detail with reference to FIG.

First, in step 1701, initial setting is performed. By performing the initial setting, the variable tknex
In t, a value indicating the processing timing of the unit tone data for instructing the start of the tone generation of the tone first after the unit tone data designated by the variable pe in absolute time is substituted, and the variable tkne
In xtoff, a value indicating the processing timing of the unit musical sound data, which is the first instruction after the unit musical sound data specified by the variable pe, for instructing the mute of the musical sound in absolute time is substituted.

Step 1702 following step 1701
Then, the value of the variable tknextoff is changed to the variable tktim.
It is determined whether or not it is smaller than the value obtained by adding the value of the variable tkchord to the value of e. If there is no tone to be muted before the time indicated by the value of the variable tkchord elapses, the determination is NO, and the series of processes ends here.
Otherwise, the determination is yes and step 1
Move to 703.

At step 1703, the variable tknext is set.
Is larger than the value obtained by adding the value of the variable tkchord to the value of the variable tktime. Variable tkc
If the next musical tone starts to be sounded before the time indicated by the value of “hord” has elapsed, the determination is NO, and the series of processing is terminated here. Otherwise, the determination is yes and the variable bchordoff is set to "T" in step 1704.
After substituting "RUE", the series of processing is terminated.

As described above, in the present embodiment, the variable tk
If there are mutings of a plurality of musical tones (note-on event) within the time indicated by the chord value, and there is no musical tone to start sounding (note-off event) within that time,
It is said that the musical tones that are considered to be the constituent tones of the chord should be silenced.

FIG. 18 is an operation flowchart of the chord off operation process executed as step 1604 in the note off main process shown in FIG. Next, the chord off operation process will be described in detail with reference to FIG.

First, in step 1801, initialization processing is executed. By executing the initialization process, the value of the variable tktime is assigned to the variable tktime0, the value of the variable pe is assigned to the variable pe0, and the variable pc.
NULL is substituted into the holdoff. In step 1802 and subsequent steps following step 1801, the variable p
While sequentially incrementing the value of e0, processing for specifying the range of musical tones regarded as chords and determining the operation content for muting the chords is performed.

In step 1802, it is determined whether or not the musical tone / fingering data has been completed. If the data designated by the value of the variable pe0 is the end data located at the end of the musical tone / fingering data, the determination is yes and step 181
If not, the determination is no and the process proceeds to step 1803.

In step 1803, it is determined whether the value of the variable pe0 is equal to the value of the variable pe. If the values are equal, the determination is yes and the process moves to step 1805. Otherwise, the determination is no, and in step 1804, the value obtained by adding the value of the time data of the unit musical sound data designated by the value of the variable pe0 to the value up to that point is substituted into the variable tktime0, and then step 1805.
Move to. In this way, the variable tktime0 is substituted with the value indicating the timing, in absolute time, at which the unit tone data subsequent to the unit tone data designated by the value of the variable pe should be processed.

At step 1805, the variable tktime is changed.
The value of 0 is changed to the value of the variable tktime by the variable tkchord.
It is determined whether it is smaller than the value obtained by adding the value of. Variable p
If the processing timings of the unit tone data designated by the values of e0 and pe are not within the time designated by the value of the variable tkchord, the determination is no and the process moves to step 1810. Otherwise, the determination is yes and the process moves to step 1806.

In step 1806, it is determined whether or not the unit musical tone data designated by the value of the variable pe0 represents the mute of a musical tone (note-off event). If the unit tone data represents a note-off event, the determination is yes and in step 1807 the variable pchorde.
Substitute the value of the variable pe0 for ndoff, then step 1
After performing the note-off operation process in 808, step 1
Move to 809. Otherwise, the determination is no and the process moves to step 1809.

At step 1809, the address of the next unit tone data is assigned to the variable pe0. After that, the process returns to step 1802. As described above, the processing loop formed in steps 1802 to 1809 is repeatedly executed until the determination in step 1802 is YES or the determination in step 1805 is NO, so that the variable pchordendoff is set to the tone that is regarded as a chord. Of these, the address value of the unit tone data of the tone to be muted at the end is substituted.

On the other hand, the determination in step 1802 is YES,
Alternatively, in step 1810 where the determination in step 1805 becomes NO and the process moves, the hand part moves on the z-axis (here, from position ZPOS2 to position ZPOS2 shown in FIG. 4).
Create part operation data that specifies the contents (which is an operation of moving to POS1). The operation data is created by
The movement is started at the timing indicated by the value of the variable tktime, and the position ZPOS1 is reached at a predetermined speed.
After the part motion data is generated and added to the unit motion data A in this way, a series of processes is ended.

As the contents of the chord silencing operation, the hand part is z
The part motion data designating the movement contents on the axis is generated as described above. As a result, in the example shown in FIG. 6, the movement of the hand part on the z-axis starts at the time when the thumb part starts to release the key to which the pitch “D #” is assigned.

Next, in step 1808 described above, or in FIG.
The note-off operation processing processing executed as step 1605 in the note-off main processing shown in FIG.
This will be described in detail with reference to the operation flowchart shown in FIG. In this process, the finger number data added to the unit musical tone data is substituted into the variable fig, and the pitch data in the unit musical tone data is substituted into the variable note.

First, in step 1901, initial setting is performed. In the initial setting, after the unit tone data designated by the value of the variable pe, the pitch data in the unit tone data representing the note-on event to which the value of the variable fig is added as the finger number data is extracted, Substitution of the extracted pitch data into the variable nextnote is performed.

Step 1902 following step 1901
Then, it is determined whether or not the finger part designated by the value of the variable fig is being used for key depression. When the musical tone specified by the value of the variable note is not actually pronounced, that is, there is no unit musical tone data for instructing the start of the generation of the musical tone, or even if the unit musical tone data exists, When the value of the variable fig is not added as the finger number data, the determination is NO, and the series of processing is ended here. Otherwise, the determination is yes and the process moves to step 1903.

In step 1903, it is determined whether the value of the variable note is equal to the value of the variable nextnote.
If the values are not equal, that is, if the user has to press a key different from the key that has been pressed by the same finger, the determination becomes NO, the process proceeds to step 1904, and the variable fig is set. The finger part specified by the value is the variable t
After generating the part operation data for starting the key release operation at the timing specified by the value of ktime, the series of processes is ended. The operation data, if had been allowed to spread out between the finger part and the adjacent finger part to start the key release operation,
The operation is also performed by narrowing the interval and returning to the relaxed state. On the contrary, otherwise, the determination is yes and the process moves to step 1905, and the finger part specified by the value of the variable fig simply starts the key releasing operation at the timing specified by the value of the variable tktime. After generating the part motion data, the series of processes is ended. According to the motion data, even if the space between the finger part for starting the key release motion and the adjacent finger part is widened, the space between them is not narrowed.

As described above, in the present embodiment, the state of the finger part after the key is released is determined in consideration of whether or not it is used for the next key depression. As a result, it is avoided that the finger part that is not used for the next key depression is left open indefinitely. Therefore, it is not necessary to simply move the finger part in the up and down (frx) direction from the start of key release to the end of the next key depression. As a result, not only can the performance movement be expressed in a more natural form, but also the movement of the finger part that causes a key depression event or a key release event, or the movement of another finger part that accompanies the movement can be made more prominent. .

In the note-off main process shown in FIG. 16,
The various subroutine processes described above are executed. Thereby, part operation data for expressing the operation of releasing the finger part of the finger part is generated in a subroutine processing unit and added to the unit operation data A. In their operation data, the number of variables
h z, one of the array variable f rx1~3, and frz is a value to be changed.

FIG. 20 is an operation flowchart of the hand x position processing executed as step 911 in the operation control data generation processing shown in FIG. Next, the x position processing of the hand will be described in detail with reference to FIG.

First, in step 2001, the finger part is moved in the lateral direction based on the key pressed by the finger part, the key to be pressed, the arrangement thereof, the length of each finger part, and the like. A rotation angle hrz around the z axis around the base point 301 (wrist) of the hand part where the maximum angle (absolute value) of the rotation angle frz is the minimum is obtained. The range where you can move your finger sideways is
Each finger is slightly different. From this, the rotation angle hrz
Is determined in consideration of the laterally movable range of each finger. This prevents each finger part from moving laterally beyond the actual lateral movement range of the finger.

By executing step 2001,
The arrangement of the hand part and each finger part on the xy plane is determined. From this, in step 2002 subsequent to step 2001, the keys that are pressed or pressed by each finger part, the arrangement of the keys, the length of each finger part, the rotation angle frz thereof, and the rotation angle of the hand part. The position hx on the x-axis of the base point 301 of the hand part is calculated based on hrz, the length La of the hand part (see FIG. 3), and the like. After that, step 2003
Move to.

In step 2003, the base point 301 of the hand part is moved to the position hx obtained in step 2002 from the timing designated by the value of the variable tkold to the timing designated by the value of the variable tktime.
If necessary, part motion data for the rotation angle hrz obtained in step 2001 is generated and added to the unit motion data A. If it is not necessary to change both the position hx and the rotation angle hrz, those part operation data are not created.

By confirming the arrangement of each finger part on the xy plane, the rotation angle frz of each finger part is determined, and the key is pressed, or the finger part to be pressed can be left extended. It is also determined whether the second joint and the third joint (the third joint needs to be bent according to the bending at the second joint) should be bent. If the finger part to be depressed is to be bent at the second joint, the angle of bending at the first joint must be made smaller than in the case of extending and pushing the key. Therefore, in the step 2003, the base point 3 of the hand part is set.
If there is a finger part whose state is to be changed by moving the rotation angle of 01 to hrz and moving it to the position hx, newly create part operation data for changing the state of the finger part as necessary. Or, the part operation data already created is also corrected.
As a result, a moving image expression is realized in which the other finger part is passed through, or the key is pushed over, or the other finger part is moved sideways according to the sideways movement of the finger part. After performing the processing of step 2003 for performing such a processing, the series of processing is ended.

When the x position processing of the above hand is executed, the variable p
Among various variables and various array variables that specify the state of each part of the finger when the event represented by the unit musical sound data specified by the value of e occurs or after the event, the variable h
The values of all but y will be fixed. The value of the variable hy is set in step 912 after the x value processing of this kind.
The y position reprocessing of the hand performed as

FIG. 21 is an operation flowchart of the sequence processing. In this sequence processing, for example, a user operates the operation unit 106 to specify a musical sound designated (selected) in advance.
When the reproduction of fingering data is instructed, the CPU 101
This is a process realized by executing a program stored in the ROM 102. The real-time processing unit 802 shown in FIG. 8 is realized by executing the sequence processing.

The operation control data B shown in FIG. 5B is stored in an area different from the tone / fingering data. As a matter of course, the number of stepwise motion data constituting the motion control data B is larger than the number of unit musical tone data constituting musical tone / fingering data, and the processing timing thereof does not normally coincide with the processing timing of the unit musical tone data. . For this reason, the motion control data B and the tone / fingering data are processed in parallel.

First, in step 2101, the variable tkt
Assigning values to various variables such as substituting 0 for imeE, tktimeM, and clock, or assigning initial values to variables pe and pm, and an interrupt output by a hard timer (not shown) at predetermined time intervals Performs initial settings such as enabling the signal and canceling execution prohibition of timer interrupt processing. By releasing the prohibition of execution, the value of the variable clock is incremented each time the timer interrupt processing is executed.

Step 2102 following step 2101
Then, an operation using the value of the variable clock and the value of the tempo data is performed to calculate a value indicating the value of the variable clock in the unit of time data in the musical tone / fingering data, and the calculation result is assigned to the variable tktime. To do. Then step 21
Move to 03.

In steps 2103 to 2109, a musical tone
The processing relating to the reproduction of fingering data is executed. First, in step 2103, it is determined whether or not the musical tone / fingering data has ended. If the data specified by the value of the variable pe is the end data located at the end of the musical tone / fingering data,
The determination is YES and the process proceeds to step 2110, and if not, the determination is NO and step 2104.
Move to.

At step 2104, the value of the time data in the unit tone data designated by the value of the variable pe is changed to the variable tk.
It is determined whether the value added to the value of timeE is greater than or equal to the value of the variable tktime. In the variable tktimeE, a value obtained by accumulating the values of time data in the unit musical sound data processed up to that point, that is, a value indicating the processing timing of the immediately preceding unit musical sound data in absolute time is substituted. Therefore, when it is time to process the unit musical sound data specified by the value of the variable pe, the determination is yes and the process moves to step 2105. Otherwise, the determination is no and the process moves to step 2110.

In step 2105, the variable tktime is changed.
The value obtained by adding the value of the time data in the unit tone data designated by the value of the variable pe to the value up to that point is substituted for E. As a result, after substituting the value indicating the timing at which the next unit time data is to be processed into the variable tktimeE, the unit tone data at the processing timing is processed according to the type of the event it represents, and Depending on the progress of the performance, the display page is switched, the display color of the note corresponding to the newly pronounced musical tone is changed, and the address value advanced by 1 event from the previous address value is added to the variable pe. Was substituted (above, step 2106-
After 2109), the process proceeds to step 2110. The processing of the unit tone data is performed by the CPU 101 by generating a command to be transmitted to the sound source 107 from the data and transmitting the command.

In steps 2110 to 2115, processing for operating each part of the finger is performed according to the operation control data B. First, in step 2110, it is determined whether the operation control data B has ended. If the staged motion data designated by the value of the variable pm does not exist in the motion control data B, the determination is yes and step 2115.
If not, the determination is no and the process moves to step 2111.

At step 2111, the value of the time data in the step motion data designated by the value of the variable pm is changed to the variable tk.
It is determined whether the value added to the value of timeM is greater than or equal to the value of the variable tktime. The variable tktimeM is, like the variable tktimeE, a value obtained by accumulating the values of the time data in the step motion data processed up to that point, that is, a value indicating the processing timing of the immediately preceding step motion data in absolute time. It has been assigned. Therefore, when it is time to process the staged motion data specified by the value of the variable pm, the determination is yes and the process moves to step 2112. Otherwise, the determination is no and the process returns to step 2102.

At step 2112, the variable tktime is changed.
The value obtained by adding the value of the time data in the stage operation data designated by the value of the variable pm to the value up to that point is substituted for M. As a result, when the value indicating the timing at which the next stage motion data should be processed is substituted into the variable tktimeM, the process proceeds to step 2113, and the state is changed in each part of the finger according to the stage motion data that has become the processing timing. A performance operation display update process is executed to change the state of the part to be performed and update the display content indicating the performance operation. After the execution, in step 2114, the address value advanced by one event from the address value so far is substituted into the variable pm, and then the process returns to step 2102. Note that the display content is updated by, for example, the CPU 101 reading from the ROM 102 the image data of a portion of the image data stored in the memory of the display unit 105 where the data should be rewritten, or using it to generate the image data. The image data thus obtained is overwritten on the portion of the memory where the image data should be rewritten.

On the other hand, the determination in step 2110 is YES.
In step 2115 where the processing shifts to S and proceeds, it is determined whether or not the musical tone / fingering data has ended. If the data specified by the value of the variable pe is the end data located at the end of the musical tone / fingering data, the determination is YES and the series of processes is terminated. Otherwise, the determination is NO. Then, the process returns to step 2102.

By executing the above-described sequence processing, an image as shown in FIG. 7 is displayed on the display unit 105, and each part of the finger displayed on it is operated as shown in FIG.

In this embodiment, as shown in FIG. 6, the hand part is basically started at the processing timing of the unit musical sound data, but this need not always be the case. As a motion during performance, for example, a lateral motion is completed before starting a key pressing motion (Sekiguchi,
Eiho: Generation and display of piano performance in virtual space,
IPSJ Research Report Vol 99, No. 16 (19
99)). From this, for example, in key depression, for example, after the movement of the hand part or the finger part is completed, the finger part is bent at the joint to start the key depression operation, and each part of the finger is divided into a plurality of phases. You may make it operate.

[0161] positioned on the base 301 Gato Ri obtained Ru z-axis of the hand part is located ZPOS1, or has been just two ways ZPOS2, its position in triplicate or more, or even be set arbitrarily good. In such a case, it becomes possible to more appropriately express the action of a finger passing through or straddling another finger.

Further, in the present embodiment, a keyboard is assumed as the performance operator group, and the performance operation for the keyboard is displayed, but the performance operation for the performance operator group other than the keyboard may be displayed. good.

In this embodiment, the sound source 107 and the display unit 105 are used.
The present invention is applied to a device equipped with, but the device to which the present invention can be applied is not limited to those equipped with them. The sound source 107 may be omitted, and the display unit 1
Also, 05 may be omitted if it can be connected to an external display device.

The program for realizing the operation of the performance operation display device may be recorded in a recording medium such as a CD-ROM, a floppy (registered trademark) disk, or a magneto-optical disk and distributed. Alternatively, a part or all of the program may be distributed using a communication line such as a public network. In such a case, the user obtains the program and selects any performance operation display device,
Alternatively, the present invention can be applied to a data processing device such as a computer by loading the device. Therefore, the recording medium may be accessible by the device that distributes the program.

[0165]

As described above, according to the present invention, the
Based on fingering data, a group of performance operators , such as the keyboard
A motion control data that indicates the state of the finger when a key is pressed or released.
A performance of a finger is displayed as a moving image based on the motion control data . Therefore, the user can easily understand the movement of the finger during the performance. In addition, the timing of the display of the finger performance operation is based on a predetermined reference time, and an event (issue
Since the timing is determined to be earlier than the (sound) timing , the finger performance operation display becomes closer to the actual one, and the user can more easily understand the finger operation during performance.

[Brief description of drawings]

FIG. 1 is a circuit configuration diagram of a performance operation display device according to the present embodiment.

FIG. 2 is a diagram illustrating a configuration of musical sound / fingering data.

FIG. 3 is a diagram illustrating a structure of fingers for display.

FIG. 4 is a diagram illustrating a movement range of a hand.

FIG. 5 is a diagram illustrating a configuration of operation control data.

FIG. 6 is a diagram illustrating an example of finger motion display.

FIG. 7 is a diagram showing an actual display example.

FIG. 8 is a functional block diagram of a performance operation display device according to the present embodiment.

FIG. 9 is an operation flowchart of operation control data generation processing.

FIG. 10 is an operation flowchart of a hand y position processing process.

FIG. 11 is an operation flowchart of z position processing of a hand.

FIG. 12 is an operation flowchart of note-on-main processing.

FIG. 13 is an operation flowchart of chord determination processing.

FIG. 14 is an operation flowchart of chord operation processing.

FIG. 15 is an operation flowchart of a note-on operation process.

FIG. 16 is an operation flowchart of a note-off main process.

FIG. 17 is an operation flowchart of chord off determination processing.

FIG. 18 is an operation flowchart of chord off operation processing.

FIG. 19 is an operation flowchart of a note-off operation process.

FIG. 20 is an operation flowchart of x position processing of a hand.

FIG. 21 is an operation flowchart of sequence processing.

[Explanation of symbols]

101 CPU 102 ROM 103 RAM 104 Data input section 105 display 106 Operation part 107 sound source 108 sound system 300 keyboards 301-306 base point 801 Data generation unit 802 Real-time processing unit 803 Operation control data generation unit 804 Data transmission unit 805 operation display 806 sound source section 807 score display

Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G10H 1/00 101-102 G10G 1/00-1/02 G09B 15/00

Claims (4)

(57) [Claims] 1. A device for displaying an operation when a performer operates a performance operator group, which is an object of a performance operation, in accordance with the progress of a performance, and is generated on the performance operator group. Musical tone / fingering data obtained from event data representing the content of an event to be caused, time data indicating the timing of occurrence of the event, and finger data designating a finger to be used to generate the event finger data acquisition means, based on the obtained tone-fingering data in said musical tone-fingering data acquisition unit, performance operator in the performance operator group
When the operation of the above is completed or the release of the operation is completed
When complete an operation control data generating means for generating an operation control data representing the state of the finger, the operation of the performance operator represented by the operation control data that is generated by the operation control data generation means when
The operation start timing that should start the transition to the finger state of
Indicates the time required to complete the operation of the performance operator.
Operation timing determining means for determining based on a predetermined reference time, operation control data from the operation control data generating means, and
Based on the operation start timing from the operation timing determining means, musical performance display device characterized by comprising, an operation display means operating for the video display of the finger to continue to operate the performance operator group. 2. A pair of performance operators that are objects of performance operation.
However, when the performer operates as the performance progresses
Is a device for displaying the contents of the event to be generated on the performance operator group.
The event data that represents and the occurrence timing of the event
The time data indicated and the finger that should be used to generate the event
Tones and fingering data composed of finger data that specifies
The musical sound / fingering data acquisition means to be acquired and the musical sound / fingering data acquired by the musical sound / fingering data acquisition means.
Based on finger data, the performance operators in the performance operators group
When the operation of the above is completed or the release of the operation is completed
Motion to generate motion control data that represents the state of the finger when
Control data generation means and operation control generated by the operation control data generation means
When the operation of the performance operator represented by the data is completed
The operation start timing that should start the transition to the finger state of
Finger data that is the same as the finger used when operating the performance operator
Data in musical tone / fingering data immediately before having
And the time required to complete the operation of the performance operator.
First reference time to represent and release of operation of the performance operator
To a second reference time that represents the time it takes to complete
Operation timing determining means for determining the operation timing based on the operation control data from the operation control data generating means,
Operation start timing from the operation timing determination means
The movement of the finger operating the performance control group based on
A performance operation display device , comprising: a motion display means for displaying a moving picture . 3. A computer should be caused to generate on a group of performance operators to be operated for performance.
Event data that represents the content of the event,
Time data indicating the timing of occurrence and the occurrence of the event
An easy operation that consists of finger data that specifies the fingers that should be used live.
Based on the acquisition procedure to acquire the sound / fingering data and the musical sound / fingering data acquired in the acquisition procedure
Complete the operation of the performance operator in the performance operator group.
Time, or the state of your finger when the release of the operation is completed
A generating step of generating an operation control data representing a table with an operation control data generated by said generation step
Move to the finger state when the operation of the performance controller is completed.
The operation start timing for starting the line
A predetermined reference time that represents the time required to complete the operation of
A decision procedure that is decided based on the interval, the operation control data generated in the generation procedure, and the decision procedure.
Based on the operation start timing determined in order,
Display that displays a movie of finger movements operating the controls
A computer in which the procedures and programs for executing the procedures are recorded
A readable recording medium . 4. A computer is a target of performance operation.
Shows the contents of events that should occur on the performance control group.
To event data, shows the generation timing of the event
Specify the time data and the finger that should be used to generate the event.
Acquires tone and fingering data composed of finger data to be set
Based on the acquisition procedure and the tone / fingering data acquired in the acquisition procedure
Complete the operation of the performance operator in the performance operator group.
Time, or the state of your finger when the release of the operation is completed
A generating step of generating an operation control data representing a table with an operation control data generated by said generation step
Move to the finger state when the operation of the performance controller is completed.
The operation start timing for starting the line
Just before having the same finger data as the finger used when operating
Time data in tone and fingering data, and performance operation
A first criterion that represents the time required to complete a child operation
It takes time and the time to complete the release of the operation of the performance operator.
Based on a second reference time that represents the time to
Decision procedure, operation control data generated in the generation procedure, and the decision procedure
Based on the operation start timing determined in order,
Display that displays a movie of finger movements operating the controls
A computer in which the procedures and programs for executing the procedures are recorded
A readable recording medium .