JP2019179158A - Fingering display program - Google Patents

Abstract

To enable drawing fingering moving image in which unnatural movement of slow moving of a fingering moving image is removed even when time from key releasing to next key pressing is long in musical instrument training fingering display.SOLUTION: When fingering drawing is generated on the basis of fingering position information corresponding to reproduction time (T) of music reproduction time, time (T2) which is nearest key pressing time after reproduction time (T) and time (T1) just before changing to key pressing information at the time (T2) are set, and when time from the time (T1) to the reproduction time (T) is longer than desired time, a computer is executed to generate fingering position information (Pm) in which a transition to a fingering position corresponding to the key pressing information at the time (T2) is completed until the reproduction time (T) expends predetermined time from the time (T1).SELECTED DRAWING: Figure 11

Description

  The present invention relates to a fingering display for practicing performance of a keyboard instrument, and draws a fingering video for playing a musical instrument during musical tone reproduction based on performance information composed of a plurality of musical sounds and finger number information corresponding to each musical tone. The present invention relates to a fingering display program to be displayed on the screen.

When practicing performance with a keyboard instrument such as an electronic piano, practicing finger use to press and release the keyboard by displaying a fingering image composed of the back of the hand and each finger over the keyboard display to be operated There has been proposed a performance operation display device for instructing the user.
For example, Patent Document 1 discloses a fingering video generation technique in which hand movement continuously shifts in order to make it easy to understand how to carry a finger. To determine the shape of each finger and the position of the back of the hand.

  That is, in Patent Document 1, a plurality of types of operation fingering image data and a plurality of types of progress fingering image data are stored in a ROM, and an operation finger corresponding to a fingering indicating the start of key operation for note-on processing. Progressive fingering image data corresponding to the fingering indicating the progress until the start of the operation of the key to be pressed next to the note-off process after the image data is selected from the ROM and read out and given to the display device Describes a fingering display device that performs fine performance guidance according to the progress of a song by selecting and reading out from a ROM and giving it to a display device for display.

  In addition, the applicant has proposed the technique disclosed in Patent Document 2 as a fingering display method capable of drawing a bill that matches the keyboard size and the keyboard display size.

Japanese Patent No. 3521838 Japanese Patent No. 4070100

  However, in the fingering display method in the fingering display device disclosed in Patent Document 1, when the time from the release of the key to the next key depression is long to some extent, the fingering moves continuously between them. Since a moving image is generated, there is a problem that the moving hand slowly spreads slowly in a long section with fingers spread out slowly in slow motion.

  The present invention has been proposed in view of the above circumstances, and when a fingering video is generated, even if the time from when the key is released until the next key is pressed is long, the fingering video is slowly An object of the present invention is to provide a fingering display program capable of drawing a fingering moving image close to an actual keyboard operation by eliminating an unnatural movement that moves.

In order to achieve the above object, the present invention is a program for drawing a fingering video for playing a musical instrument during music playback based on performance information including each musical tone of the music and the corresponding finger number and displaying the fingering on the screen. And
When generating the fingering drawing based on the fingering position information (Pm) corresponding to the playback time (T) at the time of the music playback,
A time (T2) that is the nearest key press time after the playback time (T) from the key release time when the key press changes to the key release at the playback time (T);
A time (T1) immediately before the key press information at the time (T2) is changed;
When the time from the time (T1) to the time (T2) is longer than the desired time, the time (T2) is not pressed until the predetermined time elapses from the time (T1). It is characterized by causing the computer to generate fingering position information (Pm) that completes the transition to the fingering position corresponding to the key information.

Claim 2 is the fingering display program according to claim 1,
Based on a single key pressing information including at least a time before the reproduction time (T) and a time after the reproduction time (T), and synthesized key pressing information obtained by combining the key pressing information at a plurality of different times. Thus, the fingering position information (Pm) is generated.
Claim 3 is the fingering display program according to claim 2,
The plurality of different times are
A time (T1) immediately before the key press information at the time (T2) is changed;
It includes a time (T4) immediately after the key press information at the time (T2) is changed.

Claim 4 is the fingering display program according to claim 3,
The synthetic key press information is
Key press information (FK12) obtained by combining the key press information (FK1) at the time (T1) and the key press information (FK2) at the time (T2);
Key press information (FK24) obtained by combining key press information (FK2) at the time (T2) and key press information (FK4) at the time (T4) is included.

Claim 5 is the fingering display program according to claims 1 to 4,
The key depression information is characterized by the key depression position (keyboard position) and the key depression state (key depression or key release) of each finger at the time.

Claim 6 is the fingering display program according to any one of claims 1 to 5,
The fingering position information (Pm) includes at least the center position of the back of the hand and the tip position of each finger.

Claim 7 is the fingering display device according to any one of claims 1 to 6,
The fingering position information (Pm) is
Fingering position information (Ps) at the starting point of fingering at time Ts before the reproduction time (T);
Fingering position information calculated by interpolating fingering position information (Pe) at the end of fingering at time Te after the reproduction time (T) according to the passage of time from time Ts to time Te. It is characterized by that.

  According to the present invention, the time (T2) that is the nearest key pressing time after the reproduction time (T) from the key release time that changes from the key press to the key release at the reproduction time (T), and the key press at the time (T2). After setting the time (T1) immediately before changing to the key information, detecting whether the time from the time (T1) to the time (T2) is longer than the desired time, and then releasing the key at the time (T1) By determining the elapsed time, the fingering image can be generated so that the movement of the hand is completed as much as possible before the next key pressing time within that time.

It is a functional block diagram which shows the structure of the fingering display apparatus of this invention. It is an example of a display of a score, a keyboard, and a fingering image displayed on a display part of a fingering display device. It is a time chart for demonstrating the key pressing time and key release time with respect to a key. It is a flowchart of fingering drawing production | generation in the fingering display program of this invention. It is a flowchart of fingering position information generation in the fingering display program of the present invention. It is a figure which shows the score example. It is a piano roll figure which displays the key pressing information corresponding to the score of FIG. It is a correspondence figure which shows the fingering image generated in consideration of the keypress information of the start point and the end point with respect to the reproduction time (current) (state A to state J). It is a correspondence figure which shows the fingering image generated in consideration of the keypress information of the start point and the end point with respect to the reproduction time (current) (state K to state U). It is a display example (conventional method) of a fingering image drawn on a score and a keyboard. (A) And (b) is a display example (the method of the present invention) of a fingering image by preceding movement drawn on a score and a keyboard.

Hereinafter, the fingering display program of the present invention will be described with reference to the drawings.
The fingering display device installs a fingering display program on a known tablet terminal having a CPU, a ROM, a RAM, a hard disk, etc. in order to display the musical score data and the corresponding fingering display. Built. The tablet terminal includes a display unit that can display a score and a keyboard, a touch panel having an input unit for inputting a touch operation, and a sound output device such as a speaker that reproduces sound. On the display unit, a fingering image of a bill is displayed together with the keyboard, and the user can practice fingering with a keyboard instrument or the like using the displayed fingering image.
The CPU of the fingering display device executes various processes according to the installed predetermined control program (fingering display program), and controls the entire fingering display device.
Also, a MIDI device can be connected to the tablet terminal via a MIDI interface. The performance data and fingering drawing data can be stored in the ROM in the tablet terminal, or can be provided from the outside via a MIDI interface or by a recording medium.

  As an example of use of a fingering display device, for example, a digital piano is connected to a digital piano, and the musical score data recorded on the device is displayed and the performance data corresponding to the musical score data is transmitted to the digital piano side to thereby generate the sound source of the digital piano. Play a digital piano while playing a demo performance or watching the score displayed on the fingering display device, input the performance result to the device side, and correspond to the score data displayed on the device side The performance state is confirmed (evaluated) by the user by comparing with the model performance data.

For example, a fingering display device constructed by installing a fingering display program stored in a recording medium in order to execute processing according to the present invention for a tablet terminal will be described with reference to the functional block diagram of FIG. To do.
The fingering display device includes a musical tone / finger number information storage unit 11 storing musical tone information for performing performance and finger number information for performing fingering display corresponding to each musical tone, and the musical tone / finger number information. A key pressing information generating unit 12 that generates key pressing information; a key pressing information storage unit 13 that stores key pressing information; a fingering position information generating unit 14 that generates fingering position information from the key pressing information; And a fingering image generation unit 15 that generates a fingering image from the position information.
In addition, a score image generating unit 16 that generates a score image from score information included in the fingering information is provided, and images generated by the fingering image generating unit 15 and the score image generating unit 16 are stored in the image information storage unit 17. And displayed on a display device 18 such as a display.

  In addition, the fingering display device includes a reproduction information generation unit 21 that reproduces reproduction information for reproducing music from the musical tone information stored in the musical tone / finger number information storage unit 11, and a reproduction information storage unit that stores the reproduction information. 22, a sound generation unit 23 that generates sound from reproduction information, a sound reproduction unit 24 that converts sound into an analog signal, and a sound output device 25 such as an amplifier and a speaker.

  The musical tone / finger number information storage unit 11 stores at least information necessary for music reproduction and fingering drawing such as musical notes constituting a musical score and finger numbers corresponding to the musical sounds.

The key press information generation unit 12 generates key press information at an arbitrary time from the beginning to the end of the music based on the information stored in the musical tone / finger number information storage unit 11. As the key pressing information, information on the target time and the hand and finger number, volume, note number, and presence / absence of a special finger usage flag for each musical tone being pressed at that time are created. Special fingering includes finger change, finger penetration and finger touch in the performance technique.
The key pressing information created by the key pressing information generation unit 12 is stored in the key pressing information storage unit 13.

  If the key pressing information stored in the key pressing information storage unit 13 includes a special finger usage, the finger pressing display is generated by the key pressing information generation unit 12 as requiring a special finger usage. The special finger usage information generation unit 31 generates special finger usage information from the key press information and stores it in the special finger usage information storage unit 32. Whether or not the fingering display includes a special finger operation is determined from the key pressing information generated by the key pressing information generation unit 12.

  The fingering position information generation unit 14 generates fingering position information Pm at the reproduction time (the time of the current reproduction position from the beginning of the music) T based on the information stored in the key pressing information storage unit 13. The fingering position information Pm includes a single key press information including a time before the reproduction time T and a time after the reproduction time T, and each key press information at a plurality of different times (the time before the reproduction time T). It is generated based on the combined key pressing information obtained by combining the fingering position information Ps and the fingering position information Pe) at times after the reproduction time T. A specific procedure for generating the fingering position information Pm based on the fingering position information Ps and the fingering position information Pe will be described later.

The fingering position information Pm is information for drawing the handprint and finger shape in the fingering image. As fingering position information, the position data of the center of the back of the hand, the position data of the base of each of the five fingers, the position data of the two points of the base of the wrist (the thumb side and the little finger side), and the position data of the tip of each finger Including. Then, by giving these pieces of information, a fingering image can be drawn. In addition, in order to determine the position of each part of the bill with respect to the key, data representing the positional relationship between the position data and the key may be given.
As a technique for obtaining a fingering image from fingering position information, for example, a known technique shown in Patent Document 1 (Japanese Patent No. 3521838) or Patent Document 2 (Japanese Patent No. 4070100) described above is used.

  The fingering image generation unit 15 generates a fingering image at the reproduction time T based on the information created by the fingering position information generation unit 14. The fingering image is created from information regarding the shape of the hand and the coordinates of the finger position according to the fingering position information.

  The musical score image generating unit 16 generates a musical score image based on the information stored in the musical tone / finger number information storage unit 11.

  The fingering image created by the fingering image generation unit 15 and the score image created by the score image generation unit 16 are stored in the image information storage unit 17 as image information.

In the display device 17, the image information stored in the image information storage unit 17 is displayed.
That is, based on the key pressing information corresponding to the musical tone of the score, a fingering moving image for playing the musical instrument at the time of music playback is displayed on the screen. As a specific image display, as shown in FIG. 2, a musical score is displayed on the upper stage, and a fingering image for playing each tone of the keyboard and the musical score with both hands is displayed on the lower stage. In the musical score part, a bar BAR corresponding to the reproduction time T is displayed by moving the staff part according to the progress of the performance, and the fingering image draws the key depression state at the reproduction time. The key that is pressed changes its color (indicated by hatching in FIG. 2).
Further, performance information such as a piano roll may be displayed instead of the score.

The reproduction information generation unit 21 generates reproduction information of musical sounds based on the information stored in the musical tone / finger number information storage unit 11. As the reproduction information, MIDI information of the sound generation start time and sound generation end time, sound volume, and note number of each musical tone is created.
The reproduction information created by the reproduction information generation unit 21 is stored in the reproduction information storage unit 22.
The sound generator 23 generates sound information composed of analog signals according to the reproduction information.

In the sound reproducing unit 24, an analog signal corresponding to the sound information is reproduced. In addition, in the key press information generation by the key press information generation unit 12, time information from the sound playback unit 24 is input so that key press information corresponding to the music playback time is generated.
The sound output device 25 reproduces sound by outputting an analog signal serving as sound information via an amplifier and a speaker.

Next, a processing procedure for creating a fingering drawing by the above-described fingering display program will be described with reference to the time chart for the keyboard in FIG. 3 and the flowcharts in FIGS. 4 and 5.
In the fingering display device, the stored fingering display program generates fingering position information based on the key pressing information, and generates a fingering image based on the fingering position information and draws it on the display unit.

In the fingering display program of the present invention, control is performed so that the movement of the back of the hand in the keyboard width direction is reduced and a fingering moving image close to the actual keyboard operation is drawn. Therefore, when the key pressing time and key releasing time of the keyboard are as shown in FIG. 3, for example, with respect to the reproduction time T at the time of music reproduction, the fingering image is based on the fingering position information Pm corresponding to the reproduction time T. When generating, based on the single key press information including the time before the playback time T and the time after the playback time T, and the combined key press information obtained by combining the key press information at a plurality of different times. Fingering position information Pm is generated.
The fingering position information Pm is composed of fingering position information (the root and tip coordinates of each finger, the center coordinates of the back of the hand) at the reproduction time T.

  The fingering position information Pm is calculated by interpolating from the fingering position information Ps at the start point time and the fingering position information Pe at the end point time, for example. When the reproduction time is a key release time, the start point time is the key press time T1 immediately before the nearest key release before the reproduction time in FIG. 3, and the end point time is the nearest key press time T2 after the reproduction time. Become. On the contrary, when the reproduction time is a key depression time, the start point time T1 and the end point time T2 are the same as the reproduction time T, and no interpolation is necessary.

  Next, the fingering position information generation unit 14 and the fingering image generation unit 15 generate fingering position information Ps at the start point time and fingering position information Pe at the end point time. A procedure for obtaining the fingering image will be described with reference to the fingering drawing flowchart of FIG. 4 and the fingering position information generation flowchart of FIG.

The fingering image is created by pressing the nearest T1, T2, T3, and T4 times centering on time T (reproduction time: time of the current reproduction position from the beginning of the music as the music reproduction position) in FIG. The key information is generated in consideration of necessity.
T1 is a time corresponding to the nearest time immediately before key release before T. It may be a predetermined time before the key release time. This is because if the time from the key release time to the next key press time is short, the next fingering display will be switched instantly, so the fingering display will transition smoothly by setting the time before the predetermined time. It is.
T2 is the nearest key press time after T when T is released, and the nearest key press time before T when T is pressed.
T3 is the nearest key release time after T2 (the time corresponding to the time when the key is released next to the T2 key press, for example, the time before the key release time is a predetermined time).
T4 is the nearest key pressing time after T2 (time pressed after T2 key pressing).

In the case where the key (square portion) is pressed and released as shown in FIG. 3, T1, T2, T3, and T4 are indicated by arrows with respect to key on (key depression) off (key release). Position (time).
As for the key release time, as described above, in order to make the fingering transition as smooth as possible, the time before the actual key release is associated with a predetermined time.

The fingering display device stores information FI regarding the shape of each finger and the back of the hand, which is the relative length with respect to the keyboard width and the relative position from the back of the hand (step 51).
Based on the reproduction time (music reproduction position) T, the nearest T1, T2, T3, and T4 are scanned and read from the key press information and key release information for the musical score (step 52).

Single key press information FK1, FK2, and FK4 corresponding to T1, T2, and T4 are generated (step 53).
FK1 is key depression information (key depression flag and key number of each finger) at time T1.
FK2 is key pressing information (key pressing flag and key number of each finger) at time T2.
FK4 is key depression information (key depression flag and key number of each finger) at time T4.

  Composite key press information FK12 obtained by combining FK1 (key press information at time T1) and FK2 (key press information at time T2), and FK2 (key press information at time T2) and FK4 (key press information at time T4) The synthesized key pressing information FK24 synthesized is generated (step 54).

It is determined whether or not the music playback position is before the first key depression (whether T1 is invalid and T <T2) (step 55).
When the music playback position is before the first key press, the fingering information Ps (creation of fingering start point coordinates) is determined from FK1 (key press information at time T1) and FI (step 56), and the music playback position is the first. It is determined whether or not a predetermined time or more before the key is pressed (whether T1 is invalid and T <T2-Tmin) (step 57). Tmin is a time for determining whether the time is a predetermined time or more before the key is pressed, and is set to “96” (delta time corresponding to the length of a quarter note) in this example.
If the music playback position is a predetermined time or more before the first key press, FK1 (key press information at time T1) is set as fingering information Pe (creation of fingering end point coordinates) (step 58).

The fingering information Ps is fingering position information (the root coordinates / tip coordinates of each finger and the center coordinates of the back of the hand) at the fingering start time.
The fingering information Pe is fingering position information (the root coordinates / tip coordinates of each finger and the center coordinates of the back of the hand) of the fingering end time.

  In step 57, if the reproduction time T is not a predetermined time or more before the key press, it is determined whether or not the fingering information Pe can be generated by the FK 24 (step 59). This is determined based on whether or not the fingering information of FK2 and FK4 can be combined. When the key pressing position of each finger in the combined key pressing information is equal to or greater than a predetermined range, it is determined that the combining is impossible. If the fingering information of FK2 and FK4 can be combined, fingering information Pe is generated from the key pressing information of FK24 (step 60). If synthesis is impossible, fingering information Pe is generated from the key press information of FK2 (step 61). The FK 24 is key pressing information (key pressing flag and key number of each finger) obtained by combining the key pressing information FK2 and FK4.

  HFs, which are special finger usage information at the fingering start point (each flag of finger change, finger pass, finger touch), and special finger use information at the fingering end point (whether finger change, finger pass, finger touch) HFe which are the respective flags) are generated (step 62).

Subsequently, it is determined whether or not the reproduction time T is during key depression (step 63).
When the key is being pressed, the key pressing information Ps at the fingering start point and the key pressing information Pe at the fingering end point have the same value.
When the key is being pressed at the reproduction time T, it is determined whether the time from T3 (the closest key release time after T2) to T4 (the closest key press time after T2) is shorter than a preset threshold time Tmin. (Step 64). Tmin is a threshold value of an allowable time for combining FK2 and FK4, and is set to “96” determined in advance in this example.
If the time from T3 to T4 is shorter than the threshold time Tmin, it is determined whether fingering position information Ps (fingering position information Pe) can be generated based on FK24 (synthetic key pressing information) and FI (step 65).

If it is determined in step 65 that it can be generated, fingering information Ps / Pe is generated based on FK24 (combined key pressing information) and FI (step 66).
Subsequently, using the special finger usage information HFs / HFe generated in step 62, the fingering information (start point coordinates) Ps at the start point and the fingering information (end point coordinates) Pe at the end point are corrected (step 67).

  If the time from T3 to T4 is longer than the threshold time Tmin in step 64, it is determined that it is not necessary to consider the key pressing information at a later time, and FK2 (key pressing information at time T2) and FI are used as the starting points. Fingering information Ps (fingering position information Pe at the end point) is generated (step 68). Also, if it is determined in step 65 that the generation is impossible, it is determined that the key pressing position of the finger to be synthesized is separated, and the fingering position information at the start point from FK2 (key pressing information at time T2) and FI. Ps (fingering position information Pe at the end point) is generated (step 68). Thereafter, the coordinates of the fingering position information Ps at the start point and the fingering position information Pe at the end point are corrected by the special finger usage information HFs / HFe (step 67).

  When the fingering information Ps or Pe is generated in each step, the fingering position information generating unit 14 of the fingering display device shown in FIG. 1 creates fingering position information. In this case, FIG. The procedure shown in the flowchart of FIG.

First, key pressing finger coordinates are calculated (step 41).
Next, it is determined whether the X-coordinate distance of each finger during key pressing is within the maximum finger interval tolerance (step 42).

  If the finger spacing between the key pressing fingers is within the allowable value, the coordinates of each finger being released are calculated (step 43), the center coordinate of the back of the hand is calculated (step 44), and the Y coordinate of each key pressing finger is calculated. It is determined whether or not (the depth direction of the keyboard) is within the key area corresponding to each key pressing finger (step 45).

If the Y coordinate of each key is outside the key area, all the coordinates are moved relative to each other by a distance so that the Y coordinates of all the key fingers are within the key area and maximally toward the front of the keyboard. (Step 46), fingering position information is confirmed.
If the finger interval exceeds the allowable value in step 43, the process returns without obtaining fingering position information.

Returning to the flowchart of FIG. 4, if the reproduction time T is not being depressed in step 63, it is determined whether T4 is invalid and T is equal to or greater than T2 (there is no next key depression) (step 70).
If T4 (the nearest key pressing time after T2) is not found (there is no next key pressing), the fingering start point coordinate Ps (= end point coordinate Pe) is generated from FK1 (key pressing information at time T1) and FI. (Step 71) The start point coordinates Ps and the end point coordinates Pe are corrected by the special finger usage information HFs / HFe (Step 67).

  If T4 (the closest key press time after T2) is found in step 70 (there is the next key press), T1 {the time corresponding to the nearest key release time before T (the time before the key release time is a predetermined time) Etc.)} to T2 (when T is released, the nearest key pressing time after T, when T is pressed, the nearest key pressing time before T) is shorter than a preset threshold time Tmin Is determined (step 72). Tmin is a threshold value of an allowable time for combining FK1 and FK2, and is set to “96” determined in advance in this example.

When the time from T1 to T2 is shorter than the threshold time Tmin, the fingering information Ps (based on the FK12 {key pressing information obtained by combining the key pressing information FK1 and FK2 (key pressing flag and key number)} and FI It is determined whether or not fingering position information at the starting point of the fingering can be generated (step 73).
If it can be generated, fingering position information Ps at the starting point of fingering is generated from FK12 and FI (step 74).

If the time from T1 to T2 is longer than the preset threshold time Tmin in step 72, it is determined whether the time from T1 to T (reproduction time) is shorter than the preset threshold time Tmin. (Step 75).
That is, if the time from T1 to T2 in step 72 is longer than the desired time, the fingering image of the preceding movement is displayed. If “Y” in step 75, the fingering image is in the preceding movement, step 75 In the case of “N”, the preceding movement is finished, and the next key depression state is entered.
If it is determined in step 73 that fingering information Ps (fingering position information at the starting point) cannot be generated, or if the time from T1 to T is longer than the preset threshold time Tmin in step 75 , FK1 (key press information at time T1) and FI are generated from the finger start point coordinates Ps (step 76).

When the starting point coordinate Ps of the fingering is generated in step 74 or step 76, from T3 {T2 and the nearest key release time (time corresponding to the time when the key is released next to T2 key press)} to T4 {T2 and later It is determined whether the time until the nearest key pressing time (time pressed after T2 key pressing)} is shorter than Tmin which is a preset threshold time (step 77). Tmin is a threshold value of an allowable time for combining FK2 and FK4, and is set to “96” determined in advance in this example.
If the time from T3 to T4 is shorter than the threshold time Tmin in step 77, fingering is performed based on FK24 {key pressing information obtained by combining key pressing information FK2 and FK4 (key pressing flag of each finger, key number)} and FI. It is determined whether information Pe (fingering position information at the end point) can be generated (step 78).
If it can be generated, the fingering position information Pe at the end point is generated from the FK 24 and FI (step 79), and the fingering position information Ps at the starting point and the fingering position information at the end point are obtained from the special finger usage information HFs / HFe. Each coordinate of Pe is corrected (step 67).

  If the time from T3 to T4 is longer than the threshold time Tmin in step 77, or if it is determined in step 78 that the fingering information Pe cannot be generated, the fingering position information Pe at the end point is obtained from FK2 and FI. Then, the coordinates of the fingering position information Ps at the start point and the fingering position information Pe at the end point are corrected by the special finger usage information HFs / HFe (step 67).

If the time from T1 to T is longer than the threshold time Tmin in step 75, it is determined whether the fingering position information Pe can be generated from the FK 24 (step 82). Finger position information Ps and Pe are generated (step 83). If not, fingering position information Ps and Pe are generated from FK2 (step 84). In this case, since it is in the key press waiting state, the fingering information Ps and the fingering information Pe are the same.
Then, the coordinates of the fingering position information Ps at the start point and the fingering position information Pe at the end point are corrected by the special finger usage information HFs / HFe (step 67).

After the start point coordinates of the fingering position information Ps and the end point coordinates of the fingering position information Pe are corrected in step 67, the reproduction time T is determined from the start point coordinates Ps (Psx, Psy) and the end point coordinates Pe (Pex, Pey). The movement interpolation coordinate Pm (Pmx, Pmy) at is generated (step 90).
The fingering position information Pm is composed of fingering position information (the root and tip coordinates of each finger, the center coordinates of the back of the hand) at the reproduction time T.

  When the end point coordinate Pe is generated in step 57, step 59, and step 60, since it is not based on the combined key pressing information obtained by combining the key pressing information at a plurality of different times, finger change, finger penetration, finger touch Therefore, the movement interpolation coordinates Pm (Pmx, Pmy) at the reproduction time T are generated from the start point coordinates Ps (Psx, Psy) and the end point coordinates Pe (Pex, Pey) (step 90).

The X coordinate (Pmx) and Y coordinate (Pmy) of the movement interpolation coordinate Pm are calculated by the following equations (1) and (2), where T is the start point time Ts, end point time Te, and the current time is T.
Pmx = Psx + (Pex−Psx) × (T−Ts) ÷ (Te−Ts) (1)
Pmy = Psy + (Pey−Psy) × (T−Ts) ÷ (Te−Ts) (2)

Next, drawing information is generated from the movement interpolation coordinates Pm generated in step 90 (step 91).
Subsequently, a fingering image is drawn based on the drawing information generated in step 91 (step 92), and the fingering drawing process is terminated.

  In the above-described flowchart, Tmin which is a threshold value of the allowable time for combining FK2 and FK4 in Step 56, Step 64 and Step 77, and a threshold value of the allowable time for combining FK1 and FK2 in Step 72 and Step 75 Tmin is set to “96”, which is the same time interval, but may be set to a unique time interval. The reason why Tmin is not a real time but a delta time is to make the time short when the tempo is fast and long when the tempo is slow.

Next, an example of a fingering image generated corresponding to a specific score will be described with reference to FIGS.
FIG. 6 is a musical score showing the first bar to the seventh bar of the song “For Elise”, with the upper part being the right-hand performance part and the lower part being the left-hand performance part. FIG. 7 shows the piano roll of the left-hand performance part corresponding to the score of FIG.

  In the piano roll of FIG. 7, the bars progress from the bottom to the top, the keys (note numbers 36 to 59) that are part of the keyboard are displayed in the left-right direction, and are displayed in a square shape in the upper area of each key. Indicates that the key is pressed. Therefore, the lower end position of the rectangular portion is the key press start position, the upper end position is the key release position, and the number displayed at the lower end position corresponds to the key press time (for example, the note number 45 displayed at the bottom row). The number displayed at a substantially central position is a time before the key release time by a predetermined time (for example, the time 202 before the key release of note number 45).

8 and 9 are correspondence tables showing fingering images generated from key pressing information at an arbitrary reproduction time T in the first to seventh bars of the score of FIG.
In FIG. 8, the reproduction time T is “time 0”, “time 144”, “time 192”, “time 210”, “time 216”, “time 234”, “time 240”, “time 288”, “time 336”, and “time 354”. In the states A to J, a fingering image is generated from the fingering information (fingering position information) Ps at the start point time and the fingering information (fingering position information) Pe at the end point. The fingering position information includes time (delta time) and key pressing information (note number for the finger to be pressed, special finger usage information). Information on special finger usage is managed by the presence or absence of a flag.
In FIG. 9, the reproduction time T is “time 360” “time 378” “time 384” “time 432” “time 480” “time 498” “time 504” “time 522” “time 528” “time 588” “time” 768 "is shown in the fingering images for each of the states K to U.

The generation of fingering images in each state will be described sequentially from state A (reproduction time T = 0) in FIG.
When the reproduction time T is “0”, T1 is invalid (no time corresponding to the nearest key release time before T exists) in the scan of step 52 in the flowchart of FIG. 4, and T2 is the reproduction time T. Since it is time to release the key, “time 192”, which is the nearest key pressing time after T, “time 202”, which is the nearest key releasing time after T2, and T4, which is the nearest key pressing time after T2. Time 216 ".
If the reproduction time T is “0”, T1 is invalid and T <T2 is satisfied, so step 54 is “Y”, and the fingering position information Ps at the start point is FK1 (step 56). When the condition is satisfied, the fingering position information Pe at the end point is also FK1 (step 58). In this case, FK1 serves as key press information of the initial image of the home position because T1 does not exist.
The fingering position information Pm generated at step 90 at the reproduction time T is a combination of the fingering position information Ps and the fingering position information Pe. The fingering image drawn in step 1 is also an initial image.

In the state B in which the reproduction time T is “144”, T1 is invalid and T2 is the time at which the reproduction time T is released in the scanning of step 52 in the flowchart of FIG. A certain “time 192”, T3 becomes “time 202”, and T4 becomes “time 216”.
When the reproduction time T is “144”, as in the case of T = 0, T1 is invalid and T <T2 is satisfied, so that step 55 becomes “Y” and the fingering position information Ps at the start point is It becomes FK1 (step 56).

  In step 57, since Tmin is set to “96”, the condition of step 57 is not satisfied, so “N” is determined, and it is determined whether Pe can be generated by the FK 24 (step 59). In this case, since it is possible to simultaneously press the note number 45 with the little finger and the note number 52 with the index finger, it is determined that Pe can be generated at the FK 24, and the fingering position information Pe at the end point becomes the FK 24 (step 60). In this case, the FK 24 is combined key pressing information of FK2 and FK4 in which the note number 45 is pressed with the little finger and the note number 52 is pressed with the index finger.

  The fingering position information Pm at the reproduction time T generated in step 90 is a combination of the fingering position information Ps of the initial image and the fingering position information Pe of the FK 24, and the fingering drawn by the fingering position information Pm. The image is a moving image before the note number 45 is depressed, which moves from the initial image (home position) to a chord-shaped bill for depressing the note numbers 45 and 52.

  When the reproduction time T is “192”, T1 is invalid and T2 is the key depression time in the scan of step 52 in the flowchart of FIG. A certain “time 192”, T3 becomes “time 202”, and T4 becomes “time 216”.

When the reproduction time T is “192”, T = T2 and T <T2 is not satisfied, so that the determination at step 55 is “N”. FK12 and FK24 have already been generated in step 54.
At time 192, the key is being pressed (step 63), T4-T3 = 14 <Tmin = 96 (step 64), and it is determined whether Fs 24 can generate Ps and Pe (step 65). Since the note number 45 and the note number 52 can be simultaneously pressed with the index finger, Ps and Pe at the FK 24 can be generated, and the fingering position information Ps at the start point and the fingering position information Pe at the end point are FK24 ( Step 66). In this case, the FK 24 is combined key pressing information of FK2 and FK4 in which the note number 45 is pressed with the little finger and the note number 52 is pressed with the index finger.
The fingering position information Pm generated at step 90 at the reproduction time T becomes the fingering position information Ps (Pe) of the FK 24, and the fingering image drawn with the fingering position information Pm is obtained by pressing the note numbers 45 and 52. This is a fingering image in which a key-pressed display is performed on the note number 45 with a key-like chord-shaped bill.

  In the state D in which the reproduction time T is “210”, T1 is “time 202” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time to release the key, “time 216” which is the nearest key pressing time after T, “time 226” which is the nearest key releasing time after T2, and T4 is the nearest key pressing time after T2. Time 216 ".

If the reproduction time T is “210”, T1 exists, so the determination in step 55 is “N”.
The key is being released at time 210 (step 63), and the condition of step 70 (T4 is equal to T2 but T (time 210) is not equal to or greater than T2 (time 216)) does not satisfy the condition of step 72 (T2- Since T1 = 14 <Tmin = 96) is satisfied, it is determined that Ps can be generated by FK12 because it is possible to simultaneously press note number 45 with the little finger and note number 52 with the index finger (step 73). The position information Ps is FK12 (step 74). In this case, the FK 12 is combined key press information of FK1 and FK2 in which the note number 45 is pressed with the little finger and the note number 52 is pressed with the index finger.
Further, since it is possible to simultaneously press the note number 52 with the index finger and the note number 57 with the thumb, it is determined that Pe can be generated by the FK 24, and the fingering position information Pe at the end point becomes the FK 24 (step 79). In this case, the FK 24 is combined key press information of FK2 and FK4 in which the note number 52 is pressed with the index finger and the note number 57 is pressed with the thumb.

  The fingering position information Pm at the reproduction time T generated in step 90 is a combination of the fingering position information Ps of FK12 and the fingering position information Pe of FK24, and the fingering image drawn with the fingering position information Pm. Is a moving image that moves from a chord type that presses the note numbers 45 and 52 to a chord-type bill that presses the note numbers 52 and 57 (after the note number 45 is released and before the note number 52 is pressed). .

  In the state E in which the reproduction time T is “216”, T1 is “time 202” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. 4, and T2 is the reproduction time T. “Time 216” that is the nearest key press time before T because T is the key press time, “Time 226” that is the nearest key release time after T2, and T4 is the nearest key press time after T2. Time 240 ".

If the reproduction time T is “216”, T2 is “time 202”, and T <T2 is not satisfied, so the determination in step 55 is “N”.
At time 216, the key is being depressed (step 63), and the condition of step 64 (T4-T3 = 14 <Tmin = 96) is also satisfied, so that note number 52 can be depressed simultaneously with the index finger and note number 57 can be depressed simultaneously with the thumb. Thus, determination is made that Ps and Pe can be generated at FK24 (step 65), and fingering position information Ps at the start point and fingering position information Pe at the end point are FK24 (step 66). In this case, the FK 24 is combined key press information of FK2 and FK4 in which the note number 52 is pressed with the index finger and the note number 57 is pressed with the thumb.
The fingering position information Pm generated at step 90 at the reproduction time T becomes the fingering position information Ps (Pe) of the FK 24, and the fingering image drawn with the fingering position information Pm is obtained by pressing the note numbers 52 and 57. This is a fingering image in which a key-press display is performed on the note number 52 with a key-shaped chord-shaped bill.

In the state F in which the reproduction time T is “234”, T1 is “time 226” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time for key release, “time 240” which is the nearest key press time after T, “time 263” which is the nearest key release time after T2, and T4 is the nearest key press time after T2. Time 336 "is reached.
When the reproduction time T is “234”, since T1 exists, the determination in step 54 is “N”.
The key is being released at time 234 (step 63), and the condition of step 70 is not satisfied (T4 is equal to T2, but T (time 234) is not equal to or greater than T2 (time 240)), and the condition of step 72 (T2- Since T1 = 14 <Tmin = 96) is satisfied, it is determined whether Ps can be generated by FK12 (step 73), and the fingering position information Ps at the starting point is FK12 (step 74). In this case, the FK 12 is combined key press information of FK1 and FK2 in which the note number 45 is pressed with the index finger and the note number 57 is pressed with the thumb.
The condition of step 77 (T4−T3 = 73 <Tmin = 96) is satisfied, the note number when T2 is pressed with the little finger is “40”, and the chord with note number “57” when T4 is pressed with the thumb is Since the same bill shape is impossible, it is determined in step 78 that Pe cannot be generated by FK24, and the fingering position information Pe at the end point is FK2 (step 80). FK2 in this case is key press information for pressing the note number 57 with the thumb.

  The fingering position information Pm at the reproduction time T generated in step 90 is a combination of the fingering position information Ps of FK12 and the fingering position information Pe of FK2, and the fingering image drawn with the fingering position information Pm. Is a moving image (after the release of note number 52 and before the note number 57 is pressed) that moves from a chord type that presses note numbers 52 and 57 to a single note-type hand that presses note number 57.

  In the state G in which the reproduction time T is “240”, T1 is “time 226” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time of key depression, “time 240” which is the nearest key depression time before T, T3 is “time 263” which is the nearest key release time after T2, and T4 is the nearest key depression time after T2. Time 336 "is reached.

When the reproduction time T is “240”, T2 is “time 240”, and T <T2 is not satisfied, so the determination in step 55 is “N”.
At time 240, the key is being depressed (step 63), and the condition of step 64 (T4-T3 = 73 <Tmin = 96) is satisfied, so it is determined whether Fs 24 can generate Ps and Pe (step 65). At step 65, the note number when T2 is pressed with the little finger is “40”, and the chord with note number “57” when T4 is pressed is impossible, so it is determined that Pe cannot be generated by FK24. The fingering position information Ps at the time and the fingering position information Pe at the end point are FK2 (step 68). FK2 in this case is key press information for pressing the note number 57 with the thumb.

  The fingering position information Pm generated at step 90 at the reproduction time T becomes the fingering position information Pe (Pe) of FK2, and the fingering image drawn with the fingering position information Pm can be keyed on the note number 57. This is a fingering image in which a key press display during key press is performed on the note number 57 with a simple single-note bill.

  In the state H in which the reproduction time T is “288”, T1 is “time 263” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time for key release, “time 336” which is the nearest key pressing time after T, “time 346” which is the nearest key releasing time after T2, and T4 is the nearest key pressing time after T2. Time 360 ".

If the reproduction time T is “288”, T1 exists, so the determination in step 54 is “N”.
At time 288, the key is being released (step 63), the condition of step 70 is not satisfied (T4> T2), the condition of step 72 (T2-T1 = 73 <Tmin = 96) is satisfied, and Ps can be generated by FK12. Is determined (step 73).
In step 73, it is determined that generation is impossible because the key pressing positions of the fingers to be combined are separated, and the fingering position information Ps at the start point is FK1 (step 76). FK1 in this case is key press information for pressing the note number 57 with the thumb.
Since the condition of step 77 (T4-T3 = 14 <Tmin = 96) is satisfied and the note number 40 can be pressed simultaneously with the little finger and the note number 52 can be pressed simultaneously with the thumb, it is determined that Ps and Pe can be generated by the FK 24 (step 78), the fingering position information Pe at the end point is FK24 (step 79). The FK 24 in this case is combined key press information of FK2 and FK4 in which the note number 40 is pressed with the little finger and the note number 52 is pressed with the thumb.

  The fingering position information Pm generated at step 90 at the reproduction time T is a combination of the fingering position information Ps of FK1 and the fingering position information Pe of FK24, and the fingering image drawn with the fingering position information Pm. Is a moving image (after note number 57 is released and before note number 40 is pressed) that moves from a single note type that presses note number 57 to a chordal note that presses note number 40 and note number 52. .

  In the state I in which the reproduction time T is “336”, T1 is “time 263” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. “Time 336”, which is the nearest key pressing time before T, because T is the key pressing time, “Time 346”, which is the nearest key releasing time after T2, and T4 is the nearest key pressing time after T2. Time 360 ".

If the reproduction time T is “336”, T2 is “time 263”, and T <T2 is not satisfied, so the determination in step 55 is “N”.
At time 336, the key is being depressed (step 63), and the condition of step 64 (T4-T3 = 14 <Tmin = 96) is satisfied, so that it is determined that Fs24 can generate Ps and Pe (step 65). The fingering position information Ps and the fingering position information Pe at the end point are FK24 (step 66). In this case, the FK 24 is combined key press information of FK2 and FK4 in which the note number 40 is pressed with the little finger and the note number 52 is pressed with the index finger.
The fingering position information Pm generated at step 90 at the reproduction time T becomes the fingering position information Ps (Pe) of the FK 24, and the fingering image drawn with the fingering position information Pm is obtained by pressing the note numbers 40 and 52. This is a fingering image in which a key depression display is performed on the note number 40 with a keyable chord-shaped bill.

In the state J in which the reproduction time T is “354”, T1 is “time 346” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time to release the key, “time 360” which is the nearest key pressing time after T, “time 370” which is the nearest key releasing time after T2, and T4 is the nearest key pressing time after T2. Time 384 "is reached.
When the reproduction time T is “354”, since T1 exists, the determination in step 54 is “N”.

The key is being released at time 354 (step 63), the condition of step 70 is not satisfied (T4> T2), the condition of step 72 is satisfied (T2-T1 = 14 <Tmin = 96), and Ps can be generated by FK12. (Step 73), the fingering position information Ps at the start point is FK12 (Step 74). In this case, the FK 12 is combined key press information of FK1 and FK2 in which the note number 40 is pressed with the little finger and the note number 52 is pressed with the thumb.
Since the condition of step 77 is satisfied (T4-T3 = 10 <Tmin = 96) and Pe can be generated by FK24, the fingering position information Pe at the end point is FK24 (step 79). In this case, the FK 24 is combined key pressing information of FK2 and FK4 for pressing the note number 52 with the thumb.

  The fingering position information Pm at the reproduction time T generated in step 90 is a combination of the fingering position information Ps of FK12 and the fingering position information Pe of FK24, and the fingering image drawn with the fingering position information Pm. Is a moving image (after the release of the note number 40 and before the note number 52 is pressed) that moves from a chord type that presses the note numbers 40 and 52 to a single note-type hand that presses only the note number 52.

Each state K to U indicates a fingering image obtained by the processing of the flowchart of FIG. 4 as in the states A to J, and the fingering image corresponding to each state is displayed as follows. .
State K Note number 52 being pressed Note number 52 single note State L Moving after note number 52 was released and before note number 56 was pressed
Change from note number 52 single note type to note number 56 single note type (finger grip) State M Pressing note number 56 key note number 56 single note type (finger touch)
State N Moving after note number 56 is released and before note number 45 is pressed
Change from note number 56 single note type to note number 45, 52 chord type State O Note number 45 being pressed Key number 45, 52 chord type State P Note number 45 being released, note number 52 being moved before key press
Note number 45, 52 Change from chord type to note number 52, 57 chord type State Q Note number 52 is pressed Note number 52, 57 chord type State R After note number 52 is released, note number 57 before key press During the move
Note number 52, 57 Changed from chord type to note number 57 single note state S Note number 57 pressed key Note number 57 single note state T Moved after note number 57 was released and before note number 45 key pressed
Change from note number 57 single note type to note number 45, 52 chord type State U Pressing note number 45 key number Note number 45, 52 chord type

In the state L and the state M, a fingering image is displayed so that the finger usage of “finger grip” which is a special finger usage can be understood. The presence / absence of special finger use is determined by setting a flag to indicate which operation is “finger change”, “finger pass”, or “finger touch”.
As for the image generation of the special finger, key pressing information is generated from the score information in the process of step 62 in FIG. 4 and is captured in the key pressing information correction process of step 67, and the fingering image is displayed. .

In the fingering display program of the present invention, when a fingering video is generated, if the time from the release of the key to the next key press is long, the fingering image is displayed within a short elapsed time after the key is released. It is characterized by not moving the fingering image unnaturally by moving in advance.
In the case of the musical score of FIG. 6, in the left hand performance, there are 16th and 8th rests in the fifth bar and all rests in the sixth bar. The piano roll of FIG. 7 corresponding to the score of this part is in a state T of “time 588”.

In the state T in which the reproduction time T is “588”, T1 is “time 551” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time for key release, “Time 768”, which is the nearest key press time after T, T3 is “Time 778”, which is the nearest key release time after T2, and T4 is the time which is the nearest key press time after T2. “792”.
If the reproduction time T is “588”, T1 exists, so the determination in step 55 is “N”.

  The determination as to whether or not to perform the advance movement is made by sequentially performing the processes of Step 70, Step 72, and Step 75 in the flowchart of FIG. That is, in step 70, it is determined whether or not it is after the last key press, in step 72 it is determined whether or not the next key press is shorter than a predetermined time, and in step 75 it is determined whether or not the preceding movement is performed.

For example, at time 588, the key is being released (step 63), the condition of step 70 is not satisfied (T = 588 is not T2 = 768 or more), and the condition of step 72 is not satisfied (T2-T1 = 768−). 588 = 180 is greater than Tmin 96).
At time 588, since the condition of step 75 is satisfied (T-T1 = 588-551 = 37 is smaller than Tmin96), it becomes the object of the preceding movement here. Since the time from T1 to T2 is longer than Tmin, key pressing at T1 and T2 cannot be synthesized, so the fingering position information Ps at the start point is not FK12 but FK1 (step 76). FK1 in this case is FK1 key pressing information for pressing the note number 57 with the thumb.

  Subsequently, the condition of step 77 is determined. Since T4−T3 = 792−778 = 14 is smaller than Tmin96, “Y” and Pe can be generated by FK24 (step 78). Therefore, the fingering position information Pe at the end point is It becomes FK24 (step 79). The FK 24 in this case is combined key press information of FK2 and FK4 in which the note number 52 is pressed with the index finger and the note number 45 is pressed with the little finger.

The fingering position information Pm generated at step 90 at the reproduction time T is a combination of the fingering position information Ps of FK1 and the fingering position information Pe of FK24, and the fingering image drawn with the fingering position information Pm. Is a moving image (moving before the note number 45 is pressed after the note number 57 is released) that moves from the note number 57 single note type to the note number 45, 52 chord type bill.
The preceding movement end time is time 647 after Tmin (96) from start time 551 (T1).

  Next, drawing of the fingering image by the advance movement will be described with reference to FIGS. 2, 10, and 11.

  The score of FIG. 2 is an enlarged version of the fifth to sixth measures of the score of FIG. 6. From the position of the BAR in the score, the last musical tone of the fifth measure by the left hand is pressed by the thumb with the sound of “La”. The key press image displayed is displayed. After this, even if the performance progresses, the left-hand key depression pauses until the “Ra” sound in the seventh measure.

  In such a case, as shown in FIG. 10, the conventional key press image is depicted as shown in FIG. 10. The key press image moves to the left as the BAR progresses. It will slowly move to the position where the sound of the key is pressed over a period of one and a half bars.

  On the other hand, according to the fingering drawing method described above, when the key-press image is generated, the nearest key press time after the playback time T from the key release time when the key press is changed to the key release at the playback time T. When a certain time T2 and a time T1 immediately before changing to the key press information at the time T2 are obtained and the time from the time T1 to the time T2 is longer than the desired time, an elapsed time from the time T1 to the reproduction time T is further obtained. Is determined to be short, and if it is short, the end point fingering position information (Pe) is obtained with the end point time as T1 + Tmin based on the combined key press information obtained by combining the key press information at time T2, and the fingering position information (Pm ) Is generated.

  Therefore, as shown in FIG. 11 (a), while the BAR is moving in the fifth bar, it is possible to move the key advance image to a position where the left hand key pressing image can already press the seventh bar “L”. When the BAR actually plays the first musical tone of the seventh measure, the key depression image is as shown in FIG. That is, at the time of reproduction time T in FIG. 11A, it is possible to move in advance in a short time to the position of the key to be pressed in FIG. 11B.

  According to the above-described drawing method of the key-press image (fingering display program), a fingering video for playing a musical instrument during music playback is drawn on the screen based on performance information including each musical tone of the music and the corresponding finger number. When the fingering display is performed, when the fingering drawing is generated based on the fingering position information Pm corresponding to the reproduction time T when the music is reproduced, the fingering position information Ps of the time before the reproduction time T and the reproduction time T This is performed based on the combined key pressing information obtained by combining the fingering position information Pe at the subsequent time.

  By determining the time from when the key is released until the next key is pressed, even if the time is long, the movement of the hand is completed as soon as possible until the next key pressing time. A fingering image can be generated. As a result, it is possible to instruct the practitioner to quickly prepare the hand or finger for the next key depression.

DESCRIPTION OF SYMBOLS 11 ... Musical tone / finger number information storage part 12 ... Key pressing information generation part 13 ... Key pressing information storage part 14 ... Fingering position information generation part 15 ... Fingering image generation part 16 ... Musical score image generation part 17 ... Image information storage part DESCRIPTION OF SYMBOLS 18 ... Display apparatus 21 ... Reproduction | regeneration information generation part 22 ... Reproduction | regeneration information storage part 23 ... Sound generation part 24 ... Sound reproduction part 25 ... Sound output device T1 ... Time corresponding to the nearest key release time before reproduction time T (release) (Time before key time, etc.)
T2: When the playback time T is released, the nearest key press time after T. When the playback time T is pressed, the nearest key press time before T. T3: The nearest key release time after T2 (released next to the T2 key press) Time corresponding to the keyed time)
T4 ... The nearest key pressing time after T2 (the time when the key is pressed next to T2 key pressing)
FK1 ... Key press information at time T1 (key press flag and key number of each finger)
FK2 ... Key pressing information at time T2 (key pressing flag and key number of each finger)
FK4: Key press information at time T4 (key press flag and key number of each finger)
FK12: Key pressing information (key pressing flag and key number of each finger) obtained by combining the key pressing information FK1 and the key pressing information FK2.
FK24: Key pressing information obtained by combining key pressing information FK2 and key pressing information FK4 (key pressing flag and key number for each finger)
Ps: Fingering position information at the starting point of fingering (basic coordinates and tip coordinates of each finger, center coordinates of back of hand)
Pe: Fingering position information at the end of fingering (basic coordinates and tip coordinates of each finger, center coordinates of back of hand)
Pm: fingering position information at reproduction time T (basic coordinates and tip coordinates of each finger, center coordinates of back of hand)

In the display device 18 , the image information stored in the image information storage unit 17 is displayed.
That is, based on the key pressing information corresponding to the musical tone of the score, a fingering moving image for playing the musical instrument at the time of music playback is displayed on the screen. As a specific image display, as shown in FIG. 2, a musical score is displayed on the upper stage, and a fingering image for playing each tone of the keyboard and the musical score with both hands is displayed on the lower stage. In the musical score part, a bar BAR corresponding to the reproduction time T is displayed by moving the staff part according to the progress of the performance, and the fingering image draws the key depression state at the reproduction time. The key that is pressed changes its color (indicated by hatching in FIG. 2).
Further, performance information such as a piano roll may be displayed instead of the score.

The generation of fingering images in each state will be described sequentially from state A (reproduction time T = 0) in FIG.
When the reproduction time T is “0”, T1 is invalid (no time corresponding to the nearest key release time before T exists) in the scan of step 52 in the flowchart of FIG. 4, and T2 is the reproduction time T. Since it is time to release the key, “time 192”, which is the nearest key pressing time after T, “time 202”, which is the nearest key releasing time after T2, and T4, which is the nearest key pressing time after T2. Time 216 ".
If the reproduction time T is “0”, T1 is invalid and T <T2 is satisfied, so step 55 is “Y”, and the fingering position information Ps at the start point is FK1 (step 56). When the condition is satisfied, the fingering position information Pe at the end point is also FK1 (step 58). In this case, FK1 serves as key press information of the initial image of the home position because T1 does not exist.
The fingering position information Pm generated at step 90 at the reproduction time T is a combination of the fingering position information Ps and the fingering position information Pe. The fingering image drawn in step 1 is also an initial image.

In the state F in which the reproduction time T is “234”, T1 is “time 226” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time for key release, “time 240” which is the nearest key press time after T, “time 263” which is the nearest key release time after T2, and T4 is the nearest key press time after T2. Time 336 "is reached.
If the reproduction time T is "234", the determination in step 55 since T1 is present is "N".
The key is being released at time 234 (step 63), and the condition of step 70 is not satisfied (T4 is equal to T2, but T (time 234) is not equal to or greater than T2 (time 240)), and the condition of step 72 (T2- Since T1 = 14 <Tmin = 96) is satisfied, it is determined whether Ps can be generated by FK12 (step 73), and the fingering position information Ps at the starting point is FK12 (step 74). In this case, the FK 12 is combined key press information of FK1 and FK2 in which the note number 45 is pressed with the index finger and the note number 57 is pressed with the thumb.
The condition of step 77 (T4−T3 = 73 <Tmin = 96) is satisfied, the note number when T2 is pressed with the little finger is “40”, and the chord with note number “57” when T4 is pressed with the thumb is Since the same bill shape is impossible, it is determined in step 78 that Pe cannot be generated by FK24, and the fingering position information Pe at the end point is FK2 (step 80). FK2 in this case is key press information for pressing the note number 57 with the thumb.

If the reproduction time T is "288", the determination in step 55 since T1 is present is "N".
At time 288, the key is being released (step 63), the condition of step 70 is not satisfied (T4> T2), the condition of step 72 (T2-T1 = 73 <Tmin = 96) is satisfied, and Ps can be generated by FK12. Is determined (step 73).
In step 73, it is determined that generation is impossible because the key pressing positions of the fingers to be combined are separated, and the fingering position information Ps at the start point is FK1 (step 76). FK1 in this case is key press information for pressing the note number 57 with the thumb.
Since the condition of step 77 (T4-T3 = 14 <Tmin = 96) is satisfied and the note number 40 can be pressed simultaneously with the little finger and the note number 52 can be pressed simultaneously with the thumb, it is determined that Ps and Pe can be generated by the FK 24 (step 78), the fingering position information Pe at the end point is FK24 (step 79). The FK 24 in this case is combined key press information of FK2 and FK4 in which the note number 40 is pressed with the little finger and the note number 52 is pressed with the thumb.

In the state J in which the reproduction time T is “354”, T1 is “time 346” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time to release the key, “time 360” which is the nearest key pressing time after T, “time 370” which is the nearest key releasing time after T2, and T4 is the nearest key pressing time after T2. Time 384 "is reached.
When the reproduction time T is “354”, since T1 exists, the determination in step 55 is “N”.

In the state T in which the reproduction time T is “588”, T1 is “time 551” corresponding to the nearest key release time before T in the scan of step 52 in the flowchart of FIG. Since it is time for key release, “Time 768”, which is the nearest key press time after T, T3 is “Time 778”, which is the nearest key release time after T2, and T4 is the time which is the nearest key press time after T2. “792”.
When the reproduction time T is “588”, since T1 exists , the determination in step 55 is “N”.

Claims (7)

A program that draws a fingering video for playing a musical instrument during music playback based on performance information including each musical tone and the corresponding finger number, and displays the fingering on the screen.
When generating the fingering drawing based on the fingering position information (Pm) corresponding to the playback time (T) at the time of the music playback,
A time (T2) that is the nearest key press time after the playback time (T) from the key release time when the key press changes to the key release at the playback time (T);
A time (T1) immediately before the key press information at the time (T2) is changed;
When the time from the time (T1) to the time (T2) is longer than the desired time, the time (T2) is not pressed until the predetermined time elapses from the time (T1). A fingering display program for causing a computer to generate fingering position information (Pm) that completes a transition to a fingering position corresponding to key information. Based on a single key pressing information including at least a time before the reproduction time (T) and a time after the reproduction time (T), and synthesized key pressing information obtained by combining the key pressing information at a plurality of different times. The fingering display program according to claim 1, wherein the fingering position information (Pm) is generated. The plurality of different times are
A time (T1) immediately before the key press information at the time (T2) is changed;
The fingering display program according to claim 2, comprising a time (T4) immediately after the key pressing information at the time (T2) is changed. The synthetic key press information is
Key press information (FK12) obtained by combining the key press information (FK1) at the time (T1) and the key press information (FK2) at the time (T2);
The key press information (FK24) obtained by synthesizing the key press information (FK2) at the time (T2) and the key press information (FK4) at the time (T4) is included. Finger display program.   5. The fingering display according to claim 1, wherein the key pressing information is a key pressing position (keyboard position) and a key pressing state (key pressing or key release) of each finger at the time. program.   The fingering display program according to any one of claims 1 to 5, wherein the fingering position information (Pm) includes at least the center position of the back of the hand and the tip position of each finger. The fingering position information (Pm) is
Fingering position information (Ps) at the starting point of fingering at time (Ts) before the reproduction time (T);
Fingering position information calculated by interpolating the fingering position information (Pe) at the end of the fingering at the time (Te) after the reproduction time (T) according to the passage of time from the time Ts to the time Te. The fingering display program according to any one of claims 1 to 6.

Images