JP2001337675A - Playing support device and playing support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a playing support device which can instruct to correct a difference of a tempo without complex structure. SOLUTION: A playing data of high note is inputted, and a playing part is found by searching which data of note accords with in a music data, then the music of the part is displayed. By using this result of trace transaction for searching playing part, a difference of playing tempo and difference of tempo of the right-before note are found and a player is instructed. Thus, the playing support device, which can indicate the playing tempo by using a information for paging note, is realized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a performance support apparatus and a performance support method for displaying a musical score, inputting a performance of a player, detecting a tempo deviation, and instructing a correction.

[0002]

2. Description of the Related Art There has been proposed a musical score display device (electronic musical score) for electrically displaying an image of a musical score on a display. In this type of electronic musical score, when displaying a musical score of a plurality of pages, there is a type in which the playing position of the player is automatically detected and the displayed page can be updated (page turning).

[0003]

As described above, even in the conventional electronic musical score, there is proposed an electronic musical score having a function of automatically detecting a musical performance position by inputting performance pitch data and the like and turning a page based on the data. However, there was no one provided with a performance support function for pointing out the tempo deviation of the player and instructing correction.

The present invention provides a performance support apparatus and a performance support method capable of instructing correction of a tempo deviation without complicating the configuration by using a function of detecting a performance position of a player and updating a display position of a musical score. The purpose is to provide.

[0005]

According to the present invention, there is provided a performance support apparatus for storing musical score data including a musical note data string representing musical notes, inputting means for inputting musical pitch data, and inputting. Means for estimating a current performance position and a performance tempo in the note data string based on the pitch data input from the means, and displaying the display of the music score data based on the estimated performance position and performance tempo. Means for determining the position to be displayed on the means, and predicting the input timing of the pitch data to be input next based on the estimated performance position and the tempo of the performance, and when the next pitch data is input. Control means for calculating a tempo shift based on a difference between the actual input timing and the predicted input timing; and a score image and a score image based on the score data. Characterized by comprising a display means for displaying by synthesizing modified instruction image corresponding to the tempo shift.

Further, according to the performance support method of the present invention, in a device storing musical score data including a musical note data sequence representing musical notes, a procedure for inputting musical pitch data is performed. A procedure for estimating a current performance position and a performance tempo in the note data sequence, a procedure for determining a portion to be displayed in the musical score data based on the estimated performance position and the performance tempo, and a process for estimating the estimated performance position And a procedure for predicting the input timing of the pitch data to be input next based on the tempo of the performance. When pitch data is next input, a difference between the actual input timing and the predicted input timing is calculated. Calculating a tempo shift based on the score data, and synthesizing a score image based on the score data and a correction instruction image corresponding to the tempo shift Characterized in that it has a.

The note data string is composed of a plurality of note data, and each note data includes at least pitch information indicating a pitch of each note and pitch information indicating a note length. The pitch information may be expressed in the form of pitch + octave, for example, C3, D3, or C1 = 0, C2 = 1.
2, it may be represented by the number of semitones with C3 = 24. Also,
The pitch data of the performance is also input in the same format as the pitch information, and it is possible to compare the match / mismatch.

As a method for detecting which note in the note data string the inputted pitch data corresponds to,
There are the following note trace and time scale trace methods. First, note tracing processing is performed to search for a point on the note data string that matches the input pitch data. Multiple pitch data is input depending on the performance,
If there is a lot of matches even if not all match, the position is determined as one of the candidates for the performance position. That is, a position on the note data string having a high probability that the input pitch data matches the pitch is set as a candidate. By retrieving the entire note data string in this manner, it is possible to cope with a case where the performance starts from any position, and it is possible to detect the correct performance position even if there is a mistouch or the like in the performance.

Then, a time scale process is performed on one candidate having the highest probability or a plurality of candidates having the highest probability. The time scale process waits for the input of the pitch data of the note data to be played next at the performance position given as the candidate, and determines that the position of the candidate for which the pitch data matching this is input is the performance position. This is the process to determine. When there is one candidate, the correctness of the candidate is determined, and when there are a plurality of candidates, which of the candidates is correct is determined. Then, a tempo shift is calculated based on how much the input timing deviates from the input timing predicted based on the tempo estimated up to that time,
Further, the performance tempo is newly calculated in consideration of the current input timing. As described above, the detection of the playing position is used not only for turning pages of the musical score displayed on the display means, but also for detecting a tempo deviation of the player.

The pitch data may be input by inputting a tone from a microphone and extracting pitch data from the tone, or by directly inputting pitch data from an interface such as a MIDI interface.

[0011]

FIG. 1 is a block diagram of a performance support apparatus according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of musical score data stored in the performance support apparatus. The musical score data storage unit 5 stores musical score data representing a music to be played. The musical score data has the configuration shown in FIG. 2. The apparatus determines the timing deviation of the performance using the musical score data and displays the image of the musical score on the display unit 7.
To be displayed.

In FIG. 2, the musical score data is obtained by arranging note data indicating the pitch and duration of each note and symbol data indicating various symbols other than the notes expressed in the musical score in chronological order together with their display positions. It is. The display position is represented by a time axis in units of a tempo clock, and the upper and lower positions are determined by the pitch in the case of note data, and are displayed in predetermined positions according to the type of the symbol in the case of symbol data. You. The score data is composed of a plurality of pages.

The musical score data is output to the matching detecting section 4 and the musical score image generating section 9. Pitch data obtained by extracting a pitch from a performance of a player in real time is input to the matching detection unit 4.

The microphone 1 inputs voices such as a performance sound of a musical instrument and a singing voice of a singer, and inputs this voice signal to the AD converter 2. The AD converter 2 converts this voice into digital voice data and inputs the digital voice data to the analysis unit 3. The analysis unit 3 divides the input audio data into frames of a short section, and extracts pitch data of the audio signal for each frame. The pitch data is data representing the pitch of the input musical tone signal in the 12 scales, and C3, D4
Alternatively, the whole pitch of the processed sound range may be represented by individual numerical values. Further, a slight frequency shift of the tone signal is made to converge to any pitch. The extracted pitch data is input to the matching detection unit 4.

The matching detection unit 4 includes a MIDI
Interface 11 is connected. An electronic musical instrument having a MIDI output function is connected to the MIDI interface 11. Since the MIDI data input from the MIDI interface 11 can be used as pitch data as it is, it is directly input to the matching detection unit 4.

The matching detection section 4 compares the analyzed pitch data with the note data in the musical score data to specify a performance location, and based on the input timing (or change timing) of the pitch data, the performance tempo. Detect the deviation. The matching detection unit 4 outputs the performance location information specifying the performance location to the musical score image generation unit 9 and outputs to the instruction image generation unit 10 tempo deviation information that has detected a deviation in the performance tempo.

The musical score image generating section 9 includes a matching detecting section 4
The musical score data before and after the musical score data is read out from the musical score data storage unit 5 based on the performance location information input from the CPU, and image data of the musical score is generated based on the musical score data. The generated image data is input to the display control unit 6.

The instruction image generation unit 10 generates an instruction image for instructing the player to correct the performance tempo based on the tempo deviation information. FIG. 3 shows an example of the image. The vertical line 50 indicates where the current performance progress position (current point) is on the musical score, and the arrow 51 indicates whether the current performance tempo is too fast or delayed compared to the entire performance tempo. The current performance tempo is estimated at the performance timing of the immediately preceding note, and the overall performance tempo is estimated based on the performance timings of all the notes up to now. Further, the degree of being too fast or too late is indicated by a numerical value and wording 52.

The display control section 6 combines the musical score image input from the musical score image generating section 9 and the instruction image input from the instruction image generating section 10 and displays them on the display section 7.

The operation section 8 is an operation element for instructing the matching detection section 4 in advance at a place where the performance is to be started, and the bar number and page number of the musical score are input.

The operation of the matching detecting section 4 will be described. The matching detection unit 4 detects a currently played part on the musical score data based on the inputted pitch data and tempo. This detection work is performed by searching a note data string based on the input pitch data and finding a matching portion (candidate).
To find the note trace, which is the work of detecting the matching of musical scores, and the note data that will soon be played out of the note data, and wait for the input of the musical tone (pitch data) corresponding to this note data This is performed for the two-way time scale trace, which is the work of detecting the matching of musical score → performance.

FIG. 4 is a diagram for explaining how the matching detection section 4 specifies a performance location and detects a tempo deviation.

First, referring to FIG. 4, a description will be given of a method of specifying a performance position. The performance position is specified by a note trace and a time scale trace. First, the note trace will be described. This is to generate a pitch sequence based on input pitch data, and to use this as a pattern to detect matching of musical score data with a note data sequence. In the example of FIG. 4, it is assumed that the musical note data string of the musical score data is lined up with Doremi Mired and MiRemi Sophored. FIG. 11 shows an example in which the player starts playing from the fifth note M and is played without a mistouch. That is, this is an example of a case where "Miredo ..." is played. In this embodiment, the performance location can be detected even when the performance is practiced or the like and the player is re-played from the middle.

First, "mi" is input as the first pitch data. When the musical note data that matches mi is retrieved from the musical score data, the third, fifth, eighth, and tenth sounds correspond to this. The positions of the third, fifth, eighth, and tenth sounds are stored as current position candidates 21, 22, 23, and 24, respectively. When the input pitch data matches the next note data (fourth, sixth, ninth, and eleventh notes), the candidate moves to that position.

The candidate data of each candidate is composed of "note position / number of matched notes / number of mismatched notes / number of continuous matches / number of continuous mismatches", and is updated each time new pitch data is input. The note position indicates on which note data the candidate is currently located. The number of coincident notes is the number of note data that matches the note data string among the inputted pitch data. The number of unmatched notes is the number of pieces of pitch data that did not match the note data string. The number of consecutive matches is a number indicating the number of continuous matches when the input pitch data matches the note data string. The consecutive mismatch number is a number indicating the number of consecutive mismatches when the input pitch data does not match the note data string.

Since the candidates 21 to 24 match only the first one sound, the candidate data are respectively “third sound /
1/0/1/0 "," 5th sound / 1/0/1/0 "," 8th sound / 1/0/1/0 "," 10th sound / 1/0/1/1 "
0 ".

A note is input as the pitch data of the next performance. A check is made for each of the candidates 21 to 24 to see if the next note is レ. Then, candidate 22 and candidate 2
Since the note following the third is, the note position of these candidates is moved to the position of this note data to update the candidate data. Candidate 22 is “sixth sound / 2/0/2/0”, candidate 23
Is “9th sound / 2/0/2/0”. On the other hand, candidate 2
Nos. 1 and 24 do not match the pitch data because the next note data is not レ, and candidate 21 is “third note / 1/1/0 /
1 ", and candidate 24 is" 10th sound / 1/1/0/1 "
Becomes

Next, when the musical note data of the existing candidate is not moved, the second and thirteenth sounds are found. These are set as candidates for the new current position.
6 is assumed. Candidate 25 is “second sound / 1/0/1/0” and candidate 26 is “thirteenth sound / 1/0/1/0”.

Further, since "do" is input as the next pitch data, it is checked whether the next note data is "do" for the candidates 21 to 26. Candidate 21, Candidate 22, and Candidate 26
Since the next note data is "C", the note position of these candidates is moved to the position of the note data to update the candidate data. Candidate 21 is “fourth sound / 2/1/1/0”, candidate 22 is “seventh sound / 3/0/3/0”, and candidate 26 is “first sound”.
4 sounds / 2/0/2/0 ".

For candidates 23 to 25, candidate 23 is "ninth sound / 2/1/0/1" and candidate 24 is "first sound".
0 sound / 1/2/0/2 ", and candidate 25 is" second sound / 1/1
/ 0/1 ". Next, when the musical note data of C whose existing candidate has not moved is searched, the first sound is present. This is set as a candidate for the new current position and the candidate 27 “first sound / 1
/ 0/1/0 ".

As described above, the number of matches and the number of consecutive matches of the candidate 22 indicating the current point which is a correct performance position have large values, and the number of mismatches is small (0). By examining the candidate data in this way, a correct current point can be determined. Further, even in the case of a mismatch, the reason why the candidate is not deleted immediately and the number of mismatches is memorized and stored is to cope with a case where the player makes a wrong touch and inputs an incorrect sound. That is, even when the player makes a mistouch, the number of mismatches increases by one, and does not significantly hinder finding the correct current point. For example, even if a place to be played as "Miredo ..." is played as "Miredo ..."
Is "7th sound / 3/1/2/0", which is also a potential current point candidate. If the player makes a mistake or skips a sound, the next sound starts to determine the correct candidate shown in FIG.

When the candidates for the current performance position are narrowed down by the above note trace, the current performance position is finally determined by the time scale trace.
Detects tempo deviation in performance. Hereinafter, the time scale trace will be described. If the current note candidates are narrowed down to some by the above note trace,
The musical score data existing immediately after each current point candidate is extracted one by one, and it waits for which note data coincides with the next input of pitch data. At this time, which candidate is the actual current point is determined based on the input pitch data, the tempo deviation amount is calculated and the tempo is corrected based on the input timing of the pitch data.

Extraction is performed, and the display of the current point is made to progress predictably by matching the time signature of the musical score data.

FIG. 5 is a diagram for explaining a method of estimating a tempo. If the note timing on the musical score is taken on the X-axis and the actual pitch data input timing is taken on the Y-axis, if the performance is performed at a constant tempo, it will be a straight line passing through the origin.
The straight line in the figure shows a straight line having a slope of 1 when the tempo indicated by the score and the performance tempo are the same. The actual performance tempo is detected discretely at each note timing, that is, at each timing at which pitch data corresponding to the note is input. Therefore, a straight line passing near these points is calculated by a calculation method such as the least square method. Draw and estimate tempo. If the tempo is estimated with all note timings having the same weight, it is impossible to follow the tempo change intended by the player. If the tempo is estimated only with the current note timing, the effect of mistouch or irregular touch will be large. Therefore, the tempo is estimated by making the weight smaller as the timing of the past note becomes smaller, or increasing the weight as the timing of the current note becomes larger. Then, the input timing of the pitch data of the next note (next note timing) is estimated based on the tempo, and the tempo difference between before and after the estimated timing and the input timing of the actual pitch data is defined as the tempo shift amount. .

Instead of estimating the tempo at each note timing, the tempo may be estimated (evaluated) at predetermined intervals as shown in FIG. It is only necessary to assign the maximum weight to the note timing of the most recent cycle, and to make the note timing lighter as it goes back.

The instruction image shown in FIG. 3 is displayed when a tempo deviation is detected based on the input timing of note data immediately after the note pitch data is input. If it is late, as shown in the figure, an arrow 51 pointing from the current point 50 in the musical score progression direction and (fast) characters are displayed, and the degree is displayed as a percentage. If it is running (too fast), an arrow and a (slow) arrow in the direction opposite to the direction of the score are displayed, and the degree is displayed as a percentage. The display mode is not limited to this, and for example, the degree of delay or running may be displayed by the length of the arrow. Also, this percentage display does not necessarily need to indicate the exact amount of tempo shift, and may be used as an index indicating the amount of shift. Thus, the units need not be percentages.

The degree to which the tempo is corrected by looking at the tempo-shifted instruction image differs depending on the player. Therefore, the display content may be changed by checking the degree to which the player corrects the tempo. That is, FIG.
Even if the tempo deviation is displayed as shown in (A), a player whose degree of tempo correction is small (small feedback) is displayed as shown in FIG. (C) is displayed for a player who has a large (high feedback).

FIGS. 7 and 8 are flowcharts showing the operation of the performance support apparatus. FIG. 7 is a flowchart showing the note trace operation. First, the score image generation unit 9
Reads the score data of the first page from the score data storage unit 5, synthesizes the image of the score, and outputs it to the display control unit 6. The display control unit 6 displays this on the display unit 7 (Step 101: hereinafter simply referred to as 101). Thereafter, it is detected whether pitch data is input (102) or an instruction to change the display page is input from the operation unit 8 (103). When an instruction to change the display page is input (103), the musical score data of the designated page is read from the musical score data storage unit 5 and displayed on the display unit 7 (104). And
If the current point candidate is stored, it is cleared (105).

When pitch data is input (102), a note data string of musical score data is searched with this pitch data to search for a candidate for a current point (106). This search process is the process described with reference to FIG. 4 and includes updating of the already set candidates. Then, the degree of matching of each set candidate is determined (107).
The degree of matching is determined based on the note matching rate and the number of consecutive matches / mismatches. Note match rate is the number of note matches /
It is calculated by (note match number + note mismatch number). 10
In 7, a note matching rate is calculated for each candidate, and a candidate having a note matching rate equal to or less than a predetermined value m1 and a candidate having a continuous mismatch number equal to or more than a predetermined value m2 are excluded from the candidates because the probability of being a current point is extremely low. Delete (1
08). In addition, a candidate having a high probability is determined as a current point of a candidate whose number of note matches is equal to or more than a predetermined value m3 and a note matching rate is equal to or more than a predetermined value m4 (109), and transmitted to a time scale trace (110).

FIG. 8 is a flowchart showing a time scale trace and a tempo deviation detecting operation. Receive the current point candidate from the note trace operation (1
20). When there are a plurality of candidates, all of the plurality of candidates are received. For each of the received current point candidates, pitch data to be input next and its timing are predicted (121). The next pitch data to be input is the pitch data of the next note on the score data, and its timing has progressed in the interval to the next note on the score data at the currently estimated tempo. This is the input timing of the pitch data in the case. For estimating the tempo, the one estimated in the immediately preceding time scale tracing operation is used. Then, the process waits until pitch data is input (122). When pitch data is input, it is determined whether or not the pitch data matches the predicted pitch data and timing (123). That is, it is determined whether or not the same pitch data as that predicted was input at the predicted timing. However, since there are tempo shifts and mis-touches in the actual performance, the input timing is judged to match with a certain time width,
Regarding the pitch data, a pitch that is likely to be mistouched with the pitch data to be input is recognized as a correct input in the time scale trace (an incorrect input in the note trace).
What is necessary is just to make it determine with tolerance, such as. If there are a plurality of candidates, the candidate with the highest matching probability is determined. If there is a match (124), this position is determined to be the current point, and the tempo deviation of the performance is calculated based on the input timing of the pitch data (125). Then, the tempo shift information is output to the instruction image generation unit 10. The instruction image generation unit 10 generates an instruction image as shown in FIG. 3 based on the tempo shift information and outputs the instruction image to the display control unit 6. The display control unit 6 combines this instruction image with the musical score image generated by the musical score image generating unit 9 and displays it on the display unit 7 (126).

Next, the tempo of the performance is recalculated based on the current input (127). This tempo calculation is performed by the method described with reference to FIG. As the performance proceeds at the detected current point data and the calculated tempo, the musical score image generating unit 9 is instructed on the musical score data to be read from the musical score data storage unit 5 (128).

[0042]

As described above, according to the present invention, by detecting a tempo deviation and displaying it on a musical score, the player can correct the tempo based on this and perform at the correct tempo. Can be. In addition, the detection of tempo deviation
Since the detection can be performed by using the detection result of the performance part / tempo used for the page turning control in the electronic musical score, there is no need to provide a new detecting means in the electronic musical score, and the apparatus is not complicated.

[Brief description of the drawings]

FIG. 1 is a block diagram of a performance support apparatus according to an embodiment of the present invention;

FIG. 2 is a view showing musical score data stored in the performance support apparatus.

FIG. 3 is a diagram showing an example of an instruction image displayed by the performance support device.

FIG. 4 is a view for explaining a note trace process of the performance support apparatus.

FIG. 5 is a view for explaining tempo estimation processing of the performance support apparatus.

FIG. 6 is a diagram showing a modification of the instruction image displayed by the performance support device.

FIG. 7 is a flowchart showing the operation of the performance support apparatus.

FIG. 8 is a flowchart showing the operation of the performance support apparatus.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... AD converter, 3 ... Analysis part, 4 ... Matching detection part, 5 ... Music score data storage part, 6 ... Display control part, 7 ... Display part, 8 ... Operation part

Claims (5)

[Claims] 1. A storage means for storing musical score data including a note data string representing a musical note, an input means for inputting pitch data of a performance, and the pitch data input from the input means. The current playing position and the playing tempo in the note data sequence are estimated, and based on the estimated playing position and the playing tempo, a portion of the musical score data to be displayed on the display means is determined, and the estimated position is determined. The input timing of the pitch data to be input next is predicted based on the performance position and the tempo of the performance, and when the next pitch data is input, the actual input timing and the predicted input timing are input. Control means for calculating a tempo deviation based on the deviation from the musical score, and synthesizing a score image based on the musical score data and a correction instruction image corresponding to the tempo deviation. And a display means for displaying the performance. 2. The performance support apparatus according to claim 1, wherein the matching detection means estimates the tempo of the performance by weighting previously input pitch data as newer and higher as older data. 3. The performance assisting apparatus according to claim 1, wherein the input means includes means for inputting and AD converting a musical tone and extracting a tone pitch of the musical tone from the AD-converted musical tone data. 4. The performance support apparatus according to claim 1, wherein said input means includes means for inputting performance data including pitch data from a musical instrument being played via an interface. 5. A device for storing musical score data including a musical note data sequence representing musical notes of a musical composition, comprising the steps of: inputting performance pitch data; based on the pitch data, a current performance position in the musical note data sequence. And a procedure for estimating the performance tempo; a procedure for determining a location to be displayed in the musical score data based on the estimated performance position and the performance tempo; and a procedure based on the estimated performance position and the performance tempo. A procedure for predicting the input timing of pitch data to be input next, and when pitch data is input next, a tempo shift based on a shift between the actual input timing and the predicted input timing is calculated. A performance assisting method, comprising: determining a musical score image based on the musical score data; and synthesizing a correction instruction image corresponding to the tempo deviation.