JP5808711B2 - Performance position detector - Google Patents

Description

  The present invention relates to a performance position detection apparatus, and more particularly to a performance position detection apparatus that detects a performance position on a musical score when a user plays a musical piece.

  In recent years, electronic musical scores obtained by digitizing musical scores have become widespread. An electronic score is data obtained by simply converting a score printed on paper into an electronic file. For this reason, when using an electronic score, the user must print the electronic score and play the music while viewing the printed electronic score.

  On the other hand, a technique has been proposed in which an image of a score is displayed on a monitor and a score page displayed on the monitor is changed in accordance with the performance of the user. For example, Patent Literature 1 discloses a musical score display device that shows a musical performance position on a displayed musical score in accordance with a musical performance by a user.

  In Patent Document 1, the score display device detects, as a candidate, note data that matches input pitch data from a note data string representing musical notes. The score display device refers to the note data string and predicts the next note data of the detected candidate. The musical score display device determines whether the predicted pitch data matches the pitch data to be input next. The score displaying apparatus determines a candidate whose number of note data continuously matching the pitch data is equal to or greater than a predetermined value as a performance position.

  However, if the user does not play the music according to the score due to a performance mistake, the score display apparatus may take time to specify the performance position. The musical score display apparatus detects a position that matches the pitch data based on the performance error as a candidate from the note data string. As a result, it takes time until the number of pitch data continuously matching the note data reaches a predetermined value or more.

Japanese Patent No. 4389330 JP 2001-5450 A JP 2001-148633 A Japanese Patent Laid-Open No. 2003-2070 JP 2011-257510 A JP 2007-240552 A

"Music That Listens To You-Tonara", [online], [Search April 6, 2012], <URL: http://tonara.com/>

  The objective of this invention is providing the performance position detection apparatus which can pinpoint a performance position rapidly, even if a user makes a performance mistake.

Means and effects for solving the problems

  The performance position detection apparatus according to the present invention detects a performance position on a musical score when a user plays a musical composition based on musical score data in which notes described in the musical score are recorded. The performance position detection device includes a storage unit, a scale specifying unit, an array creation unit, a selection unit, a likelihood calculation unit, and a performance position determination unit. The storage unit stores a musical score arrangement including musical note scales arranged in order of performance. The scale specifying unit inputs an acoustic signal including the performance sound of the music by the user, and specifies one or more scales from the input acoustic signal. The arrangement creating unit creates a performance arrangement including one or more scales arranged in time series. The selection unit selects a plurality of reference arrays, each including the same number of scales as the number of scales included in the performance array, and having different positions on the score array from the score array. The likelihood calculating unit calculates the likelihood for each reference sequence that will be each reference sequence for which the performance sequence is selected. The performance position determination unit identifies the reference sequence having the maximum likelihood from among the selected reference sequences, and determines the position of the last scale of the reference sequence having the maximum likelihood as the performance position.

  According to the present invention, the performance position is determined based on the likelihood that the performance array will be the reference array. As a result, even if the performance array includes a scale associated with a user's performance mistake, the calculation amount does not increase, so that the performance position can be determined quickly.

  Preferably, the performance position determination unit determines the position of the last scale located behind the previous performance position in the time direction as the performance position.

  According to the present invention, the position of the scale before the previous performance position is not determined as a new performance position. Therefore, it is possible to prevent a position unrelated to the position of the scale where the user is actually playing from being erroneously determined as the performance position.

  Preferably, when the performance position detection device identifies a plurality of candidate sequences as the reference sequence having the maximum likelihood, the position of the last scale closest to the immediately preceding performance position among the positions of the last scale of each candidate array Is determined as the performance position.

  According to the present invention, the position of the last musical scale closest to the previous performance position among the positions of the last musical scale of the plurality of candidate sequences is determined as the performance position. Therefore, it is possible to prevent a position unrelated to the position of the scale where the user is actually playing from being erroneously determined as the performance position.

  Preferably, the performance position determination unit determines the position of the last musical scale located within a predetermined range with reference to the previous performance position as the performance position.

  According to the present invention, the position of the scale that is far from the previous performance position is not determined as the performance position. Therefore, it is possible to prevent a position unrelated to the position of the scale where the user is actually playing from being erroneously determined as the performance position.

  Preferably, the performance position determination unit determines that the performance position cannot be determined when the last scale is outside the predetermined range. The performance position detection apparatus further includes a count unit and an array initialization unit. When the performance position determination unit determines that the performance position cannot be determined, the count unit counts the number of times of updating stop indicating that the performance position is not updated. The sequence initialization unit initializes the performance sequence when the number of update stops counted by the count unit exceeds a threshold value.

  When the number of renewal stops exceeds a threshold value, the performance position detection device initializes the performance array, and determines the performance position from the beginning. Therefore, even if the user misses the performance and stops the performance and then resumes the performance, the performance position detection device can quickly detect the position where the performance has been resumed.

  Preferably, the array creation unit creates a new performance array when the performance array is initialized by the array initialization unit. The performance position determination unit determines the position of the scale located a predetermined number of times before the previous performance position as the performance position.

  According to the present invention, when the performance arrangement is initialized, the performance position is changed to a musical scale position that is located a predetermined number of times before the previous performance position. Thereby, even if a user restarts a performance from the position before a stop position after stopping a performance, a performance position can be determined rapidly.

  Preferably, the sequence creation unit includes an addition unit and a replacement unit. When the scale specifying unit specifies one or more scales, the adding unit adds the specified one or more scales to the performance array. The replacement unit replaces past musical scales other than the one or more musical scales added by the adding unit among musical scales included in the performance arrangement, with musical scale arrangements corresponding to the past musical scales.

  According to the present invention, even if a past musical scale due to a performance mistake is included in the performance arrangement, the past musical scale due to the performance mistake is corrected to a musical scale corresponding to the score arrangement. Thereby, it is possible to prevent a position different from the actual performance position from being erroneously detected as the performance position.

  Preferably, the musical score arrangement includes a plurality of scales constituting a chord. The scale specifying unit specifies a plurality of scales constituting the chord from the acoustic signal. The sequence creation unit creates a performance sequence including a plurality of scales specified by the scale specification unit.

  According to the present invention, even when the user plays a musical piece using a score including chords, the scale of the played chord is specified, and the performance position is determined based on the specified scale of the chord. can do.

  The performance position detection program according to the present invention is used in the performance position detection apparatus according to the present invention.

It is a functional block diagram which shows the structure of the performance position detection apparatus by embodiment of this invention. It is a figure which shows the score image data shown in FIG. It is a figure which shows the score arrangement | sequence shown in FIG. It is a figure which shows the performance arrangement | sequence shown in FIG. It is a flowchart of the performance position detection program shown in FIG. It is a figure which shows the acoustic signal input into the microphone shown in FIG. It is a figure which shows the acoustic signal after filtering the acoustic signal shown to FIG. 6A. It is the spectrum produced | generated by performing a fast Fourier transform with respect to the acoustic signal shown to FIG. 6B. It is a figure explaining the process which specifies a musical scale from the spectrum shown to FIG. 6C. It is a flowchart of the performance arrangement | sequence creation process shown in FIG. It is a figure which shows the change of the performance arrangement | sequence in the performance arrangement | sequence creation process shown in FIG. It is a flowchart of the likelihood calculation process shown in FIG. It is a figure explaining the method to calculate the likelihood that the performance arrangement | sequence shown in FIG. 1 will be a reference arrangement | sequence. It is a flowchart of the performance position determination process shown in FIG. It is a figure which shows the positional relationship of the candidate position specified by the likelihood calculation process shown in FIG. 11, and the last performance position. It is a flowchart of the detection state determination process shown in FIG. It is a figure which shows the change of a performance position when a performance position is forcibly changed by the detection state determination process shown in FIG. It is a figure which shows the modification of the tolerance | permissible_range shown in FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

{Configuration of performance position detection device}
FIG. 1 is a functional block diagram showing a configuration of a performance position detection apparatus 1 according to an embodiment of the present invention. With reference to FIG. 1, the performance position detection apparatus 1 displays on the touch panel 13 score image data 7 in which notes described in the score of a song are recorded. The performance position detection device 1 detects the performance position on the score when the user plays the music based on the score array 4. The detected performance position is shown on the score image data 7 displayed on the touch panel 13.

  The performance position detection apparatus 1 is a tablet terminal. The tablet type terminal operates as the performance position detection device 1 by installing the performance position detection program 2.

  The performance position detection apparatus 1 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a touch panel 13, operation buttons 14, a microphone 15, and a flash memory 16.

  The CPU 11 executes a program stored in the flash memory 16. The RAM 12 is a main memory of the performance position detection device 1. The CPU 11 loads the program into the RAM 12 and executes the loaded program to control the performance position detection device 1.

  The touch panel 13 displays the execution result of the program and outputs the position touched by the user to the CPU 11 as operation information. The operation buttons 14 are hardware keys such as a power button and a volume control button. The microphone 15 inputs the performance sound of the music played by the user as an acoustic signal.

  The flash memory 16 is a rewritable nonvolatile semiconductor memory. The flash memory 16 stores a performance position detection program 2, score data 3, a reference array 5, and a performance array 6.

  The score data 3 includes a score array 4 and score image data 7. The musical score image data 7 is image data in which notes described in the musical score of a music are recorded in the order of performance, and is composed of a plurality of pages. The score data 3 may include MIDI data in which performance symbols such as musical notes are described by MIDI (Musical Instrument Digital Interface) instead of the score image data 7. When the musical score data 3 includes MIDI data, the performance position detection apparatus 1 creates musical score image data 7 based on the MIDI data. Alternatively, the musical score data 3 may be MIDI data in which performance symbols such as musical notes are described in MIDI. In this case, the score data 3 is described in, for example, a MusicXML (Extensible Markup Language) format. The performance position detection apparatus 1 analyzes musical score data 3 in the MusicXML format, and creates musical score image data 7 and a performance array 6.

  The musical score array 4 is data in which the scales of the notes described in the musical score are arranged in the order of performance. The reference array 5 is an array of scales of a predetermined length selected from the score array 4. The performance array 6 is data in which the scales of the performance sounds of music by the user are arranged in time series. Details of the score array 4, the reference array 5, and the performance array 6 will be described later.

{Outline of operation of performance position detection device 1 and structure of various arrangements}
The performance position detection device 1 displays the score image data 7 on the touch panel 13. The user plays the music while viewing the musical score image data 7 displayed on the touch panel 13.

  FIG. 2 shows the score image data 7. Referring to FIG. 2, the score image data 7 includes a score 31 for a high-pitched part and a score 32 for a low-pitched part. In the area 3a of the musical score image data 7, musical notes 312 and 322 are described. The musical notes 312 and 322 constitute a chord. Similarly, the notes 316 and 326 described in the area 3b constitute a chord. The notes 311, 313 to 315, and 317 to 319 described in the score 31 do not constitute chords.

  The musical score array 4 includes musical note scales described in the musical scores 31 and 32, and the musical note musical scales are arranged in the order of performance. FIG. 3 is a diagram showing the musical score array 4. Referring to FIG. 3, in the musical score array 4, the position number is a parameter for specifying the position of the note in the musical score image data 7. Corresponds to the value. That is, a large position number corresponds to a note positioned backward in the time direction, and a small position number corresponds to a note positioned forward in the time direction.

  The musical score array 4 includes a high score part 41 and a low score part 42. In the musical score treble part 41, MIDI scales of notes described in the musical score 31 are arranged in the order of performance. The MIDI scale is a numerical value of a scale defined based on the MIDI standard. In the musical score bass part 42, MIDI scales of notes described in the musical score 32 are recorded in the order of performance.

  For example, in the musical score array 4, the scale of position number “1” indicates the musical scale of the note 311 positioned at the head of the musical score 31. The scale of the note 311 is “So”. In the column of the position number “1” of the musical score treble part 41, the MIDI scale “65” corresponding to “So” is described. Since the musical score 32 does not describe a note corresponding to the musical note 311, a MIDI scale “0” is described in the position number “1” column of the musical score bass part 42.

  The position number “2” of the score array 4 corresponds to the region 3a. For this reason, a plurality of scales constituting a chord are described in the position number “2” column. Specifically, the MIDI scale “67” is described as the scale of the note 312 in the column of the position number “2” of the musical score treble part 41. In the position number “2” column of the musical score bass part 42, the MIDI scale “53” is described as the scale of the note 322. That is, in the musical score array 4, the number of scales constituting the chord is 2 at the maximum.

  As described above, the user plays the music while viewing the score image data 7 displayed on the touch panel 13. The performance position detection apparatus 1 inputs an acoustic signal including a performance sound of a music piece by a user from the microphone 15 and specifies one or more scales from the acoustic signal input from the microphone 15. The performance position detection apparatus 1 creates a performance array 6 including one or more scales arranged in time series.

  FIG. 4 is a diagram showing an example of the performance array 6. Referring to FIG. 4, performance array 6 includes one or more scales specified from the acoustic signal, and one or more scales included in performance array 6 are arrayed in time series. The configuration of the performance array 6 includes a performance treble part 61 and a performance bass part 62, similarly to the score array 4. The maximum number of scales included in the performance array 6 (hereinafter referred to as “array maximum value”) is 4. Here, a chord is counted as one scale. The performance array 6 is updated every time the scale of the performance sound of the music by the user is specified.

  When the user performs “So”, “Shi”, “Shi”, “Mi” in this order, MIDI scales “65”, “67”, “67”, “70” are described in the performance treble part 61. Is done. The second musical performance bass part 62 describes the MIDI scale “53”. That is, the user plays the second sound as a chord.

  The performance position detection apparatus 1 searches the musical score array 4 for the same array as the performance array 6. Specifically, the performance position detection apparatus 1 selects a plurality of reference arrays 5 from the score array 4. The reference array 5 includes the same number of scales as the number of scales included in the performance array 6. The plurality of reference arrays 5 are different from each other in position on the score array 4.

  Referring to FIG. 3, the reference array 51 is a scale whose position numbers are “1” to “4” among the scales of the score array 4. The reference array 52 is a scale having position numbers “2” to “5” among the scales of the score array 4. In this way, the performance position detection apparatus 1 selects a plurality of reference arrays 5 by shifting the position of the leading scale one by one.

  The performance position detection device 1 compares the performance array 6 with each selected reference array 5 to calculate the likelihood that the performance array 6 will be each reference array 5 for each reference array 5. Details of the likelihood calculation method will be described later.

  The performance position detection apparatus 1 identifies the reference array 5 having the maximum likelihood from the likelihoods of the respective reference arrays 5, and determines the position of the last scale of the identified reference array 5 as the performance position. For example, when it is assumed that the reference array 53 has the maximum likelihood, the performance position detection apparatus 1 determines the position of the last scale (position number “8”) in the reference array 53 as the performance position.

  The performance position detection device 1 surrounds the note 318 corresponding to the position number “8” with hatching in the musical score image data 7 displayed on the touch panel 13. As a result, the user can recognize that the musical note 318 has been played. Although FIG. 2 shows an example in which the notes at the performance position are surrounded by hatching, the present invention is not limited to this. Any display method may be used as long as the user can grasp the current performance position on the score displayed on the touch panel 13.

  As described above, the performance position detection apparatus 1 calculates the likelihood that the performance array 6 will be the reference array 5 for each reference array 5, and determines the position of the last scale of the reference array 5 having the maximum likelihood as the performance position. Determine as. Even if the performance array 6 includes a scale due to a performance error, the performance position detection device 1 can quickly specify the performance position without increasing the amount of calculation.

{Details of performance position detection program 2}
Hereinafter, the operation of the performance position detection apparatus 1 when the performance position detection program 2 is executed will be described in detail.

  FIG. 5 is a flowchart of the performance position detection program 2. Referring to FIG. 5, performance position detection apparatus 1 performs initialization (step S1). In step S1, the performance position detection device 1 sets the performance position to “0” and sets all the scales recorded in the performance array 6 to “0”. On the touch panel 13, the first page of the score image data 7 is displayed.

  The performance position detection apparatus 1 determines whether or not to end the performance position detection program 2 (step S2). Specifically, when the performance position matches the position number at the end of the musical score array 4 (Yes in step S2), the performance position detection device 1 determines that the music performance by the user has been completed, and FIG. The processing shown in FIG.

  On the other hand, if the performance position does not match the position number at the end of the musical score array 4 (No in step S2), the performance position detection device 1 determines that the performance by the user has not ended and executes steps S3 to S10. To do.

  The performance position detection apparatus 1 inputs an acoustic signal, and specifies the scale of the performance sound of the music by the user from the input acoustic signal (step S3). The performance position detection apparatus 1 arranges the specified scales in time series and creates a performance arrangement 6 (step S4).

  When the user starts playing, the number of scales specified in step S3 may not reach the array maximum value. In this case, the performance position detection apparatus 1 sequentially adds the scales specified in step S3 to the performance array 6. On the other hand, when the number of scales recorded in the performance array 6 has reached the maximum array value, the performance position detection apparatus 1 deletes the first added scale from the performance array 6 among the scales of the performance array 6. Then, the newly specified scale is added to the performance array 6.

  If the number of scales recorded in the performance array 6 is smaller than the start reference value (No in step S5), the performance position detection device 1 returns to step S2 without determining the performance position. The array maximum value can be used as the starting reference value. By using a value smaller than the array maximum value as the start reference value, the performance position may be determined promptly when the user starts performance. For example, half of the maximum array value can be used as the start reference value.

  On the other hand, if the number of scales recorded in the performance array 6 is equal to or greater than the start reference value (Yes in step S5), the performance position detection device 1 determines that the performance position can be determined, and the score array 4 A plurality of reference sequences 5 are selected from (step S6). As described above, the performance position detection apparatus 1 selects a plurality of reference arrays 5 by shifting the leading numbers of the reference arrays 5 one by one. Accordingly, the number of the reference arrays 5 to be selected varies depending on the number of scales included in the score array 4.

  The performance position detection apparatus 1 calculates the likelihood that the performance array 6 will be the reference array 5 (step S7). The likelihood is calculated for each reference sequence 5. The performance position detection apparatus 1 determines a performance position based on the reference sequence 5 having the maximum likelihood (step S8). The performance position detection apparatus 1 extracts the reference sequence 5 having the maximum likelihood, and specifies the position number of the last scale of the extracted reference sequence 5 as a candidate position for the performance position. This is because the number of reference sequences 5 having the maximum likelihood is not necessarily one. The performance position detection apparatus 1 determines a candidate position that satisfies a predetermined condition as a performance position. Details of the determination of the performance position will be described later.

  The performance position detection apparatus 1 determines the performance position detection state based on the performance position determination result (step S9). Specifically, the performance position detection device 1 determines whether or not the performance position update has been stopped (step S9). When the number of times that the performance position cannot be continuously determined (number of update stops) is equal to or greater than the initialization reference value, or when the number of update stops is equal to or greater than the initialization reference value, the performance position detection device 1 It is determined that the user is not performing according to the score. In this case, the performance position detection apparatus 1 determines that it is unlikely that the performance position can be determined based on the current performance array 6 and initializes the performance array 6.

  The performance position detection apparatus 1 updates the performance position on the score image data 7 displayed on the touch panel 13 based on the updated performance position (step S10).

  Hereinafter, each process of the performance position detection program 2 will be described in detail.

{Musical scale identification process (step S3)}
FIG. 6A is a diagram illustrating an example of an acoustic signal input to the microphone 15. Referring to FIG. 6A, periods 15a to 15d indicate periods in which the user played the instrument, and the amplitude of the acoustic signal varies greatly. Variations in the acoustic signal during other periods indicate noise components.

  The performance position detection apparatus 1 specifies a scale from the acoustic signal shown in FIG. 6A. Specifically, the performance position detection device 1 applies a peak filter to the acoustic signal shown in FIG. 6A. Thereby, as shown to FIG. 6B, the acoustic signal by which the noise component was suppressed is produced | generated. The peak filter is a process of extracting an acoustic signal that fluctuates over a predetermined width from the acoustic signal input to the microphone 15.

  The performance position detection apparatus 1 sets a period in which the amplitude of the acoustic signal subjected to the peak filter is varied as a target to be subjected to the fast Fourier transform. That is, among the acoustic signals shown in FIG. 6B, the acoustic signals in the periods 15a to 15d are set as a period (sampling period) for applying the fast Fourier transform. Here, it is assumed that the acoustic signal 16a shown in FIG. 6B is set in the sampling period. The performance position detection apparatus 1 generates a spectrum shown in FIG. 6C by performing a fast Fourier transform on the acoustic signal 16a.

  FIG. 6D is a diagram showing processing from the spectrum shown in FIG. 6C until the MIDI scale is specified. With reference to FIG. 6C and FIG. 6D, the performance position detection apparatus 1 extracts frequencies having peaks from the spectrum, and sorts the extracted frequencies in descending order of amplitude. That is, the frequencies are sorted in the order of 466 Hz, 415 Hz, 470 Hz,. In the musical score array 4, the maximum value of the number of scales constituting the chord is 2, so the performance position detection device 1 selects the top two frequencies that exceed the reference amplitude among the sorted frequencies. The performance position detecting device 1 converts the selected frequency into a MIDI scale based on a predetermined relationship between the MIDI scale and the frequency. In this way, the MIDI scale is specified from the acoustic signal.

  When the amplitudes of the frequencies 466 Hz and 415 Hz are larger than the reference amplitude, the performance position detection apparatus 1 selects both the frequencies 466 Hz and 415 Hz. The frequency 466 Hz is converted to the MIDI scale “70”, and the frequency 415 Hz is converted to the MIDI scale “68”.

  When the amplitude of the frequency 415 Hz is larger than the reference amplitude and the amplitude of the frequency 466 Hz is smaller than the reference amplitude, the performance position detection device 1 selects only the frequency 415 Hz. The frequency 415 Hz is converted to MIDI scale “68”. In this case, the number of scales specified by the scale specifying process (step S3) is one.

{Performance array creation processing (step S4)}
The performance position detecting device 1 executes the performance arrangement creating process (step S4) after the MIDI scale is specified by the scale specifying process (step S3).

  FIG. 7 is a flowchart of the performance array creation process (step S4). Referring to FIG. 7, performance position detection apparatus 1 checks whether or not the number of scales included in performance array 6 is equal to the array maximum value (step S41).

  When the number of scales is smaller than the maximum arrangement value (No in step S41), the performance position detection device 1 adds the MIDI scale specified in step S3 after the last scale in the performance arrangement 6 (step S42). ), The process shown in FIG.

  Specifically, among the specified MIDI scales, the higher MIDI scale is added to the performance treble part 61, and the lower MIDI scale is added to the performance bass part 62. If the number of specified MIDI scales is one, the specified MIDI scale is added to the performance treble part 62. “0” is added to the performance bass part 62.

  On the other hand, when the number of scales is equal to the array maximum value (Yes in step S41), the performance position detection apparatus 1 executes steps S43 to S45 to update the performance array 6. Steps S43 to S45 will be described in detail with reference to FIGS.

  FIG. 8 is a diagram showing a change in the performance array 6 when the performance array 6 includes a scale having the maximum array value. In FIG. 8, the scales of the musical score array 4 and the performance array 6 are indicated by circles, and the display of the high-frequency part and the low-frequency part is omitted. Assume that the array maximum is 4.

  The performance position detection apparatus 1 deletes the first MIDI scale 6a from the performance array 6 before update (step S43). The element numbers of the MIDI scales 6b to 6d in the performance array 6 are incremented one by one. The performance position detection apparatus 1 adds the newly specified MIDI scale 6e to the performance array 6 from which the MIDI scale 6a has been deleted (step S44). The element number of the MIDI scale 6e is 4. In this way, the performance array 6 is updated.

  The performance position detection device 1 replaces the past scales (MIDI scales 6b to 6d) other than the MIDI scale 6e among the MIDI scales 6b to 6e with the scales of the musical score array 4 corresponding to the MIDI scales 6b to 6d (steps). S45).

  When step S45 is executed, since the performance position has already been determined, it is possible to specify the scale of the musical score array 4 corresponding to the MIDI scales 6b to 6d. The reason why the performance position has already been determined will be described. When the number of MIDI scales included in the performance array 6 is equal to or greater than the start reference value (Yes in step S5), a performance position determination process (step S9) is executed. Therefore, if the number of MIDI scales included in the performance array 6 is the array maximum value, the performance position has already been determined.

  In the performance array 6 after the update, the MIDI scale 6b corresponds to the MIDI scale 4b (position number “73”) of the score array 4. Similarly, the MIDI scales 6c and 6d correspond to the MIDI scales 4c and 4d (position numbers “74” and “75”) of the musical score array 4, respectively. The performance position detection device 1 replaces the MIDI scales 6b, 6c, and 6d in the performance array 6 with the MIDI scales 4b, 4c, and 4d in the score array 4, respectively. Even if the user misses the performance in the past, the performance array 6 does not include the MIDI scale specified from the performance sound due to the performance mistake. Therefore, the performance position determination accuracy can be improved.

{Reference sequence selection process (step S6)}
Hereinafter, the reference sequence selection process (step S6) will be described in detail with reference to FIG. It is assumed that the number of scales included in the performance array 6 is 4, and the previous performance position determined by the performance position determination process (step S9) is the position number “4”.

  Since the number of scales included in the performance array 6 is 4, the performance position detection apparatus 1 sets the number of scales included in each reference array 5 to 4. The performance position detection device 1 selects a plurality of reference arrays 5 from the scales located behind the immediately preceding performance position (position number “4”) among the scales of the musical score array 4. As the performance position advances, the number of reference arrays 5 selected in step S6 decreases, so that the amount of calculation when determining the performance position can be reduced.

  The performance position detection apparatus 1 selects a plurality of reference arrays 5 such that the scale positions in the score array 4 are different from each other. The performance position detection apparatus 1 first selects the scales of the position numbers “5” to “8” of the score array 4 as the reference array 53. Next, the scales of position numbers “6” to “9” of the score array 4 are selected as the reference array 54. Thus, the performance position detection apparatus 1 selects a plurality of reference arrays 5 by shifting the position of the first scale of the reference array 5 one by one.

{Likelihood calculation processing (step S7)}
FIG. 9 is a flowchart of the likelihood calculation process (step S7). FIG. 10 is a diagram for explaining a likelihood calculation method for calculating the likelihood that the performance array 6 will be the reference array 55. Hereinafter, unless otherwise specified, the likelihood calculation process (step S7) will be described using a case where the likelihood that the performance array 6 is the reference array 55 is calculated as an example.

  With reference to FIGS. 9 and 10, the performance position detection apparatus 1 selects a reference array 55 from the plurality of reference arrays 5 selected in step S6 (step S701). The performance position detection apparatus 1 sets the likelihood (L) to 0, sets the element number (S) of the performance array 6 to 1, and sets the number of scales (Ns) included in the performance array 6 to 4. (Step S702).

  The performance position detection apparatus 1 determines whether or not the S-th (first) MIDI scale of the performance array 6 matches the first MIDI scale of the reference array 55 (step S703). Specifically, the performance position detection apparatus 1 confirms that the first MIDI scale of the performance treble part 61 matches the first MIDI scale of the treble part 551 of the reference array 55. In the bass part 552 of the reference array 55, since the first MIDI scale is 0, the performance position detection device 1 does not compare the bass parts. As a result, the performance position detection apparatus 1 determines that the first MIDI scale of the performance array 6 matches the first MIDI scale of the reference array 55 (Yes in step S703). The performance position detection apparatus 1 adds 3 to the likelihood (L) (step S704). The likelihood (L) is 3.

  In step S703, the performance position detection apparatus 1 sets the first MIDI scale of the performance bass part 62 as the first MIDI scale of the bass part 552 even if the first MIDI scale of the bass part 552 is 0. You may compare.

  The performance position detection apparatus 1 increments the element number (S) (step S706), and checks whether the element number (S) is larger than the number of scales (Ns) (step S707). Since the element number (S = 2) is smaller than the number of scales (Ns = 4) (No in step S707), the performance position detection device 1 returns to step S703. Thereafter, steps S703 to S706 are repeated until S = 5 (Yes in step S707).

  The performance position detection apparatus 1 determines that the second MIDI scale of the performance array 6 matches the second MIDI scale of the reference array 55 (Yes in step S703). The performance position detection apparatus 1 adds 3 to the likelihood (L) (step S704). The likelihood (L) is 6.

  The performance position detection apparatus 1 determines that the third MIDI scale in the performance array 6 does not match the third MIDI scale in the reference array 55 (No in step S703). This is because the third MIDI scale of the performance treble part 61 does not match the third MIDI scale of the high pitch part 551. In this case, the performance position detection apparatus 1 subtracts 1 from the likelihood (L) (step S705). The likelihood (L) is 5.

  The performance position detection apparatus 1 determines that the fourth MIDI scale in the performance array 6 matches the fourth MIDI scale in the reference array 55 (Yes in step S703). The performance position detection apparatus 1 adds 3 to the likelihood (L) (step S704). The likelihood (L) is 8.

  As the element number (S) is incremented (step S706), the element number (S = 5) becomes larger than the scale number (Ns = 4) (Yes in step S707). In this case, the performance position detection apparatus 1 stores the likelihood (L = 8) of the reference array 55 (step S708).

  The performance position detection apparatus 1 checks whether or not the likelihoods of all the reference sequences 5 have been calculated (step S709). If the performance position detection device 1 has not calculated the likelihood of all the reference sequences 5 (No in step S709), the performance position detection device 1 returns to step S701 to select another reference sequence 5. Thereby, the performance position detection apparatus 1 can calculate the likelihood (L) for each reference sequence 5.

  When the performance position detection apparatus 1 calculates the likelihoods of all the reference sequences 5 (Yes in step S709), the performance position detection device 1 extracts the reference sequence 5 having the maximum likelihood from the reference sequences 5 (step S710). When the maximum likelihood is 8, the reference sequence 55 is extracted. If there are other reference sequences 5 having a likelihood (L) of 8, a plurality of reference sequences 5 are extracted in step S710.

  The performance position detection apparatus 1 stores the extracted position of the last scale of the reference array 5 as a candidate position in the RAM 12 (step S711). When the reference sequence 55 is extracted, the performance position detection apparatus 1 stores the last scale position (position number “44”) of the reference sequence 55 in the RAM 12 as a candidate position.

{Performance position determination process (step S8)}
FIG. 11 is a flowchart of the performance position determination process (step S8). FIG. 12 is a diagram illustrating the relationship between the immediately preceding performance position and the candidate position. In FIG. 12, the scale of the musical score array 4 is indicated by a circle, and the display of specific numerical values of the scale is omitted.

  Referring to FIGS. 11 and 12, the performance position detection apparatus 1 sets the immediately preceding performance position and allowable value (step S801). The immediately preceding performance position is the performance position determined before the scale is specified by the scale specifying process (step S3). It is assumed that the immediately preceding performance position is the position number “53” of the musical score array 4. Therefore, the previous performance position (pos) is set to 53. The allowable value is a parameter used when determining a candidate position as a performance position. Assume that the tolerance (P) is set to 3.

  It is assumed that candidate positions 251 to 253 have been specified in step S711 (see FIG. 9). The position numbers of the candidate positions 251 to 253 are already stored in the RAM 12.

  The performance position detection apparatus 1 selects a candidate position 251 from the candidate positions 251 to 253 (step S802). The performance position detection apparatus 1 checks whether or not the candidate position 251 is within an allowable range set based on the immediately preceding performance position (step S803). The allowable range is a range that is greater than the position number “53” of the immediately preceding performance position and is equal to or less than the position number “56” obtained by adding the allowable value (P = 3) to the immediately preceding performance position (pos = 53). Since the position number “55” of the candidate position 251 is within the allowable range (Yes in step S803), the performance position detection apparatus 1 continuously stores the position number “55” of the candidate position 251 in the RAM 12 (step S804).

  Since the performance position detection apparatus 1 has not selected all candidate positions (No in step S806), the performance position detection apparatus 1 returns to step S802 to select other candidate positions. That is, the performance position detection apparatus 1 executes the processes of steps S803 to S805 by selecting the candidate positions 252 and 253.

  Since the candidate position 252 exists within the allowable range (Yes in step S803), the position number “56” of the candidate position 252 is continuously stored in the RAM 12 (step S804). Since the candidate position 253 does not exist within the allowable range (No in step S803), the performance position detection apparatus 1 determines that the candidate position 253 is not determined as the performance position, and the position number of the candidate position 253 “59” is erased from the RAM 12. Since the user sequentially plays the notes described in the score, the newly determined performance position is not expected to be far from the previous performance position. The candidate position 253 does not exist within the allowable range, and is too far from the previous performance position. For this reason, the performance position detection apparatus 1 determines that the candidate position 253 is not a performance position.

  When all the candidate positions are selected (Yes in step S806), the performance position detection apparatus 1 checks whether the position number of the candidate position is stored in the RAM 12 (step S807).

  Since the position numbers of the candidate positions 251 and 252 are stored in the RAM 12 (Yes in step S807), the performance position detection apparatus 1 newly selects a candidate position 251 closest to the previous performance position among the candidate positions 251 and 252. The performance position is determined (step S808). The performance position detection apparatus 1 sets the number of update stops to 0 (step S809), and ends the process shown in FIG. The update stop count indicates the number of times that a new performance position has not been determined by the performance position determination process (step S8), and is used in the detection state determination process (step S9).

  On the other hand, when the position number of the candidate position is not stored in the RAM 12 (No in step S807), the performance position detection device 1 determines that the performance position cannot be newly determined. The performance position is not updated from the previous performance position. The performance position detection apparatus 1 counts the number of times the update is stopped (step S810) and ends the process shown in FIG.

  As described above, the performance position detection device 1 determines a candidate position closest to the previous performance position as a new performance position among the candidate positions existing within the allowable range. As a result, the performance position is prevented from largely moving from the previous performance position.

{Detection state determination process (step S9)}
FIG. 13 is a flowchart of the detection state determination process (step S9). FIG. 14 is a diagram illustrating a change in the performance position when the performance position is forcibly changed by the detection state determination process (step S9). In FIG. 14, the scale of the musical score array 4 is indicated by a circle, and the display of specific numerical values of the scale is omitted.

  Referring to FIG. 13, performance position detection apparatus 1 determines whether or not the number of update stops is equal to or greater than an initialization reference value (for example, 4) (step S91). It is assumed that the immediately preceding performance position is the position number “56” of the musical score array 4 as shown in FIG.

  If the number of update stops is equal to or greater than the initialization reference value (Yes in step S91), the performance position detection apparatus 1 cannot determine the performance position based on the current performance array 6 because the user continues to make mistakes. Is determined. The performance position detection device 1 initializes the performance array 6 in order to redo the determination of the performance position (step S92). The performance position detection apparatus 1 changes the performance position to the position of the musical scale that is three notes before the previous performance position (position number “56”) (step S93). When the value of N is set to 3, the position number “53” is determined as a new performance position. Note that the value of N may be a value other than 3.

  If the user misses the performance, it is considered that the performance is stopped and the performance is resumed from a position before the position where the performance was missed (for example, the first note of the measure where the performance was missed). For this reason, the performance position detection device 1 changes the performance position in step S93 so that when the user resumes the performance, a new performance position can be determined from a position before the previous performance position. . Further, since the performance array 6 is initialized in step S92, the number of scales included in the performance array 6 is zero. Since the performance position detection apparatus 1 can newly create the performance array 6 based on the resumed performance, the performance position corresponding to the resumed performance can be quickly determined.

  On the other hand, if the number of update stops is smaller than the initialization reference value (No in step S91), the performance position detection device 1 cannot determine the performance position due to a user's performance mistake, but determines that the user continues to perform. To do. In this case, the performance position detection apparatus 1 ends the process shown in FIG. 13 without initializing the performance array 6.

  Referring to FIG. 5, when the performance position is changed by the performance position determination process (step S <b> 8) or the detection state determination process (step S <b> 9), the performance position detection device 1 displays the score image displayed on the touch panel 13. The performance position on the data 7 is changed. Since the performance position on the score image data 7 advances on the score according to the performance of the user, the user can easily confirm which note on the score is being played.

{Modification}
Hereinafter, modifications of the above embodiment will be described.

  In the said embodiment, although the performance position detection apparatus 1 demonstrated the example which specifies the position of the last scale of the reference arrangement | sequence 5 which has the maximum likelihood as a candidate position, it is not restricted to this. In step S710 (see FIG. 9), the performance position detection apparatus 1 determines the position of the last musical scale of the reference sequence 5 having the maximum likelihood when the number of the reference sequences 5 having the maximum likelihood is one. The position may be determined. For example, referring to FIG. 3 and FIG. 9, when only the reference sequence 53 is extracted as the reference sequence 5 having the maximum likelihood (step S710), the performance position detection device 1 determines the last scale of the reference sequence 53. The position (position number “8”) may be determined as the performance position.

  In the above embodiment, the example in which the reference array 5 is selected from the scales located behind the immediately preceding performance position among the scales of the score array 4 has been described, but the present invention is not limited to this. The performance position detection apparatus 1 may select the reference array 5 from the entire score array 4 regardless of the previous performance position. In other words, the performance position detection device 1 may select a reference array from among the musical score array 4 that includes a scale located behind the previous performance position.

  Even in this case, referring to FIG. 11 and FIG. 12, a candidate position located behind the immediately preceding performance position is determined as the performance position in step S803. Even if the reference array 5 is selected from the entire score array 4, it is possible to prevent the scale located before the previous performance position from being determined as a new performance position. In other words, the performance position detection device 1 is the last when the position of the last scale of the reference array 5 having the maximum likelihood is located behind the previous performance position in the time direction regardless of the selection method of the reference array 5. The scale position may be determined as a new performance position.

  In the above embodiment, the example in which the allowable range is set behind the immediately preceding performance position has been described. However, as shown in FIG. 15, the performance position detection apparatus 1 may set the allowable range so as to include the scale before the previous performance position. For example, it is conceivable that the user interrupts the performance without making a mistake in the performance and restarts the performance from a position before the position where the user interrupted. Even in this case, the performance position detection apparatus 1 can determine an accurate performance position by setting the allowable range before and after the immediately preceding performance position. That is, the performance position detection apparatus 1 determines the position of the last musical scale as a new performance position when the last musical scale of the reference array 5 having the maximum likelihood is within the allowable range. In this case, the performance position detection device 1 needs to select the reference array 5 from the scale of the musical score array before the previous performance position. Further, when the last scale is located outside the predetermined range, the performance position detection device 1 counts the number of times of updating stop (see step S810, FIG. 11).

  In the above embodiment, in the performance position determination process (step S8), the performance position detection apparatus 1 sets a candidate position closest to the previous performance position as a new performance position among the candidate positions existing within the allowable range. The example to determine was demonstrated (refer step S803, S811, FIG. 11). However, the performance position detection apparatus 1 may determine a candidate position closest to the immediately preceding performance position as a new performance position without determining whether the candidate position is within the allowable range.

  In the above embodiment, the example has been described in which the musical score array 4 includes a single scale and a plurality of scales constituting a double chord. However, the present invention is not limited to this. The musical score array 4 may include a plurality of scales constituting a triple chord, a quad chord, and the like. That is, in the musical score array 4, a plurality of scales constituting a chord are described in the same position number column. In the scale specifying process (step 3), out of the sorted frequencies, the number of frequencies equal to the maximum number of scales constituting the chord is extracted in the score array 4. In the performance array creation process (step S4), the performance position detection device 1 may add a plurality of scales specified from the acoustic signal to the performance array 6.

  In the said embodiment, the example in which the performance position detection program 2 was installed in the tablet type terminal was demonstrated. The method for installing the performance position detection program 2 is not particularly limited. For example, the performance position detection program 2 may be downloaded from a server connected to a network and installed in a tablet terminal. Alternatively, when a computer-readable medium (for example, an optical disk, a USB (Universal Serial Bus) memory, a flexible disk, etc.) on which the performance position detection program 2 is recorded is distributed, the performance position detection program 2 is downloaded from the medium to the tablet. May be installed on the terminal.

  In the above embodiment, the case of a tablet terminal has been described as an example, but the present invention is not limited to this. The performance position detection apparatus 1 may be a computer such as a notebook PC (Personal Computer), a desktop PC, a smartphone, or a mobile phone. That is, the performance position detection device 1 may be any device that includes the microphone 15 and the touch panel 13 and can execute the performance position detection program 2.

  While the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and can be implemented by appropriately modifying the above-described embodiment without departing from the spirit thereof.

DESCRIPTION OF SYMBOLS 1 Performance position detection apparatus 2 Performance position detection program 3 Musical score data 4 Musical score arrangement | sequence 5, 51-55 Reference arrangement | sequence 6 Performance arrangement | sequence 11 CPU
12 RAM
13 Touch Panel 14 Operation Button 15 Microphone 31, 32 Musical Score 41 Musical Score High Part 42 Musical Score Low Part 61 Performance High Frequency Part 62 Performance Bass 311-319, 322, 326 Musical Note

Claims (8)

A performance position detecting device for detecting a performance position on the score when a user plays the music based on score data in which notes described in the score of the music are recorded,
A storage unit for storing a musical score arrangement including the scales of the notes arranged in the order of performance;
An acoustic signal including a performance sound of the music by the user is input, and a scale specifying unit that specifies one or more scales from the input acoustic signal;
An array creating unit for creating a performance array including the one or more scales arranged in time series;
A selection unit that selects a plurality of reference arrays, each of which includes the same number of scales as the number of scales included in the performance array, and has different positions on the score array from the score array;
A likelihood calculating section for calculating the likelihood that the performance array will be the selected reference array for each reference array;
When the reference sequence having the maximum likelihood is identified from among the selected reference sequences, and the last scale of the reference sequence having the maximum likelihood is located within a predetermined range based on the immediately preceding performance position A performance position determining unit that determines the position of the last scale as the performance position, and determines that the performance position cannot be determined when the last scale is outside the predetermined range ;
If the performance position determination unit determines that the performance position cannot be determined, a counting unit that counts the number of times of updating stop indicating that the performance position is not updated;
A performance position detection apparatus comprising: an array initialization unit that initializes the performance array when the number of update stops counted by the count unit exceeds a threshold value . The performance position detection device according to claim 1,
The playing position determination unit, the position of the last of the scale located behind at the time direction from the performance position of the immediately preceding is determined as the playing position, the playing position detecting device. The performance position detection device according to claim 1 or 2,
When the plurality of candidate sequences are specified as the reference sequence having the maximum likelihood, the performance position determination unit determines the position of the last scale closest to the previous performance position among the positions of the last scale of each candidate sequence. A performance position detection device that determines a performance position. It is a performance position detection apparatus of any one of Claims 1-3 ,
The array creation unit creates the performance array newly when the performance array is initialized by the array initialization unit,
The performance position determination device, wherein the performance position determination unit determines, as the performance position, a position of a scale located a predetermined number of times before the previous performance position. A performance position detecting device for detecting a performance position on the score when a user plays the music based on score data in which notes described in the score of the music are recorded,
A storage unit for storing a musical score arrangement including the scales of the notes arranged in the order of performance;
An acoustic signal including a performance sound of the music by the user is input, and a scale specifying unit that specifies one or more scales from the input acoustic signal;
An array creating unit for creating a performance array including the one or more scales arranged in time series;
A selection unit that selects a plurality of reference arrays, each of which includes the same number of scales as the number of scales included in the performance array, and has different positions on the score array from the score array;
A likelihood calculating section for calculating the likelihood that the performance array will be the selected reference array for each reference array;
A performance position determining unit for identifying a reference sequence having the maximum likelihood from among the selected reference sequences, and determining a position of the last scale of the reference sequence having the maximum likelihood as the performance position;
The array creation unit
When the scale specifying unit specifies the one or more scales, an adding unit that adds the specified one or more scales to the performance arrangement;
A replacement unit that replaces past scales other than the one or more scales added by the addition unit among the scales included in the performance sequence, with scales of the score sequence corresponding to the past scales; Performance position detection device. It is a performance position detection apparatus of any one of Claims 1-5 , Comprising:
The musical score arrangement includes a plurality of scales constituting a chord,
The scale specifying unit specifies a plurality of scales constituting a chord from the acoustic signal,
The performance position detection device, wherein the sequence creation unit creates a performance sequence including a plurality of scales specified by the scale specification unit. A performance for causing a computer having a storage device to execute a process of detecting a performance position on the score when a user plays the song based on score data in which notes described in the score of the song are recorded A position detection program,
Reading out from the storage device a musical score arrangement including scales of the notes arranged in order of performance;
Inputting an acoustic signal including a performance sound of the music by the user;
Identifying one or more scales from the input acoustic signal;
Creating a performance array including the one or more scales arranged in time series;
Selecting a plurality of reference sequences each of which includes the same number of scales as the number of scales included in the performance sequence and having different positions on the score sequence from among the read score sequences;
Calculating the likelihood that the performance sequence will be each selected reference sequence for each reference sequence;
Identifying a reference sequence having the maximum likelihood among each of the selected reference sequences ;
Determining the position of the last scale as the performance position when the last scale of the reference sequence having the maximum likelihood is located within a predetermined range with respect to the previous performance position;
Determining that the performance position cannot be determined when the last scale is located outside the predetermined range;
If it is determined that the performance position cannot be determined, counting the number of update stops indicating that the performance position is not updated;
A performance position detection program comprising: initializing the performance array when the counted number of update stops exceeds a threshold value .   A performance for causing a computer having a storage device to execute a process of detecting a performance position on the score when a user plays the song based on score data in which notes described in the score of the song are recorded A position detection program,
  Reading out from the storage device a musical score arrangement including scales of the notes arranged in order of performance;
  Inputting an acoustic signal including a performance sound of the music by the user;
  Identifying one or more scales from the input acoustic signal;
  Creating a performance array including the one or more scales arranged in time series;
  Selecting a plurality of reference sequences each of which includes the same number of scales as the number of scales included in the performance sequence and having different positions on the score sequence from among the read score sequences;
  Calculating the likelihood that the performance sequence will be each selected reference sequence for each reference sequence;
  Identifying a reference sequence having the maximum likelihood from among the selected reference sequences, and determining the position of the last scale of the reference sequence having the maximum likelihood as the performance position,
  The step of creating the performance array includes:
  If the one or more scales are identified, adding the identified one or more scales to the performance arrangement;
  A performance position detection program comprising: replacing a scale other than the one or more scales to be added with a scale of the score array corresponding to the past scale among the scales included in the performance array. .

Images