JP5678935B2 - Musical instrument performance evaluation device, musical instrument performance evaluation system - Google Patents

Description

  The present invention relates to a musical instrument performance evaluation apparatus for evaluating performance of musical instruments.

  As a conventional technique for evaluating the performance of a musical instrument, a plucked string musical instrument performance evaluation apparatus described in Patent Document 1 is known. In the prior art described in Patent Document 1, it is said that the performance of the plucked string instrument can be evaluated by comparing the performance sound of the plucked string instrument converted into an electric signal with a predetermined evaluation standard. For this purpose, the karaoke apparatus is provided with a chord database that stores various information related to chords (chords) of plucked strings as performance evaluation information, in addition to a database that stores song data for outputting karaoke accompaniment music. It is disclosed.

JP 2011-69900 A

  In the prior art described above, a chord database designed exclusively is used to evaluate the chord of a plucked string instrument. In such a conventional technique, it is necessary to design a database for evaluation corresponding to each musical instrument every time the number of types of musical instruments to be evaluated is increased.

  However, if a database for evaluation corresponding to a plurality of types of musical instruments is prepared for a huge number of songs, a problem arises that a great deal of development resources are required. In addition, there is a problem that the storage capacity of the musical instrument performance scoring apparatus having a scoring database is consumed.

  The present invention has been made to solve the above problems. The purpose is to provide a technique for enabling performance evaluation by various instruments without using a database dedicated to scoring.

The musical instrument performance evaluation apparatus of the present invention made to achieve the above object comprises a MIDI storage means, a first determination means, a second determination means, a conversion means, and an evaluation means. The MIDI storage means stores MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments . The first determining means determines a musical instrument connected to the musical instrument performance evaluation apparatus. The second determining means determines a MIDI channel in the MIDI data of the karaoke song corresponding to the instrument determined by the first determining means. In accordance with the note information belonging to the MIDI channel determined by the second determination means in the MIDI data stored in the MIDI storage means, the conversion means converts the note information into the stroke timing, stroke time, and stable time point of the musical performance sound. to convert the evaluation reference data, including. The evaluation means includes a musical instrument sound including a stroke timing, a stroke time, and a stable time point based on an input signal of a musical instrument sound input from a musical instrument connected to the musical instrument performance evaluation apparatus and the evaluation reference data converted by the conversion means. Evaluate the performance.

The performance information indicated by the MIDI data is assigned to a plurality of MIDI channels. In general, one or a plurality of MIDI channels correspond to one part (one musical instrument) in music performance. Therefore, in the present invention, the MIDI channel corresponding to the musical instrument connected to the musical instrument performance evaluation apparatus is determined, and the note information of the MIDI data belonging to the MIDI channel is converted into the evaluation reference data, thereby evaluating the performance of the musical instrument. It can be carried out. In this way, since MIDI data used for performance of a song can be used for evaluation of musical instrument performance, data dedicated for evaluation is not required, and data development resources and storage capacity of the musical instrument performance scoring device can be saved. . Even if the types of musical instruments to be evaluated are increased, if the musical instrument is assigned to a MIDI channel, the musical instrument is created using existing MIDI data without adding new data dedicated to evaluation. The performance can be evaluated.

  Note that, as the evaluation reference data converted from the note information of the MIDI data, as shown in claim 2, there is data indicating the sound feature amount derived by a predetermined analysis method. Alternatively, the evaluation reference data converted from the note information of the MIDI data includes data indicating a sound waveform as described in claim 3.

  By the way, as a method for evaluating performance in the case where both chords (chords) and solos (single notes) can be performed by a plucked string instrument such as a guitar, it is conceivable to configure as described in claim 4. That is, the musical instrument performance evaluation apparatus further includes a chord storage unit and a determination unit. The chord storage means stores chord evaluation reference data for evaluating chords played by a musical instrument connected to the musical instrument performance evaluation apparatus. The determination means determines whether the musical instrument sound input by the musical instrument connected to the musical instrument performance evaluation apparatus is a chord or a single sound.

  When the determination means determines that the chord is a chord, the evaluation means includes an instrument sound input signal input from a musical instrument connected to the musical instrument performance evaluation apparatus, and chord evaluation reference data stored in the chord storage means. Based on the above, the performance of the musical instrument is evaluated. On the other hand, when it is determined that the sound is a single sound by the determination means, the evaluation means is based on the input signal of the instrument sound input from the musical instrument connected to the musical instrument performance evaluation apparatus and the evaluation reference data converted by the conversion means. To evaluate the performance of the instrument.

  According to such a configuration, when a solo part is played with a single note by an instrument, data dedicated to scoring of the solo part is not required, and MIDI data used for playing a song can be used for evaluation of the performance of the instrument. it can. On the other hand, when a chord is played by a musical instrument, the performance of the musical instrument is evaluated using chord evaluation reference data prepared separately. For example, when playing a chord with a guitar, by continuously playing a plurality of strings provided in the guitar in a predetermined direction, sounds are sequentially generated from the strings, and these sounds overlap to form a chord. In this way, the chord pronunciation process in a plucked string instrument such as a guitar is more complicated than a single note. However, there is a case where even the complicated sound production process of the chord is not reproduced in the performance information of the MIDI data. Therefore, when a chord is played by an instrument, the performance evaluation is performed more accurately than by using an evaluation standard converted from MIDI data by using dedicated high-precision chord evaluation reference data for performance evaluation. It can be performed.

  Alternatively, when the chord and the solo are clearly separated as parts according to the type of the musical instrument, such as a lead guitar (solo) and a side guitar (chord), it may be configured as described in claim 5. . That is, the musical instrument performance evaluation apparatus further includes code storage means for storing chord evaluation reference data for evaluating a chord played by a musical instrument connected to the musical instrument performance evaluation apparatus. The first determining means has a function of determining a main guitar and a sub guitar as musical instruments connected to the musical instrument performance evaluation apparatus.

  When the main determining guitar is determined by the first determining means, the evaluating means converts the instrument sound input signal from the instrument connected to the musical instrument performance evaluation apparatus and the evaluation reference data converted by the converting means. Based on this, the performance of the instrument is evaluated. On the other hand, when the sub-guitar is determined by the first determination means, the evaluation means includes the input signal of the instrument sound input from the instrument connected to the instrument performance evaluation device and the chord evaluation reference data stored in the chord storage means. Based on the above, the performance of the musical instrument is evaluated.

  According to such a configuration, when the main guitar (single tone dedicated guitar: lead guitar) is determined as the musical instrument to be played, the MIDI data used for the performance of the song is not required, and the data dedicated to the solo part scoring is not required. It can be used to evaluate the performance of musical instruments. On the other hand, when a sub-guitar (a guitar dedicated to chords: a side guitar) is determined as a musical instrument to be played, it is converted from MIDI data by evaluating the performance of the musical instrument using separately prepared chord evaluation reference data. The performance can be evaluated with higher accuracy than using the evaluation criteria.

As a specific application of the instrument performance evaluation apparatus of the present invention, the karaoke apparatus and the like to perform playing karaoke based on the M IDI data. According to the karaoke apparatus to which the present invention is applied, it is possible to evaluate the performance of musical instruments corresponding to various songs by utilizing a large number of MIDI data resources used in the performance of karaoke songs.

Next, an invention according to claim 6 relates to a musical instrument performance evaluation system including an information processing device and an evaluation server. The information processing apparatus is inputted using a first determination means for determining a musical instrument connected to the information processing apparatus, a notification means for notifying the evaluation server of the musical instrument determined by the first determination means, and the information processing apparatus. Transmitting means for transmitting the input signal of the instrument sound to the evaluation server.

On the other hand, the evaluation server includes MIDI storage means for storing MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments, and an information processing apparatus. Belongs to the MIDI channel determined by the second determination means in the MIDI data stored in the MIDI storage means and the second determination means for determining the MIDI channel in the MIDI data of the karaoke song corresponding to the musical instrument notified from Conversion means for converting the note information into evaluation reference data including the stroke timing, stroke time, and stable time point of the musical performance sound according to the note information , the input signal of the instrument sound transmitted from the information processing apparatus, and the conversion means based on the converted evaluation criteria data, the stroke Thailand And a evaluation means for evaluating a performance of the musical instrument comprising a stable point and the ring and the stroke time.

  In the musical instrument performance evaluation system configured as described above, the information processing apparatus is responsible for determining a musical instrument to be played and inputting musical instrument sound. The evaluation server determines a MIDI channel corresponding to the musical instrument to be played, converts MIDI data based on the determined MIDI channel into evaluation reference data, and evaluates performance based on the converted evaluation reference data. Bear. As described above, with the configuration in which the information processing device and the evaluation server share the role related to performance evaluation, the same effect as the musical instrument performance evaluation device according to claim 1 can be obtained.

Alternatively, the musical instrument performance evaluation system including the information processing device and the evaluation server may be configured as described in claim 7 . The information processing apparatus determines a musical instrument connected to the musical instrument performance evaluation apparatus, and determines a MIDI channel in the MIDI data of the karaoke song corresponding to the musical instrument determined by the first determination means. Means, a notifying means for notifying the evaluation server of the MIDI channel determined by the second determining means, and a transmitting means for transmitting the input signal of the musical instrument sound input using the information processing apparatus to the evaluation server.

On the other hand, the evaluation server includes MIDI storage means for storing MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments, and a MIDI storage means. Conversion for converting note information into evaluation reference data including stroke timing, stroke time, and stable time point of musical performance sound according to the note information belonging to the MIDI channel notified from the information processing device in the MIDI data stored in The performance of the musical instrument including the stroke timing, the stroke time, and the stable time point is evaluated based on the means, the input signal of the instrument sound transmitted from the information processing apparatus, and the evaluation reference data converted by the conversion means. And an evaluation means.

In the musical instrument performance evaluation system configured as described above, the information processing apparatus is responsible for determining a musical instrument to be played, determining a MIDI channel corresponding to the determined musical instrument, and inputting instrument sound. The evaluation server is responsible for conversion from MIDI data based on the determined MIDI channel to evaluation standard data and performance evaluation based on the converted evaluation standard data. As described above, with the configuration in which the information processing device and the evaluation server share the role related to performance evaluation, the same effect as the musical instrument performance evaluation device according to claim 1 can be obtained.

1 is a block diagram showing a schematic configuration of a karaoke system 1. FIG. The functional block diagram which shows the principal part of the performance evaluation function of the karaoke apparatus. Explanatory drawing which shows an example of a musical instrument-MIDI channel correspondence table. The flowchart which shows the procedure of a performance evaluation process. The flowchart which shows the procedure of evaluation reference | standard data conversion. The flowchart which shows the procedure of music analysis. The flowchart which shows the procedure of an input sound analysis.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is possible to implement in various aspects.
[Description of configuration of karaoke system 1]
As shown in FIG. 1, the karaoke system 1 includes a karaoke device 10 installed in a karaoke store such as a karaoke box, and an external karaoke server 30. The karaoke apparatus 10 is connected to a karaoke box LAN (hereinafter simply referred to as “LAN”) 40. The LAN 40 is connected to the router 50. Thereby, the karaoke apparatus 10 can perform data communication with the karaoke server 30 via the LAN 40 and the router 50 via the Internet 60. In FIG. 1, only one karaoke device 10 is illustrated for easy understanding, but a plurality of karaoke devices 10 may be connected to the LAN 40 in the karaoke store. Moreover, the network in karaoke store which consists of the karaoke apparatus 10, LAN40, and the router 50 which were each installed in the some karaoke store may be connected to the internet 60.

  The karaoke apparatus 10 is mainly configured of a control unit 11 that is a computer that performs overall control of the apparatus. The control unit 11 is an information processing device mainly composed of a CPU, a ROM, a RAM, an I / O, and the like, and executes various processes related to the karaoke service based on programs and data read from the HDD 19. As a main function of the karaoke apparatus 10, the accompaniment music of the karaoke music requested by the user via the operation unit 14a is played. At this time, the control unit 11 displays the background image and the lyrics telop corresponding to the music to be played on the display 25. Further, the controller 11 performs a performance evaluation process for evaluating the instrument sound input from the instrument 26 connected to the karaoke apparatus 10 via the instrument sound input unit 21 when the karaoke music piece is played. Detailed contents of this performance evaluation process will be described later.

  The control unit 11 is connected to the bus line 12, and the audio control unit 13 is connected to the control unit 11. An operation processing unit 14, an interface unit 16, a video RAM 17, a video reproduction unit 18, and a hard disk drive (HDD) 19 are connected to the bus line 12. In addition, a microphone 20, a MIDI sound source 22, and a speaker 23 are connected to the audio control unit 13. The operation processing unit 14 processes a signal from the operation unit 14a. From the operation unit 14a, information such as a music number for specifying music data to be played, a performance speed of the music data, and a performance key is input by the user. The operation unit 14a is embodied as, for example, a remote controller that performs wireless communication with the apparatus main body. The interface unit 16 is configured to connect the karaoke apparatus 10 to the LAN 40. Thereby, as described above, data communication with the karaoke server 30 via the Internet 60 becomes possible.

The video RAM 17 temporarily stores video information. The video RAM 17 is connected to the video reproduction unit 18. The video playback unit 18 plays back video based on video information from the video RAM 17 and video data transmitted via the bus line 12. The video reproduced by the video reproduction unit 18 includes a background and a lyrics image displayed together with the karaoke performance. A video control unit 24 is connected to the video playback unit 18. A display 25 is connected to the video control unit 24. The display 25 is embodied as a monitor device. With such a configuration, the video control unit 24 can display the video output from the video playback unit 18 on the display 25.

  The HDD 19 is a so-called hard disk drive device, and stores various data such as a karaoke database in which a large number of karaoke music data are recorded, system programs and application programs for controlling the operation of the karaoke device 10, and data used for reproducing various information contents. Remember.

  Next, a configuration connected to the voice control unit 13 will be described. The microphone 20 is an input device for inputting karaoke singing sound by a user. The MIDI sound source 22 generates an audio signal based on MIDI (Musical Instrument Digital Interface) data. The music data stored in the HDD 19 includes MIDI data indicating music performance information. The speaker 23 converts the audio signal output from the audio control unit 13 into sound and outputs the sound.

  The sound control unit 13 converts the music data stored in the HDD 19 into an audio signal via the MIDI sound source 22. The voice control unit 13 converts an external input sound from the microphone 20 and an external input sound from the musical instrument 26 input through the control unit 11 into an audio signal. Then, the voice control unit 13 outputs the converted audio signal to the speaker 23.

  A musical instrument sound input unit 21 is connected to the control unit 11. Thereby, the control unit 11 acquires the musical instrument sound played by the user using the musical instrument 26. The musical instrument sound input unit 21 includes a microphone, a terminal directly connected to the musical instrument 26, an amplifier 71, an A / D converter 72, and the like. The musical instruments 26 connected to the musical instrument sound input unit 21 include stringed instruments, percussion instruments, wind instruments, keyboard instruments, and the like. Further, the instrument sound input unit 21 may be configured as a separate device from the karaoke device 10 or may be configured as an integrated device incorporated with the karaoke device 10. Since other functions and configurations of the karaoke apparatus 10 are in accordance with known techniques, detailed description thereof is omitted here.

  Next, the configuration of the karaoke server 30 will be described. The karaoke server 30 is a known server device having appropriate information processing capability. The karaoke server 30 performs information communication with the karaoke device 10 via the Internet 60 and executes a predetermined service (processing) in response to a request from the karaoke device 10. The karaoke server 30 has a database 31. This database 31 holds MIDI data of karaoke music. Detailed contents of the processing performed by the karaoke server 30 will be described later.

[Description of the main part of the performance evaluation function]
Next, the main part of the performance evaluation function of the karaoke apparatus 10 will be described with reference to FIG. Among the components shown in FIG. 2, the amplifier 71 and the A / D converter 72 are provided in the instrument sound input unit 21. Further, the instrument sound analysis means 73, the evaluation channel determination means 74, the evaluation reference data conversion means 76, the discrete evaluation means 77 (the reference stroke period reference means 78, the instrument sound evaluation means 79), the performance evaluation means 80, and the evaluation result The display unit 81 is configured by a control function provided in the control unit 11.

  The electric signal of the instrument sound input from the instrument 26 connected to the karaoke apparatus 10 via the instrument sound input unit 21 is amplified by the amplifier 71 and then converted to a digital audio signal by the A / D converter 72. The musical instrument sound converted into the digital audio signal is supplied to the musical instrument sound analyzing means 73.

  The musical instrument sound analyzing means 73 performs frequency analysis on the digital audio signal of the musical instrument sound, and calculates the characteristic amount of the musical instrument sound. Here, as a frequency analysis of the musical instrument sound, a known fast Fourier transformation (FFT) and chroma vector feature (Chroma Vector) are calculated. As the procedure, first, the digital audio signal of the instrument sound supplied from the A / D converter 72 is developed into frequency components by fast Fourier transform. Next, the developed frequency component is applied to a predetermined comb filter (a filter through which the center frequency of each scale passes) to be separated into a power spectrum for each scale to some extent. Then, a chroma vector feature amount as a frequency analysis result is obtained by adding the power spectrum between a plurality of octaves for each scale. Such a frequency analysis technique is a well-known technique (see, for example, Patent Document 1).

  On the other hand, the evaluation channel determination means 74 determines a MIDI channel to be used as an evaluation standard for musical instrument performance. The MIDI data of the karaoke music stored in the karaoke database 75 of the HDD 19 includes performance information allocated to a plurality of MIDI channels. In general, one or a plurality of MIDI channels correspond to one part (one musical instrument) in music performance. In the present embodiment, it is assumed that the musical instrument 26 that can be connected to the karaoke apparatus 10 is associated with a plurality of MIDI channels that constitute MIDI data of karaoke music. The evaluation channel determination means 74 determines the MIDI channel corresponding to the type of the musical instrument 26 determined as the musical instrument to be used for performance as the evaluation reference MIDI channel.

  The correspondence relationship between the musical instrument used for performance and the MIDI channel is defined in advance in a musical instrument-MIDI channel correspondence table as shown in FIG. 3 and stored in a predetermined storage area such as the HDD 19. As shown in FIG. 3, the musical instrument-MIDI channel correspondence table is described in association with MIDI channels corresponding to parts played by the musical instrument for each type of musical instrument. The evaluation channel determination means 74 determines an evaluation reference MIDI channel corresponding to the musical instrument used for performance based on the musical instrument-MIDI channel correspondence table.

  Returning to the description of FIG. The evaluation standard data conversion means 76 generates evaluation standard data in a predetermined format when outputting a karaoke performance based on music data stored in the karaoke database 75 of the HDD 19. Specifically, the evaluation reference data conversion means 76 converts note information belonging to the MIDI channel determined by the evaluation channel determination means 74 from the MIDI data to be played into evaluation reference data. The evaluation reference data conversion means 76 performs frequency analysis on the digital audio signal of the karaoke music generated from the note information of the evaluation reference MIDI channel, and calculates the characteristic amount of the performance sound of the karaoke music as the evaluation reference data. In this embodiment, as the frequency analysis of the performance sound, the above-described fast Fourier transform and calculation of the chroma vector feature amount are performed. Moreover, the evaluation reference data conversion means 76 determines the stroke timing, stroke time, and stable time point of the performance sound of karaoke music.

The discrete evaluation means 77 compares the sound feature amount obtained from the result of the frequency analysis of the instrument sound with the sound feature amount indicated by the evaluation reference data, so that the unit performance is performed a plurality of times at predetermined time intervals for each unit performance. Perform a discrete evaluation of Among these, the reference stroke period reference means 78 refers to a reference stroke period indicating a period corresponding to a unit performance for one stroke with respect to the performance based on the MIDI data of the karaoke music piece. The musical instrument sound corresponding to the reference stroke period referred to by the reference stroke period reference means 78 is subjected to frequency analysis a plurality of times at predetermined time intervals (for example, every 0.1 second) by the instrument sound analysis means 73. Then, the instrument sound evaluation means 79 compares the correspondence between the frequency analysis result at each time point and the frequency analysis result in the evaluation reference data corresponding to the reference stroke period. As a result of the comparison, the musical instrument sound evaluation means 79 determines a chord or a single sound at each time point. And based on the determination result of each sound in these multiple times of discrete evaluation, the chord or single sound played during the reference stroke period is determined. For example, a chord or a single tone having the largest number of determinations among chords or single notes obtained as a result of a plurality of discrete evaluations is determined as a chord or a single note played during the reference stroke period.

  The performance evaluation means 80 includes the chord or single sound determined by the discrete evaluation means 77, the output timing thereof, and the chord or single sound defined in the note information belonging to the evaluation reference MIDI channel in the MIDI data relating to the performance of karaoke music. And its output timing are compared. Based on the comparison result, the skill of playing the musical instrument 26 in the reference stroke period is evaluated. Such a performance evaluation method is a well-known technique (see, for example, Patent Document 1).

  The evaluation result display unit 81 causes the display 25 to display a video regarding the evaluation result obtained by the performance evaluation unit 80 via the video control unit 24. The evaluation result display means 81 compares the correct sound (chord or single sound) based on the MIDI data of the music and the chord or single sound actually played by the performer with the musical instrument 26 as a video relating to the evaluation result by the performance evaluation means 80. Displaying in a possible manner is an example. Such an image related to the evaluation result may be displayed in synchronization with the karaoke performance, or is displayed in association with the timing, lyrics, or the like related to the evaluation result after the karaoke performance is completed. May be. Moreover, you may display the score calculated combining the evaluation result for 1 music at the time of a karaoke performance being complete | finished.

[Description of performance evaluation process]
The procedure of performance evaluation processing executed by the control unit 11 of the karaoke apparatus 10 will be described based on the flowchart of FIG. This process is a process executed in parallel with the process of performing a performance based on the MIDI data of the karaoke music requested by the user under the condition that the mode for evaluating the musical instrument performance is specified.

  In S100, the control unit 11 determines the type of musical instrument used for performance. Specifically, the type of the musical instrument 26 connected to the musical instrument sound input unit 21 may be determined and determined as a musical instrument used for performance, or the musical instrument input from the user through the operation unit 14a. The instrument used for the performance may be determined from the type. In the present embodiment, as an example, one of the musical instrument 1 (main guitar), the musical instrument 2 (sub guitar), the musical instrument 3 (bass), and the musical instrument 4 (drum) is determined as a musical instrument used for performance. I will give you.

In the next S101, the control unit 11 branches the process according to the type of instrument determined in S100. When the musical instrument 1 (main guitar) is determined as the musical instrument to be used for performance, the control unit 11 proceeds to S102 corresponding to the operations of the evaluation channel determining unit 74 and the evaluation reference data converting unit 76 described above. In S102, the control unit 11 evaluates the note information of the MIDI channel corresponding to the musical instrument 1 (main guitar) out of the MIDI data of the karaoke piece to be played based on the musical instrument-MIDI channel correspondence table (see FIG. 3). Process to convert to data. When the musical instrument 2 (sub guitar) is determined as the musical instrument to be used for performance, the control unit 11 evaluates the MIDI channel note information corresponding to the musical instrument 2 (sub guitar) in the MIDI data of the karaoke piece to be played. Processing for conversion to reference data is performed (S104). When the musical instrument 3 (bass guitar) is determined as the musical instrument to be used for performance, the control unit 11 evaluates the MIDI channel note information corresponding to the musical instrument 3 (bass) among the MIDI data of the karaoke piece to be played. Processing for conversion to data is performed (S106). When the musical instrument 4 (drum) is determined as the musical instrument to be used for performance, the control unit 11 uses the MIDI channel note information corresponding to the musical instrument 4 (drum) as the evaluation reference data among the MIDI data of the karaoke piece to be played. (S108). A detailed procedure for converting MIDI data into evaluation reference data will be described later.

  The subsequent processes of S110 to S116 are executed in parallel with the processes of S102 to S108, which are continuously executed during the performance of karaoke music. In S110, the control part 11 determines whether the performance of the karaoke music based on MIDI data was complete | finished. When the performance of the karaoke music has not ended (S110: NO), the control unit 11 proceeds to S112. On the other hand, when the performance of the karaoke music is completed (S110: YES), the control unit 11 proceeds to S118.

  In S <b> 112, which proceeds when the performance of the karaoke music has not ended, the control unit 11 determines whether or not there is an input of an instrument sound played by the instrument 26 from the instrument sound input unit 21. When there is an input of musical instrument sound (S112: YES), the control unit 11 proceeds to S114. On the other hand, when no instrument sound is input (S112: NO), the control unit 11 returns to S110.

  In S <b> 114 that proceeds when there is an input of an instrument sound, the control unit 11 performs an input sound analysis process on the input instrument sound as an operation corresponding to the above-described discrete evaluation unit 77. The detailed procedure of this input sound analysis process will be described later.

  In the next S116, the control unit 11 evaluates the performance of the musical instrument as an operation corresponding to the performance evaluation means 80 based on the analysis result in the input sound analysis process. Here, the chord or single sound of the instrument sound determined by the input sound analysis process and its output timing are compared with the chord or single sound defined in the note information belonging to the evaluation reference MIDI channel and its output timing. Based on the comparison result, the skill of the performance is evaluated, for example, by adding points when a correct performance is performed or by deducting points when an incorrect performance is performed. After S116, the control unit 11 returns to S110.

  On the other hand, in S118 which proceeds when it is determined in S110 that the performance of the karaoke music has been completed, the control unit 11 generates an image related to the evaluation result in S116 as an operation corresponding to the above-described evaluation result display unit 81, and displays it. 25. Here, the correct sound (chord or single note) based on the MIDI data of the music and the chord or single note actually played on the musical instrument 26 by the performer can be compared with the timing or lyrics related to the evaluation result. It is conceivable to display them in association with each other. Moreover, you may display the result of having performed comprehensive evaluation through the performance of one music. After S118, the control unit 11 ends this process.

[Explanation of evaluation standard data conversion process]
The procedure of the evaluation reference data conversion process executed by the control unit 11 of the karaoke apparatus 10 will be described based on the flowchart of FIG. This process is a process executed by the control unit 11 as an operation corresponding to the above-described evaluation reference data conversion unit 76 in S102 to S108 (FIG. 4) of the above-described performance evaluation process. This process is executed in parallel with S110 to S116 executed continuously during the performance of the karaoke music piece in the performance evaluation process described above.

  In S200, the control part 11 determines whether the performance of the karaoke music based on MIDI data was complete | finished. When the performance of the karaoke music has not ended (S200: NO), the control unit 11 proceeds to S202. In S <b> 202, the control unit 11 determines whether or not an instrument sound played by the instrument 26 is input from the instrument sound input unit 21. When there is an input of musical instrument sound (S202: YES), the control unit 11 proceeds to S204. On the other hand, when there is no instrument sound input (S202: NO), the control unit 11 returns to S200.

In S204, which proceeds when there is an input of an instrument sound, the control unit 11 is MIDI data played when the instrument sound is input, and belongs to the evaluation reference MIDI channel corresponding to the instrument 26 in use. A karaoke music analysis process for analyzing note information is executed. In this karaoke music analysis process, the stroke timing, the evaluation timing performance sound corresponding to the timing when the instrument sound is input from the performance sound based on the MIDI channel note information of the evaluation criterion corresponding to the musical instrument 26 used for performance, Determine the stroke time and stable point.

  The detailed procedure of the karaoke music analysis process executed in S204 will be described based on the flowchart of FIG. In S300, the control unit 11 starts to output the performance sound (stroke timing) output based on the note information of the evaluation reference MIDI channel at the timing when the instrument sound is input, that is, when the performance sound is played. The reference timing indicating whether or not it has been determined is determined. Specifically, for the performance sound output based on the note information of the evaluation reference MIDI channel, the control unit 11 outputs the time when the level of the output waveform of the performance sound instantaneously increases. It is determined as the start timing (stroke timing).

  In S302, the control unit 11 determines a performance time (stroke time) corresponding to the performance sound output based on the note information of the evaluation reference MIDI channel at the timing when the instrument sound is input. For example, in the case of chord performance in a plucked string instrument such as a guitar, the time required for unit performance in which a predetermined plurality of strings are struck for one stroke corresponds to the stroke time. As an example of a method for determining a stroke time of a chord performance sound by a plucked string instrument, there is the following method. That is, the control unit 11 detects the occurrence of a plurality of scales in a unit performance by obtaining the above-described chroma vector feature amount corresponding to the note information of the evaluation reference MIDI channel. When the occurrence of a plurality of scales is detected in a unit performance, the control unit 11 determines a stroke time corresponding to the unit performance from the time interval at the time when the two or more scales are generated. Specifically, if the time interval at the time of occurrence of a plurality of scales included in a unit performance is small, it is considered that the stroke is relatively fast. On the other hand, if the time interval is large, it is considered that the stroke is relatively slow. Therefore, the control unit 11 calculates the stroke time as a value proportional to the time interval at the time of occurrence of the scale.

  In S304, the control unit 11 detects a time point when the performance sound output based on the note information of the evaluation reference MIDI channel is stabilized at the timing when the instrument sound is input. It is generally known that even when the performance sound is stable, the sound disappears quietly, and when the sound is stable, a relatively sharp frequency spectrum is exhibited. In this embodiment, it is assumed that information for determining whether or not the frequency spectrum related to the performance sound has reached a predetermined stable state is stored. Then, using this information, the control unit 11 is when the frequency analysis result of the performance sound based on the note information of the evaluation reference MIDI channel is in a predetermined sharp state (each component is clearly separated). Is detected as the time when the performance sound is stabilized.

  Returning to the flowchart of FIG. In step S <b> 206, the control unit 11 inputs evaluation reference data corresponding to the input instrument sound to the discrete evaluation unit 77. The evaluation reference data includes the above-described chroma vector feature amount obtained by frequency analysis based on the note information belonging to the evaluation reference MIDI channel played when the instrument sound is input. In addition, the evaluation reference data includes each feature amount determined in the karaoke song processing of S204. After S206, the control unit 11 returns to S200. When it is determined in S200 that the performance of the karaoke music has been completed (S200: YES), the control unit 11 ends the process.

[Description of input sound analysis processing]
The procedure of the input sound analysis which the control part 11 of the karaoke apparatus 10 performs is demonstrated based on the flowchart of FIG. This process is a process executed by the control unit 11 as an operation corresponding to the above-described discrete evaluation unit 77 in S114 (FIG. 4) of the above-described performance evaluation process.

In S400, the control unit 11 determines whether or not it is the start timing of the standard stroke period for the unit performance to be evaluated as an operation corresponding to the above-described standard stroke period reference means 78. When it is the start timing of the reference stroke period (S400: YES), the control unit 11 proceeds to S402. On the other hand, when it is not the start timing of the reference stroke period (S400: NO), the control unit 11 returns to S400.

  In S <b> 402 that proceeds when the start timing of the reference stroke period is reached, the control unit 11 performs an operation corresponding to the instrument sound evaluation unit 79 described above on the instrument sound input in the reference stroke period corresponding to the unit performance to be evaluated. On the other hand, discrete evaluation at a predetermined time interval is started using the evaluation reference data. Here, preferably, the control unit 11 changes the time interval of the discrete evaluation for the unit performance according to the stroke time of the unit performance determined in S302 (FIG. 6) of the karaoke music analysis process described above. You may comprise. For example, when the stroke time is relatively long, the time interval for discrete evaluation is increased. On the other hand, when the stroke time is relatively short, the time interval for discrete evaluation is shortened.

  In next step S404, the control unit 11 determines whether or not it is the end timing of the standard stroke period for the unit performance to be evaluated as an operation corresponding to the above-described standard stroke period reference means 78. If it is not the end timing of the reference stroke period (S404: NO), the control unit 11 proceeds to S406. In S406, the control unit 11 continues the discrete evaluation at predetermined intervals. After S406, the control unit 11 returns to S404.

  On the other hand, when it is the end timing of the reference stroke period (S404: YES), the control unit 11 proceeds to S408. In S408, the control unit 11 ends the discrete evaluation related to the reference stroke period corresponding to the unit performance to be evaluated. In the next S410, the control unit 11 operates as the operation corresponding to the musical instrument sound evaluation means 79 described above, based on the respective determination results in a plurality of discrete evaluations, and the chord or single sound played during the reference stroke period to be evaluated. Determine. Here, preferably, the control unit 11 performs the discrete in the vicinity of the time when the performance sound is stable, which is determined in S304 (FIG. 6) of the karaoke music analysis process described above, among the plurality of discrete evaluations in S402 and S406. You may comprise so that the determination result by evaluation may be employ | adopted as a chord or a single sound played in the reference | standard stroke | time period of evaluation object.

[effect]
According to the karaoke apparatus 10 of the embodiment, the following effects can be obtained.
Since MIDI data used for performance of karaoke music can be used for evaluation of musical instrument performance, data dedicated for evaluation is not required, and data development resources and storage capacity of the karaoke apparatus 10 can be saved. Even if the types of musical instruments to be evaluated are increased, if the musical instrument is assigned to a MIDI channel, the musical instrument is created using existing MIDI data without adding new data dedicated to evaluation. The performance can be evaluated.

[Modification]
A modification of the above embodiment will be described below.
<Modification 1> In the above-described embodiment, the musical instrument 26 is used by using the evaluation reference data obtained by converting the note information belonging to the MIDI channel as the evaluation reference into the sound feature quantity such as the chroma vector feature quantity among the MIDI data. The example of evaluating the musical instrument sound performed by is explained. The present invention is not limited to this, and note information belonging to a MIDI channel that is an evaluation reference for musical instrument performance in MIDI data is converted into evaluation reference data indicating a sound waveform such as a digital audio signal, for example, and musical instrument sound is evaluated. You may comprise as follows. In this case, it is conceivable to evaluate the instrument sound based on the degree of coincidence between the input instrument sound waveform and the sound waveform indicated by the evaluation reference data.

<Modification 2> When both chords and single notes can be played by a plucked string instrument such as a guitar,
The following evaluation method may be adopted. That is, the karaoke apparatus 10 includes a code database that stores code evaluation reference data for evaluating a chord played by the connected musical instrument 26 in the HDD 19 or the like. The code database may be, for example, a frequency analysis result obtained experimentally in advance from exemplary performance sounds obtained by actually playing various chords for each chord as chord evaluation reference data. However, for relatively simple chords, it is considered that chord evaluation reference data can be created from MIDI data of karaoke music.

  When evaluating the instrument sound, the control unit 11 determines whether the instrument sound input by the instrument 26 is a chord or a single sound. The following method is exemplified as a specific method for determining whether the instrument sound is a chord or a single note. In association with the performance information indicated by the song data of the karaoke song, song data to which a marker indicating a chord or a single-tone part is added in advance is stored in the karaoke database 75 of the HDD 19. The control unit 11 determines whether the input instrument sound is a chord or a single sound according to a marker assigned to the music data to be output. When it is determined that the instrument sound input by the instrument 26 is a chord, the control unit 11 uses the chord evaluation reference data stored in the chord database to evaluate the input instrument sound. In addition, since the evaluation method of the performance sound of the chord using a chord database is well-known, detailed description here is abbreviate | omitted (for example, refer patent document 1). On the other hand, when it is determined that the instrument sound input by the instrument 26 is a single sound, the instrument sound is evaluated using the evaluation reference data converted from the MIDI data, as in the above-described embodiment. In this way, only when the chord is played by the musical instrument 26, the chord performance sound having a complicated pronunciation process is accurately obtained by using the highly accurate chord evaluation reference data prepared for the performance. Can be evaluated. On the other hand, evaluation of a single performance sound can save development resources for evaluation-dedicated data and the storage capacity of the karaoke apparatus 10 by using evaluation reference data converted from existing MIDI data.

  <Modification 3> If the chord and solo are clearly separated as part according to the type of instrument, such as the main guitar (solo) and sub guitar (chord), the following evaluation method may be adopted. Good. That is, the karaoke apparatus 10 includes a code database that stores code evaluation reference data for evaluating a code in the HDD 19 or the like, as in <Modification 2>.

  When the main guitar is determined as the musical instrument 26 used for performance, the control unit 11 evaluates the musical instrument sound using the evaluation reference data converted from the MIDI data, as in the above-described embodiment. On the other hand, when the sub-guitar is determined as the musical instrument 26 used for the performance, the musical instrument sound is evaluated using the chord evaluation reference data stored in the chord database as in the above <Modification 2>. It is assumed that the type of musical instrument corresponding to the chord or solo part is stored in the HDD 19 or the like.

  As described above, when the sub-guitar for playing the chord part is determined as the musical instrument 26 used for the performance, the highly accurate chord evaluation reference data prepared for the performance is used for the performance evaluation. The performance sound of chords can be evaluated accurately. On the other hand, when the main guitar corresponding to the solo part is determined as the musical instrument 26 used for the performance, the evaluation reference data converted from the existing MIDI data is used for the evaluation of the performance. Data development resources and the storage capacity of the karaoke apparatus 10 can be saved.

<Modification 4> In the above-described embodiment, the karaoke apparatus 10 is a single unit and inputs instrument sounds, determines MIDI channels for evaluation criteria, converts MIDI data to evaluation criteria data, and inputs instrument sounds. An example of evaluation was explained. Not limited to this, a part of a series of functions related to performance evaluation of the present invention is configured by an external karaoke server 30 (corresponding to the evaluation server of the present invention) that can communicate with the karaoke apparatus 10 via a network. There may be.

  Specifically, among the components in the functional block diagram shown in FIG. 2, for example, instrument sound analysis means 73, evaluation channel determination means 74, karaoke database 75, evaluation reference data conversion means 76, discrete evaluation means 77, and It can be considered that the karaoke server 30 has a function corresponding to the performance evaluation means 80.

  In that case, the karaoke apparatus 10 includes means for the control unit 11 to transmit information indicating the type of the musical instrument 26 used for performance to the karaoke server 30 via the interface unit 16. Further, the karaoke apparatus 10 includes means for the control unit 11 to transmit an instrument sound input signal input from the musical instrument 26 to the karaoke server 30 via the interface unit 16. Further, the karaoke apparatus 10 includes means for the control unit 11 to acquire the performance sound evaluation result performed in the karaoke server 30 from the karaoke server 30 via the interface unit 16. The control unit 11 of the karaoke apparatus 10 displays the evaluation result of the musical instrument performance on the display 25 based on the evaluation result of the performance sound acquired from the karaoke server 30.

  <Modification 5> or an external karaoke server capable of communicating a part of a series of functions related to performance evaluation of the present invention with the karaoke apparatus 10 via a network in a form different from the above <Modification 4> 30 (corresponding to the evaluation server of the present invention) may be employed.

  Specifically, among the components in the functional block diagram shown in FIG. 2, for example, the instrument sound analysis means 73, karaoke database 75, evaluation reference data conversion means 76, discrete evaluation means 77, and performance evaluation means 80 are equivalent. It is conceivable that the karaoke server 30 has a function to perform the function.

  In this case, the karaoke apparatus 10 is a means for the control unit 11 to transmit information indicating the evaluation reference MIDI channel determined by the function of the evaluation channel determination unit 74 to the karaoke server 30 via the interface unit 16. Is provided. Further, the karaoke apparatus 10 includes means for the control unit 11 to transmit an instrument sound input signal input from the musical instrument 26 to the karaoke server 30 via the interface unit 16. Further, the karaoke apparatus 10 includes means for the control unit 11 to acquire the performance sound evaluation result performed in the karaoke server 30 from the karaoke server 30 via the interface unit 16. The control unit 11 of the karaoke apparatus 10 displays the evaluation result of the musical instrument performance on the display 25 based on the evaluation result of the performance sound acquired from the karaoke server 30.

  In the musical instrument performance evaluation system described in the above <Modification 4> and <Modification 5>, the apparatus on which the instrument sound is input is not limited to a commercial karaoke apparatus, but a personal computer or a high-function mobile phone terminal. A general-purpose information processing apparatus such as a so-called smartphone may be used.

<Modification 6>
In the above-described evaluation reference data conversion process (FIG. 5), the example in which the evaluation reference such as the stroke timing and the stroke time corresponding to the stroke performance of the guitar is determined in the karaoke piece analysis (FIG. 6) in S204. However, the present invention is not limited to this, and indices such as tone color and pronunciation timing according to the characteristics of other musical instruments such as bass and drum may be determined from the music data and used as evaluation reference data corresponding to those musical instruments.

[Correspondence with configuration described in claims]
The correspondence between the configuration of the karaoke system 1 according to the embodiment and the configuration described in the claims is as follows.

  The karaoke database 75 of the HDD 19 corresponds to MIDI storage means. The control unit 11 corresponds to first determination means. The evaluation channel determination unit 74 of the control unit 11 corresponds to a second determination unit. The evaluation reference data conversion unit 76 of the control unit 11 corresponds to the conversion unit. The discrete evaluation unit 77 and the performance evaluation unit 80 of the control unit 11 correspond to the evaluation unit.

  On the other hand, in the above <Modification 3>, the code database of the HDD 19 corresponds to the code storage means, and the control unit 11 corresponds to the determination means. In <Modification 4>, the code database of the HDD 19 corresponds to the code storage means.

In the <Modification 4>, database 31 of the karaoke server 30 corresponds to a MIDI storage means according to claim 6, the control unit of the karaoke server 30, the second determining means according to claim 6, conversion means, and the evaluation Corresponds to means. Moreover, in the said <modification 4>, the control part 11 of the karaoke apparatus 10 is equivalent to the notification means and transmission means of Claim 6 .

In the <Modification 5>, database 31 of the karaoke server 30 corresponds to a MIDI storage means according to claim 7, the control unit of the karaoke server 30, conversion means of claim 7, and correspond to the evaluation unit. Further, in the above <Modification 5>, the control unit 11 of the karaoke apparatus 10 corresponds to the notification unit and the transmission unit of claim 7 .

  DESCRIPTION OF SYMBOLS 1 ... Karaoke system, 10 ... Karaoke apparatus, 11 ... Control part, 12 ... Bus line, 13 ... Audio | voice control part, 14 ... Operation processing part, 14a ... Operation part, 16 ... Interface part, 17 ... Video RAM, 18 ... Image | video Playback unit, 19 ... Hard disk drive (HDD), 20 ... Microphone, 21 ... Instrument sound input unit, 22 ... MIDI sound source, 23 ... Speaker, 24 ... Video control unit, 25 ... Display, 26 ... Instrument, 30 ... Karaoke server, 31 ... Database, 40 ... LAN in the karaoke box, 50 ... Router, 60 ... Internet, 71 ... Amplifier, 72 ... A / D converter, 73 ... Instrument sound analysis means, 74 ... Channel determination means for evaluation, 75 ... Karaoke database, 76: Evaluation reference data conversion means, 77 ... Discrete evaluation means, 78 ... Reference stroke period reference means, 7 ... instrument sound evaluation means, 80 ... performance evaluation means, 81 ... evaluation result display means.

Claims (7)

MIDI storage means for storing MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments ;
First determining means for determining a musical instrument connected to the musical instrument performance evaluation apparatus;
Second determining means for determining a MIDI channel in the MIDI data of the karaoke song corresponding to the instrument determined by the first determining means;
In accordance with the note information belonging to the MIDI channel determined by the second determination means in the MIDI data stored in the MIDI storage means, the note information includes the stroke timing, the stroke time, and the stable time point of the musical performance sound. Conversion means for converting into evaluation standard data;
The performance of the instrument including stroke timing, stroke time, and stable time point based on the input signal of the instrument sound input from the instrument connected to the instrument performance evaluation apparatus and the evaluation reference data converted by the conversion means An evaluation means for evaluating
A musical instrument performance evaluation apparatus comprising: In the musical instrument performance evaluation apparatus according to claim 1,
The conversion means converts the note information into data indicating a sound feature amount as the evaluation reference data. In the musical instrument performance evaluation apparatus according to claim 1,
The converting means converts the note information into data indicating a sound waveform as the evaluation reference data. In the musical instrument performance evaluation apparatus according to any one of claims 1 to 3,
Code storage means for storing chord evaluation reference data for evaluating chords played by a musical instrument connected to the musical instrument performance evaluation device;
A determination means for determining whether the instrument sound input by the instrument connected to the instrument performance evaluation device is a chord or a single sound;
When the determination means determines that the chord is a chord, the evaluation means includes an input signal of an instrument sound input from an instrument connected to the instrument performance evaluation device, and a chord evaluation stored in the chord storage means Based on the reference data, evaluate the performance of the instrument,
When the determination means determines that the sound is a single sound, the evaluation means includes an instrument sound input signal input from an instrument connected to the instrument performance evaluation device, and evaluation reference data converted by the conversion means; A musical instrument performance evaluation apparatus characterized by performing evaluation of musical instrument performance based on the above. In the musical instrument performance evaluation apparatus according to any one of claims 1 to 3,
Code storage means for storing code evaluation reference data for evaluating a chord played by a musical instrument connected to the musical instrument performance evaluation apparatus;
The first determining means has a function of determining a main guitar and a sub guitar as musical instruments connected to the musical instrument performance evaluation device,
When the main guitar is determined by the first determining means, the evaluating means includes an instrument sound input signal input from a musical instrument connected to the instrument performance evaluation device, and evaluation reference data converted by the converting means. On the basis of the performance of the instrument,
When the sub-guitar is determined by the first determination means, the evaluation means includes an input signal of an instrument sound input from an instrument connected to the instrument performance evaluation device, and a chord evaluation stored in the chord storage means. A musical instrument performance evaluation apparatus characterized by evaluating performance of a musical instrument based on reference data. An instrument performance evaluation system comprising an information processing device and an evaluation server, wherein the information processing device and the evaluation server communicate with each other,
The information processing apparatus includes:
First determining means for determining a musical instrument connected to the information processing apparatus;
Notification means for notifying the evaluation server of the musical instrument determined by the first determination means;
Transmission means for transmitting an instrument sound input signal input using the information processing apparatus to the evaluation server,
The evaluation server
MIDI storage means for storing MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments;
Second determining means for determining a MIDI channel in the MIDI data of the karaoke song corresponding to the musical instrument notified from the information processing apparatus;
In accordance with the note information belonging to the MIDI channel determined by the second determination means in the MIDI data stored in the MIDI storage means, the note information includes the stroke timing, the stroke time, and the stable time point of the musical performance sound. Conversion means for converting into evaluation standard data;
Evaluation that evaluates the performance of the instrument including the stroke timing, the stroke time, and the stable time based on the input signal of the instrument sound transmitted from the information processing apparatus and the evaluation reference data converted by the conversion means Means,
A musical instrument performance evaluation system. An instrument performance evaluation system comprising an information processing device and an evaluation server, wherein the information processing device and the evaluation server communicate with each other,
The information processing apparatus includes:
First determining means for determining a musical instrument connected to the musical instrument performance evaluation apparatus;
Second determining means for determining a MIDI channel in the MIDI data of the karaoke song corresponding to the instrument determined by the first determining means;
Notification means for notifying the evaluation server of the MIDI channel determined by the second determination means;
Transmission means for transmitting an instrument sound input signal input using the information processing apparatus to the evaluation server,
The evaluation server
MIDI storage means for storing MIDI data indicating a plurality of MIDI channels corresponding to a plurality of musical instruments used for performance as performance information of karaoke songs performed by a plurality of musical instruments;
In accordance with the note information belonging to the MIDI channel notified from the information processing device in the MIDI data stored in the MIDI storage means, the note information is evaluated including the stroke timing, stroke time, and stable time point of the musical performance sound Conversion means for converting into reference data;
Evaluation that evaluates the performance of the instrument including the stroke timing, the stroke time, and the stable time based on the input signal of the instrument sound transmitted from the information processing apparatus and the evaluation reference data converted by the conversion means Means,
A musical instrument performance evaluation system.

Images