JP6288197B2 - Evaluation apparatus and program - Google Patents

Description

  The present invention relates to an evaluation apparatus and a program.

  Some karaoke apparatuses have a function of scoring the skill of singing. For example, Patent Document 1 proposes an apparatus for scoring a singing voice signal of a karaoke song by weighting each section such as rust and climax. Further, Patent Document 2 proposes an apparatus for scoring using scoring standard data in which scoring is determined in advance for a pitch that is considered to be correct in terms of music theory, such as a pitch corresponding to a chord (chord) with respect to the main melody pitch. Has been. According to the apparatus described in Patent Literature 2, since the score is high when a singer sings a pitch that is correct in terms of music theory, the user's satisfaction with the scoring results can be improved.

JP-A-10-78749 JP 2010-230854 A

By the way, in the scoring of the song as described in Patent Document 1, there is a difference between the skill of the song felt by the person listening to the song (hereinafter referred to as “listener”) and the scoring result of the apparatus. There was a case. For example, there are cases where the grading result of the device is low even for a song that the listener feels good, and conversely, the grading result of the device is high even for a song that the listener does not feel good. Moreover, although the technique described in Patent Document 2 can improve the user's satisfaction with the scoring results, it is necessary to prepare scoring reference data consisting of a score distribution for each piece of music in advance, which is complicated. There was a case. The technique described in Patent Document 1 and Patent Document 2 is the same as described above even when applied to the performance evaluation of musical instruments.
The present invention has been made in view of the above-described background, and in an apparatus for evaluating the skill of singing and playing, the evaluation result by the apparatus and the evaluation felt by the listener without preparing scoring reference data for each piece of music in advance. The purpose is to reduce the gap between the results.

In order to solve the above-described problems, the present invention provides an evaluation data acquisition unit that acquires evaluation data representing a sound that serves as a model of music, and accompaniment sound data that acquires accompaniment sound data representing the accompaniment sound of the music Sound that identifies the characteristics of the sound data acquired by the acquisition unit, the sound data acquisition unit that acquires the singing voice sung to the accompaniment sound or the sound data that represents the played performance sound, and the sound data acquisition unit Acquired by the data feature specifying unit, the accompaniment data feature specifying unit for specifying the feature of the accompaniment sound data acquired by the accompaniment sound data acquiring unit, the feature specified by the sound data feature specifying unit, and the evaluation data acquiring unit An evaluation unit that evaluates the sound data in an evaluation mode according to the feature specified by the accompaniment data feature specifying unit when evaluating the sound data based on the evaluation data The accompaniment data feature specifying unit, the number of notes contained within a predetermined time in the obtained accompaniment sound accompaniment sound indicated by the data, a phrase of this該伴locations key changes in Soon, and the accompaniment tone There is provided an evaluation apparatus characterized by specifying at least one of structures as a characteristic of the accompaniment sound.

In a preferred aspect of the present invention, the evaluation unit includes an item-based evaluation unit that evaluates the sound data for a plurality of evaluation items based on the feature specified by the sound data feature specifying unit, and the accompaniment data feature specifying unit. A coefficient specifying unit that specifies the value of the weighting coefficient for each evaluation item according to the specified feature, and the evaluation result for each evaluation item by the item-based evaluation unit, the specification specified by the coefficient specifying unit It performs weighting using the weighting coefficient for each evaluation item, may include a sound data evaluation unit for evaluating the sound data according to the result of the weighting.

  Further, in a further preferred aspect of the present invention, the item-by-item evaluation unit evaluates the sound data for a plurality of evaluation items including a pitch based on the features specified by the sound data feature specifying unit, and the accompaniment data The feature specifying unit specifies the number of notes included in a predetermined time in the accompaniment sound indicated by the accompaniment sound data as a feature of the accompaniment sound data, and the coefficient specifying unit is determined by the accompaniment data feature specifying unit. The value of the weighting coefficient for one or more evaluation items including the pitch may be increased according to the specified number of notes.

  Further, in another preferable aspect of the present invention, the item-by-item evaluation unit evaluates the sound data for a plurality of evaluation items including a pitch based on the feature specified by the sound data feature specifying unit, and the accompaniment The data feature specifying unit analyzes the accompaniment sound data and specifies the chord progression of the accompaniment sound, and the coefficient specifying unit is a pattern in which the chord progression specified by the accompaniment data feature specifying unit is predetermined. In this case, the value of the weighting coefficient for one or a plurality of evaluation items including the pitch may be increased.

  Moreover, in another preferable aspect of the present invention, the item-based evaluation unit evaluates the sound data for a plurality of evaluation items including a rhythm based on the features specified by the sound data feature specifying unit, and the accompaniment The data feature specifying unit specifies, as the feature of the accompaniment sound data, a portion where the rhythm represented by the accompaniment sound data differs from the rhythm represented by the evaluation data, and the coefficient specifying unit is configured to include the accompaniment data feature. In the location specified by the specifying unit, the value of the weighting coefficient for one or a plurality of evaluation items including the rhythm may be increased.

Further, the present invention provides a computer singing in accordance with the accompaniment sound, the step of acquiring evaluation data representing a sound that serves as an example of the music, the step of acquiring accompaniment sound data representing the accompaniment sound of the music Obtaining sound data representing the singing voice or played performance sound, identifying the characteristic of the acquired sound data, analyzing the acquired accompaniment sound data, and characteristics of the accompaniment sound data the method comprising the steps of identifying phrases in the number of notes contained within a predetermined time in the acquired accompaniment sound accompaniment sound indicated by the data, place the key in person該伴Soon changes, and the accompaniment tone Identifying at least one of the structures as a characteristic of the accompaniment sound, based on the characteristic of the identified sound data and the acquired evaluation data When evaluating Kion data, provides a program for executing a step of evaluating the sound data, an evaluation mode according to the identified accompaniment tone characteristics of the data.

  According to the present invention, in a device for evaluating the skill of singing or playing, the difference between the evaluation result by the device and the evaluation result felt by the listener is reduced without preparing scoring reference data in advance for each song. be able to.

System configuration diagram of an embodiment of the present invention Block diagram showing hardware configuration of karaoke equipment Schematic diagram showing the contents of the accompaniment sound data storage area Block diagram showing an example of the functional configuration of a karaoke device Block diagram showing an example of the functional configuration of the scoring unit The figure which shows an example of the content of the scoring process of the score part 119 Flow chart showing the processing flow of this embodiment Flow chart showing the flow of scoring process The figure which shows an example of the content of the scoring process of the score part 119

<Embodiment>
<Configuration>
FIG. 1 is a diagram showing the configuration of a system according to an embodiment of the present invention. This system includes a karaoke device 100, a server device 200, and a network NW. The karaoke device 100 is a device that reproduces karaoke music according to a request from a singer and evaluates singing by the singer about the karaoke music to be played. The network NW is a LAN (Local Area Network) or the Internet, and is a communication network in which data communication is performed between the karaoke apparatus 100 and the server apparatus 200. The server device 200 stores various data such as content data related to karaoke music in a storage means such as an HDD (Hard Disk Drive) provided inside or outside the server device 200, and via the network NW according to a request from the karaoke device 100 The content data is supplied to the karaoke apparatus 100. Here, the content refers to a combination of audio and video related to karaoke music. That is, the content data is composed of so-called accompaniment sound data composed of accompaniment and chorus without the main melody singing voice, and video data composed of the lyrics displayed on the background of the music and the lyrics. Note that a plurality of karaoke apparatuses 100 may exist for the server apparatus 200. Further, a plurality of server devices 200 may exist for the karaoke device 100.

  FIG. 2 is a block diagram showing the hardware configuration of the karaoke apparatus 100. Karaoke apparatus 100 is an example of an evaluation apparatus according to the present invention. The karaoke apparatus 100 includes a control unit 10, a storage unit 20, an operation unit 30, a display unit 40, a communication control unit 50, an audio processing unit 60, a microphone 61, and a speaker 62, and these units are connected via a bus 70. Has been. The control unit 10 includes a processor such as a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. In the control unit 10, a processor such as a CPU reads out a computer program stored in the ROM or the storage unit 20, loads it into the RAM, and executes it to control each unit of the karaoke apparatus 100.

  The operation unit 30 includes various operators and outputs an operation signal representing the operation content of the singer to the control unit 10. The display unit 40 includes, for example, a liquid crystal panel, and displays various images such as lyrics telop and background video according to each karaoke piece under the control of the control unit 10. The communication control unit 50 connects the karaoke device 100 and the network NW by wire or wirelessly, and controls data communication between the karaoke device 100 and the server device 200 via the network NW.

  The server device 200 is a computer including a processor and various memories (not shown), and particularly includes a network storage 210. The network storage 210 is an HDD, for example, and stores various data such as content data of karaoke music. In FIG. 2, the server device 200 includes one network storage 210, but the number of network storages is not limited to this, and the server device 200 may include a plurality of network storages. When content data of karaoke music reserved by a singer is stored in the network storage 210, the karaoke device 100 communicates with the server device 200 in accordance with control by the communication control unit 50 and is read from the network storage 210. While downloading the content data via the network NW, streaming playback is performed in which playback is performed sequentially from the part where the download is completed.

  The microphone 61 outputs an analog audio signal representing the collected audio to the audio processing unit 60. The audio processing unit 60 includes an A / D (Analog / Digital) converter, converts the analog audio signal output from the microphone 61 into digital audio data, and outputs the digital audio data to the control unit 10. get. The audio processing unit 60 includes a D / A (Digital / Analog) converter, converts digital audio data received from the control unit 10 into an analog audio signal, and outputs the analog audio signal to the speaker 62. The speaker 62 emits a sound based on the analog audio signal received from the audio processing unit 60. In this embodiment, a case where the microphone 61 and the speaker 62 are included in the karaoke apparatus 100 will be described. However, the audio processing unit 60 is provided with an input terminal and an output terminal, and the input terminal is connected to the input terminal via an audio cable. An external microphone may be connected, and similarly, an external speaker may be connected to the output terminal via an audio cable. In this embodiment, the audio signal output from the microphone 61 to the speaker 62 is an analog audio signal. However, digital audio data may be input / output. In such a case, the audio processing unit 60 does not need to perform A / D conversion or D / A conversion. The same applies to the operation unit 30 and the display unit 40, and an external input terminal and an external output terminal may be provided to connect a keyboard and an external monitor.

  The storage unit 20 is a storage unit for storing various data, and is, for example, an HDD or a nonvolatile memory. The storage unit 20 includes a plurality of storage areas such as an accompaniment sound data storage area 21, a video data storage area 22, a GM (Guide Melody) data storage area 23, and a singing voice data storage area 24.

  FIG. 3 is a schematic diagram showing the contents of the accompaniment sound data storage area 21. The accompaniment sound data storage area 21 stores information related to accompaniment sound data representing the accompaniment sound of each music piece. The accompaniment sound data storage area 21 stores a plurality of accompaniment sound data records including a plurality of items such as “song number”, “song name”, “singer name”, “genre”, and “file storage location”. “Song number” is a number for uniquely identifying a music piece, and is composed of, for example, a 4-digit parent number and a 2-digit branch number. “Song name” represents the name of each music piece. “Singer name” represents the name of each singer. “Genre” represents the genre of music to which each musical piece belongs among a plurality of genres classified according to a predetermined classification standard. The “file storage location” represents a storage location of a data file that is accompaniment sound data itself of each musical piece. When a folder “server1” is included, a data file of accompaniment sound data is stored in the server device 200 and is called server1. When the folder is not included, it means that the accompaniment sound data file is stored in the karaoke apparatus 100. For example, in FIG. 3, a song whose song name is “BBB” indicates that the accompaniment sound data data file is stored in the server device 200, and a song whose song name is “CCC” is an accompaniment sound data file. Is stored in the storage unit 20 of the karaoke apparatus 100. The accompaniment sound data file is, for example, a MIDI (Musical Instrument Digital Interface) format file. In this embodiment, the accompaniment sound data is preliminarily provided with key information indicating the key of the accompaniment sound and tempo information indicating the tempo of the accompaniment sound.

  The video data storage area 22 stores lyrics data indicating the lyrics of each song and background video data representing a background video displayed on the background of the lyrics. The lyrics shown by the lyrics data are displayed on the display unit 40 as lyrics telop as the music progresses during karaoke singing. In addition, the background video represented by the background video data is displayed on the display unit 40 as the background of the lyrics telop as the music progresses during karaoke singing. In the GM data storage area 23, data indicating the melody of the vocal part of the music, that is, guide melody data (hereinafter referred to as “GM data”), which is data specifying the content of the constituent sound to be sung, is stored. The GM data serves as a reference for comparison when the control unit 10 evaluates the skill of singing by a singer, and is a reference set in advance for a song to be sung. Since the evaluation process performed by the control unit 10 will be described later, a detailed description thereof will be omitted here. The GM data is described in the MIDI format, for example.

  In the singing voice data storage area 24, the singing voice of the singer collected by the microphone 61 during the period in which the accompaniment sound data is being reproduced for each piece of music that is the subject of karaoke is converted into digital data by the voice processing unit 60. The voice data generated by the conversion to is stored. This voice data is called singing voice data. That is, the singing voice data is sound data representing singing voice that has been sung in accordance with the accompaniment sound represented by the accompaniment sound data. The singing voice data is voice data representing a voice waveform, and is stored as a data file in a WAVE (RIFF waveform Audio Format) format, for example. The singing voice data for each song is associated with the GM data of the song by the control unit 10.

  FIG. 4 is a block diagram illustrating an example of a functional configuration of the karaoke apparatus 100. 4, the accompaniment playback unit 11 and the scoring unit 12 are realized by a processor such as a CPU of the control unit 10 reading out a computer program stored in the ROM or the storage unit 20 and loading it into the RAM for execution. . The accompaniment playback unit 11 plays back karaoke music. Specifically, the accompaniment reproducing unit 11 causes the speaker 62 to emit sound based on the accompaniment sound data and the GM data, and causes the display unit 40 to display a video based on the video data. The scoring unit 12 specifies the pitch of the singing voice data, and evaluates the singing voice of the singer based on the specified pitch.

  FIG. 5 is a block diagram illustrating an example of a functional configuration of the scoring unit 12. In this embodiment, the scoring unit 12 specifies the characteristics of the accompaniment sound data, and performs an evaluation in consideration of the specified characteristics. In FIG. 5, the pitch acquisition part 111 acquires the singing voice data memorize | stored in the singing voice data storage area 24, and specifies the pitch of the acquired singing voice data. The pitch acquisition unit 111 outputs data representing the specified pitch (hereinafter referred to as “pitch data”).

  The volume acquisition unit 112 acquires the singing voice data stored in the singing voice data storage area 24, and specifies the volume of the acquired singing voice data. The volume acquisition unit 112 outputs data representing the specified volume (hereinafter referred to as “volume data”).

  The voice quality acquisition unit 113 acquires the singing voice data stored in the singing voice data storage area 24, and specifies the voice quality of the acquired singing voice data. The voice quality specifying process is performed as follows, for example. First, the voice quality acquisition unit 113 performs processing such as FFT (Fast Fourier Transform) on the singing voice data to calculate the level (volume) (hereinafter referred to as “high frequency level”) of the high frequency band. The ratio of the high frequency level to the volume is specified as the voice quality. In this embodiment, the voice quality acquisition unit 113 calculates a frequency band level of 1 kHz or higher as a high frequency level. In addition, the specific aspect of voice quality is not limited to the process mentioned above, What kind of thing may be used if voice quality is specified. The pitch acquisition unit 111, the volume acquisition unit 112, and the voice quality acquisition unit 113 are an example of a sound data acquisition unit and a sound data feature identification unit according to the present invention.

  The pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118 are the pitch acquired by the pitch acquisition unit 111, the volume acquired by the volume acquisition unit 112, and the voice quality, respectively. The singing voice is evaluated using at least one of the characteristics of the voice quality acquired by the acquisition unit, and a score indicating the evaluation result is calculated. The pitch score calculation unit 114 evaluates the pitch according to the pitch of the singing voice acquired by the pitch acquisition unit 111. More specifically, the pitch score calculation unit 114 acquires pitch data representing the pitch specified by the pitch acquisition unit 111 and GM data corresponding to this singing voice. The control unit 10 associates the pitch data with the GM data in the time axis direction, compares the pitch indicated by the pitch data with the pitch of the GM data according to the association result, and scores the pitch based on the difference between the two. Calculate More specifically, for example, the pitch score calculation unit 114 compares the change in pitch of the singing voice data with the change in GM data, and calculates an evaluation value indicating the degree of coincidence thereof. The evaluation value is 100% (that is, there is no deduction) if the difference between the two pitches is within a predetermined allowable range in a certain note, and the period during which the difference between the two pitches does not fall within the above range. However, in the case of GM data, it may be 50% if the note length is half of the note length in this note. That is, in a certain note, a value obtained by dividing a period in which the difference in pitch between the two is within the above range by the sound length in this note in GM data is used as an evaluation value.

  The singing technique calculation unit 115 evaluates the singing voice data for each singing technique based on the pitch of the singing voice acquired by the pitch acquisition unit 111 and the volume of the singing voice acquired by the volume acquisition unit 112. In this embodiment, the singing technique calculation unit 115 specifies the singing technique by changing the pitch of the singing voice data, and evaluates the specified singing technique for each note. In this embodiment, the singing technique calculation unit 115 evaluates each of the singing techniques “vibrato”, “shakuri”, “fist”, and “fall”. For “vibrato”, the singing technique calculation unit 115 analyzes the pattern of temporal change in pitch, and identifies a section where the pitch continuously fluctuates within a predetermined range above and below the central pitch. A note including the specified section is a note in which “vibrato” is used. For the “vibrato” singing technique, the singing technique calculation unit 115 evaluates and scores the uniformity of pitch fluctuation, the depth of pitch fluctuation, the duration, the uniformity of volume, and the like.

  As for “shakuri”, the singing technique calculation unit 115 analyzes the pattern of the temporal change in pitch, and identifies and specifies the section where the pitch continuously changes from the low pitch to the target pitch at the beginning of the phrase. A note including a section is assumed to be a note in which the singing technique of “shakuri” is used. As for the “shakuri” singing technique, the singing technique calculation unit 115 evaluates and scores the fluctuation range and the fluctuation time of the pitch.

  For “Fall”, the singing technique calculation unit 115 analyzes the pattern of the temporal change in pitch, and calculates a section in which the pitch continuously changes from a long tone that extends long at a constant pitch to a low pitch at the end of the phrase. A note including the specified section is defined as a note in which the “fall” singing technique is used. For the “fall” singing technique, the singing technique calculation unit 115 evaluates and scores the fluctuation range and the fluctuation time of the pitch.

  For “fist”, the singing technique calculation unit 115 analyzes a pattern of temporal change in pitch, identifies a section where the pitch goes up and down in a short time in the phrase, and “fists” a note including the identified section. Suppose that is used. For the “fist” singing technique, the singing technique calculation unit 115 evaluates and scores the pitch fluctuation range and the fluctuation time.

  The inflection calculation unit 116 evaluates the skill of inflection for each note based on the volume acquired by the volume acquisition unit 112. As an aspect of the evaluation, for example, a volume reference value is set for each note included in the GM, and the intonation calculation unit 116 compares the volume reference value with the volume specified from the singing voice data, The skill of inflection may be evaluated according to the difference between the two. Further, for example, a volume reference value is set for each predetermined section such as each measure, and the intonation calculation unit 116 determines the volume specified from the set volume reference value and the singing voice data for each section. And may be evaluated according to the comparison result.

  The rhythm calculation unit 117 evaluates the rhythm of the singing voice for each note based on the pitch of the singing voice acquired by the pitch acquisition unit 111 and the volume of the singing voice acquired by the volume acquisition unit 112. The rhythm calculation unit 117 identifies the sound generation timing of the sound corresponding to each note included in the GM from the pitch and volume of the singing voice, compares the specified sound generation timing with the sound generation timing of the note included in the GM, Evaluation is performed according to the difference between the two.

  The voice quality calculation unit 118 evaluates the voice quality of the singing voice for each note based on the voice quality acquired by the voice quality acquisition unit 113. In this embodiment, the voice quality calculation unit 118 evaluates the voice quality according to the voice quality acquired by the voice quality acquisition unit 113 (the ratio of the high frequency level to the volume of the entire band). At this time, the voice quality calculation unit 118 calculates the evaluation value of the voice quality so that the higher the ratio of the high frequency level to the volume of the entire band is, the higher the evaluation is. In addition, the aspect of evaluation of voice quality is not limited to this, You may evaluate voice quality using other well-known techniques.

  The score unit 119 evaluates the singing voice using the evaluation values calculated by the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118. At this time, the score unit 119 obtains a code for the evaluation value for each note calculated by the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118. Weighting is performed using the weighting coefficients notified from the unit 121, the key change acquisition unit 123, the simultaneous pronunciation number acquisition unit 124, the accompaniment rhythm acquisition unit 125, and the tempo change acquisition unit 127, and the score of the singing voice is calculated. This weighting process will be described later. The pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, the voice quality calculation unit 118, and the score unit 119 are examples of the evaluation unit according to the present invention. The score unit 119 outputs the calculated result to the display unit 40 and causes the display unit 40 to display the calculation result.

  The accompaniment feature extraction unit 120 acquires the song number of the song selected by the singer from the operation unit 30, acquires the accompaniment sound data corresponding to the acquired song number from the accompaniment sound data storage area 21, and acquires the accompaniment sound. Identify the characteristics of the data. The accompaniment feature extraction unit 120 is an example of an accompaniment sound data acquisition unit and an accompaniment data feature identification unit according to the present invention. The accompaniment feature extraction unit 120 includes a chord acquisition unit 121, a key acquisition unit 122, a key change acquisition unit 123, a simultaneous pronunciation number acquisition unit 124, an accompaniment rhythm acquisition unit 125, a tempo acquisition unit 126, and a tempo change acquisition. Part 127.

  The chord acquisition unit 121 identifies a chord (chord progression) for each unit time of the accompaniment sound data by analyzing a pitch pattern included in the unit time (for example, one measure) of the accompaniment sound data. The specific mode of chord progression of the accompaniment sound data is not limited to this. For example, data representing chord progression (chord progression data) is preliminarily added to the accompaniment sound data, and the chord acquisition unit 121 includes the accompaniment sound data. The chord for each unit time may be specified by reading the chord progression data given to the sound data. Moreover, the chord acquisition part 121 may specify a chord using well-known techniques, such as a chord progress detection from an audio signal shown by Unexamined-Japanese-Patent No. 2004-184510, and data conversion technique, for example.

  In this embodiment, the chord acquisition unit 121 identifies and specifies weighting coefficient values for a plurality of evaluation items such as “pitch”, “rhythm”, “intonation”, and “technique” according to the identified chord. The obtained weighting coefficient is notified to the score unit 119. More specifically, in this embodiment, when the chord progression is a predetermined pattern (for example, V → I), the chord acquisition unit 121 selects one or a plurality of evaluation items including “pitch”. Increase the value of the corresponding weighting factor. For example, the melody stops at places where the chord progression changes from V to I, and the pitch is regarded as important. Therefore, when such a place is sung well, the listener often feels that the singing is good. Therefore, the evaluation of the apparatus can be made closer to the evaluation of the listener by increasing the weighting for the evaluation item of “pitch” at the portion of the pattern where the chord progression is determined in advance. On the other hand, if the chord progression is not a predetermined pattern, the chord acquisition unit 121 specifies a predetermined value for each evaluation item as the value of the weighting coefficient.

  The key acquisition unit 122 specifies the key of the accompaniment sound data by acquiring key information given to the accompaniment sound data. In addition, the specific aspect of the key of accompaniment sound data is not restricted to this. For example, the key acquisition unit 122 may identify the key of the accompaniment sound data by analyzing the pitch pattern included in the accompaniment sound data. The key acquisition unit 122 outputs information indicating the acquired key to the key change acquisition unit 123.

  The key change acquisition unit 123 specifies a location where the key changes in the accompaniment sound represented by the accompaniment sound data according to the information output from the key acquisition unit 122. The key change acquisition unit 123 specifies weighting coefficient values for a plurality of evaluation items such as “pitch”, “rhythm”, “intonation”, and “technique” according to the specifying result, and obtains the specified weighting coefficient score unit 119. Notify At this time, the key change acquisition unit 123 increases the value of the weighting coefficient corresponding to one or a plurality of evaluation items including the “pitch” after the key is changed. Since it is difficult to obtain a pitch at a location where the key changes, it is possible to make the evaluation of the device closer to the listener's evaluation by increasing the weighting for the evaluation item of “pitch” at the location where the key changes. On the other hand, the key change acquisition unit 123 specifies a predetermined value for each evaluation item as the value of the weighting coefficient at a portion where the key is not changed.

  The simultaneous sound number acquisition unit 124 specifies the number of notes included in a predetermined unit time (for example, 0.1 s, one measure, etc.) in the accompaniment sound represented by the accompaniment sound data as a feature of the accompaniment sound data. . The simultaneous sound number acquisition unit 124 specifies weighting coefficient values for a plurality of evaluation items according to the specifying result, and notifies the specified weighting coefficient to the score unit 119. At this time, the simultaneous sound number acquisition unit 124 adjusts the value of the weighting coefficient for one or a plurality of evaluation items including “pitch” according to the specified number of notes (number of notes sounded within a certain time). To do. In this embodiment, the simultaneous pronunciation number acquisition unit 124 increases the value of the weighting coefficient for the evaluation item of “pitch” as the number of specified notes is smaller. In the part where the number of simultaneous sounds of accompaniment is small, the state becomes close to a cappella and the singing is conspicuous. Therefore, by increasing the weight of the “pitch” evaluation, the evaluation of the apparatus can be made closer to the evaluation of the listener. On the other hand, since the singing is not conspicuous in the portion where the number of simultaneous pronunciations is large, it is not necessary to increase the weight of the “pitch” evaluation in such a portion.

  In addition, the simultaneous sound number acquisition unit 124 specifies a change in the simultaneous sound number of the accompaniment (the number of notes included in the unit time) within a predetermined unit time in the accompaniment sound represented by the accompaniment sound data, and the specified result According to the above, the value of the weighting coefficient for a plurality of evaluation items may be specified. At this time, if the variation in the number of simultaneous pronunciations (number of notes included in the unit time) is small, the simultaneous sound number acquisition unit 124 determines that the composition of the music in the unit time is gentle (ballad portion). Since the singing is conspicuous even in such a place, the weight of the evaluation item of “pitch” may be increased. By doing in this way, evaluation of a device can be brought close to evaluation of a listener.

  The accompaniment rhythm acquisition unit 125 analyzes the accompaniment sound data, identifies the rhythm represented by the accompaniment sound data, and identifies the value of the weighting coefficient for each evaluation item according to the identification result. More specifically, in this embodiment, the accompaniment rhythm acquisition unit 125 specifies the rhythm represented by the GM data, and a location where the rhythm of the accompaniment sound data and the rhythm of the GM data are shifted (for example, syncopation, In triplets, etc., the value of the weighting coefficient for one or more evaluation items including “rhythm” is increased. If the rhythm of the accompaniment and the singing rhythm are out of sync (such as syncopation or triplet), the rhythm is difficult to take. Often feel good. Therefore, the evaluation of the apparatus can be made closer to the evaluation of the listener by increasing the weighting on the evaluation item of “rhythm” at the location where the rhythm of the accompaniment sound data and the rhythm of the GM data are shifted.

  The tempo acquisition unit 126 specifies the tempo of the accompaniment sound data by acquiring tempo information given to the accompaniment sound data. In addition, the specific aspect of the tempo of accompaniment sound data is not restricted to this. For example, the tempo acquisition unit 126 may specify the tempo by analyzing the sound generation timing of each note included in the accompaniment sound data. The tempo acquisition unit 126 outputs information indicating the specified tempo to the tempo change acquisition unit 127.

  The tempo change acquisition unit 127 specifies a location where the tempo changes in the accompaniment sound represented by the accompaniment sound data according to the information output from the tempo acquisition unit 126. The tempo change acquisition unit 127 specifies the weighting coefficient value of each evaluation item so that the value of the weighting coefficient for one or a plurality of evaluation items including “rhythm” is increased at the location where the tempo changes. The weighted coefficient is notified to the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118. In this embodiment, since it becomes difficult to take a rhythm at a location where the tempo changes, the weight of the rhythm where the tempo changes is increased so that the score increases when singing at the tempo at such a location.

  The score unit 119 includes a chord acquisition unit 121 for the evaluation values for each note calculated by the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118. Weighting is performed using the weighting coefficients notified by the key change acquisition unit 123, the simultaneous pronunciation number acquisition unit 124, the accompaniment rhythm acquisition unit 125, and the tempo change acquisition unit 127, and the score of the singing voice is calculated.

  FIG. 6 is a diagram for explaining an example of the content of the score calculation process performed by the score unit 119. In the example shown in FIG. 6, for each note included in the singing voice data, “score” and “weight” are obtained for each of a plurality of evaluation items such as “pitch”, “rhythm”, “intonation”, and “technique”. The score unit 119 is notified. In FIG. 6, “score” is a note unit for each evaluation item notified from each of the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118. Shows the score. “Weight” is the average value of the weighting coefficient for each evaluation item notified from each of the chord acquisition unit 121, the key change acquisition unit 123, the simultaneous pronunciation number acquisition unit 124, the accompaniment rhythm acquisition unit 125, and the tempo change acquisition unit 127. Show. The score unit 119 multiplies these “score” and “weight” for each evaluation item, and obtains an average value thereof to obtain an evaluation value for each note. The scoring unit 119 evaluates the entire singing voice by counting the evaluation values for each note.

<Operation>
FIG. 7 is a flowchart showing the flow of processing performed by the control unit 10. When a song is reserved by a singer via the operation unit 30 (step S100; Yes), the control unit 10 searches for a reserved song from the storage unit 20 (step S102). Specifically, in step S102, the control unit 10 uses the song number or song name of the selected song from each of the accompaniment sound data storage area 21, the video data storage area 22, and the GM data storage area 23 as a key. Data relating to the music is searched, and the search result data is read into the RAM.

  Next, the control unit 10 reproduces karaoke music based on the accompaniment sound data, video data, and GM data stored in the RAM (step S104). Specifically, in step S104, the control unit 10 causes the speaker 62 to emit sound based on the accompaniment sound data and the GM data, and causes the display unit 40 to display a video based on the video data. And the control part 10 memorize | stores the singing voice data in which the singing voice of the singer picked up with the microphone 61 was converted into digital data by the voice processing part 60 in the singing voice data storage area 24 (step). S106). When the reproduction of the karaoke music is finished, the control unit 10 scores the song based on the song voice data and the GM data stored in the song voice data storage area 24 (step S108). And the control part 10 displays a scoring result on the display part 40 (step S110).

  FIG. 8 is a flowchart showing the flow of the scoring process (step S108 in FIG. 7) performed by the control unit 10. First, the control unit 10 performs the processes of the pitch acquisition unit 111, the volume acquisition unit 112, and the voice quality acquisition unit 113 described above. That is, the control unit 10 specifies the characteristics (pitch, volume, voice quality) of the sound indicated by the singing voice data (step S200). Next, the control unit 10 performs the processes of the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118 described above. That is, the control part 10 evaluates singing voice data for every evaluation item using the characteristic specified by step S200 (step S210). Next, the control unit 10 performs the process of the accompaniment feature extraction unit 120 described above. That is, the control unit 10 specifies the characteristics (code, key, key change, number of notes within a predetermined time, rhythm, tempo, tempo change) of the accompaniment sound data (step S220), and the evaluation item based on the specified characteristics The value of each weighting coefficient is specified (step S230). Subsequently, the control part 10 performs the process of the score part 119 mentioned above. That is, the control unit 10 weights the evaluation result for each evaluation item specified in step S210 using the weighting coefficient for each evaluation item specified in step S230, and evaluates the singing voice data according to the weighting result. (Step S240).

  In the example shown in FIG. 8, the control unit 10 specifies the characteristics of the singing voice data (step S200), then evaluates the singing voice data for each evaluation item (step S210), and then the characteristics of the accompaniment sound data. However, the order of the processing performed by the control unit 10 is not limited to this. For example, the control unit 10 may evaluate the singing voice data for each evaluation item after switching the order of the processes of step S210 and step S220 shown in FIG. 8 and specifying the characteristics of the accompaniment sound data. Further, for example, the control unit 10 may be configured to process the processing in step S200 and the processing in step S220 in parallel by a multitask OS or the like. As described above, the order of the processing performed by the control unit 10 is arbitrary, and any configuration may be used as long as the singing voice data is weighted to the evaluation value for each evaluation item according to the characteristics of the accompaniment sound data.

  As described above, in the present embodiment, the singing voice is evaluated using accompaniment features such as chord progression, accompaniment rhythm, and the number of notes in a unit time (number of simultaneous pronunciations). By doing in this way, scoring closer to the feeling heard can be performed. For example, since the singing is conspicuous in a portion where the number of simultaneous pronunciations of accompaniment is small, scoring can be performed such that the scoring result of “pitch” in that portion is emphasized. In the present embodiment, the accompaniment feature extraction unit 120 identifies the characteristics of the accompaniment sound data, so it is not necessary to prepare scoring reference data for scoring in advance for each song, and the accompaniment used in existing karaoke apparatuses. Using the sound data, it is possible to perform scoring close to the listener's feeling.

<Modification>
The above-described embodiment can be modified as follows. In addition, you may implement the following modifications suitably combining. Moreover, you may implement combining the said embodiment and the following modifications.

<Modification 1>
In the above-described embodiment, the control unit 10 specifies the pitch, volume, and sound quality of the sound collected by the microphone 61 as the characteristics of the sound data, but the characteristics of the sound data are not limited to these. For example, the control unit 10 uses the frequency analysis result obtained by using FFT or the like, the fluctuation of the power of a specific harmonic, the ratio of the power of the specific harmonic and the fundamental, the ratio of the power of the harmonic component and the fundamental power, SN The ratio and the like may be specified as the characteristics of the sound data. As another example, for example, the control unit 10 determines the sound characteristics such as loudness (a value obtained by correcting the sound volume according to the frequency characteristics of hearing, also referred to as “A characteristic sound pressure level” or “sound level”). May be specified. As described above, any structure may be used as long as it represents the characteristics of the sound, and any structure may be used as long as the sound is evaluated using the specified characteristics. In the above-described embodiment, the control unit 10 specifies the pitch, volume, and sound quality of the sound as the characteristics of the sound data. For example, the evaluation is performed using only the pitch without specifying the sound volume and sound quality. It is good also as a structure.

<Modification 2>
In the above-described embodiment, the control unit 10 has evaluated the evaluation items such as “pitch”, “rhythm”, “intonation”, and “technique”, but the evaluation items are not limited to those shown in the above-described embodiment. Absent. For example, a technique such as tremolo, and so on may be evaluated. In short, what is necessary is that the control unit 10 evaluates the sound indicated by the singing voice data for each evaluation item. Also good.

<Modification 3>
In the above-described embodiment, the control unit 10 outputs the data indicating the evaluation result by causing the display unit 40 to display the evaluation result. However, the output mode of the data indicating the evaluation result is not limited thereto. For example, the control unit 10 may output data indicating the evaluation result to an externally connected storage device, or, for example, by transmitting the data to a server device connected via a communication network. Data may be output. In the above-described embodiment, the singer is notified by outputting the scoring result to the display unit 40. However, the notification mode is not limited to this, and for example, a voice message, a notification sound, and a paper or electronic printed result. You may alert | report by data and what kind of thing may be sufficient as long as it informs a singer of a scoring result.

<Modification 4>
In the above-mentioned embodiment, although the control part 10 evaluated the song voice of a singer, it may replace with the song voice of a singer, and may evaluate the performance sound of the musical instrument by a player. That is, the sound data according to the present invention may be sound data representing a singing voice sung in accordance with the accompaniment sound of the music, or sound data representing a performance sound performed in accordance with the accompaniment sound. May be. The “speech” referred to in the present embodiment includes various sounds such as a sound generated by a person and a performance sound of a musical instrument.

<Modification 5>
In the above-described embodiment, the pitch score calculation unit 114 responds to the chord progression acquired by the chord acquisition unit 121 as the process of specifying the value of the weighting coefficient for each evaluation item according to the feature extracted from the accompaniment sound data. The case where the value of the weighting coefficient for the “pitch” is increased, the case where the value of the weighting coefficient for the “pitch” is increased at the key change point acquired by the key change acquisition unit 123, and the like have been described. The process of specifying the value of the weighting coefficient for each evaluation item according to the feature extracted from the accompaniment sound data is not limited to that described in the above-described embodiment.

  For example, a progress sound specifying unit 128 (shown by a dotted line in FIG. 5) that compares the pitch of notes included in accompaniment sound data with the pitch of GM data and specifies the elapsed sound based on the comparison result is provided. The sound specifying unit 128 may increase the weighting coefficient of the evaluation item of “pitch” for the specified elapsed sound. In general, since the elapsed sound is difficult to take a pitch, the evaluation result of the apparatus can be made closer to the evaluation of the listener by increasing the weighting coefficient of the “pitch” in the elapsed sound.

  As another example, for example, the accompaniment data is analyzed to identify the accompaniment phrase structure, and from the identified phrase structure, the peak position acquisition unit 129 that estimates the apex (the place where the volume peak should be reached) in the phrase is estimated. The peak position acquisition unit 129 may increase the weighting coefficient value of “intonation” at the estimated peak position. The peak position estimation processing performed by the peak position acquisition unit 129 includes, for example, “(Hitoda Mitsuyo, Tanaka Junji, Katayose Haruhiro,“ Performance Expression Design Support Environment Responding to User Intent ”, Information Processing Society of Japan Research Report (Music Information Science), February 2010) "is used.

  In the above-described embodiment, the scoring unit 12 evaluates the singing voice for each evaluation item, and weights the evaluation result for each evaluation item using a weighting coefficient, thereby evaluating the entire singing voice. However, the aspect of evaluation is not limited to this. For example, the scoring unit 12 may evaluate the singing voice based only on the pitch. In this case, it is not necessary to use a weighting coefficient, and the scoring unit 12 may perform processing such as strict scoring criteria when the chord progression is a predetermined pattern, for example. The scoring unit 12 may be anything as long as it evaluates the singing voice data in the evaluation mode based on the characteristics specified from the accompaniment sound data based on the characteristics specified from the singing voice data. .

  In the above-described embodiment, the key information indicating the key of the accompaniment sound and the tempo information indicating the tempo of the accompaniment sound are pre-assigned to the accompaniment sound data. A configuration to which information or tempo information is given may be used. In this case as well, the accompaniment feature extraction unit 120 may acquire key information and tempo information from GM data and specify a weighting coefficient for each evaluation item according to the acquired key information and tempo information, as in the above-described embodiment. .

<Modification 6>
In the above-described embodiment, the pitch score calculation unit 114, the singing technique calculation unit 115, the intonation calculation unit 116, the rhythm calculation unit 117, and the voice quality calculation unit 118 score the singing voice data for each note, and the scoring result for each note. Although the singing voice data is evaluated by counting, the manner of evaluation of the singing voice data is not limited to this. For example, as shown in FIG. 9, scoring is performed for all notes in each evaluation item, and each scoring result is multiplied by a weighting coefficient, and an average value or total value is calculated for each evaluation item. The evaluation value may be obtained. In this case, the score unit 119 calculates the average value or the total value of the scores for each obtained evaluation item, thereby obtaining the score of the singing voice data. When calculating the total value, the score unit 119 may perform normalization so that the maximum value becomes 100 points.

<Modification 7>
In the above-described embodiment, the control unit 10 stores the singing voice data in the singing voice data storage area 24 and performs the scoring after the singing is finished. However, not limited to this, the scoring process is performed in real time during the singing. May be performed. When performing the scoring process in real time during the singing, the control unit 10 may display the scoring result at any time during the singing of the singer or after the singing ends.

<Modification 8>
In the above-described embodiment, two or more devices connected by a communication network share the functions related to the karaoke device 100 of the above-described embodiment, and a system including the plurality of devices uses the karaoke device 100 of the same embodiment. It may be realized. For example, a computer device including a microphone, a speaker, a display device, an operation unit, and the like may be configured as a system in which a server device that executes scoring processing is connected via a communication network. In this case, for example, the computer apparatus converts the sound collected by the microphone into an audio signal and transmits it to the server apparatus, and the server apparatus analyzes and scores the received audio signal, and the scoring result is calculated by the computer apparatus. May be sent to. According to this aspect, the processing load of the karaoke terminal is reduced, and statistical processing in the server becomes possible.

  In the above-described embodiment, the accompaniment reproduction unit 11 and the scoring unit 12 shown in FIG. 4 are realized by the control unit 10 reading and executing the computer program stored in the ROM or the storage unit 20. The unit 11 and the scoring unit 12 are not limited to being realized as software by a control unit such as a CPU. The karaoke apparatus 100 may have dedicated hardware (circuit) corresponding to each of the functional elements shown in FIG.

<Modification 9>
In addition to the evaluation apparatus, the present invention can be understood as a method for realizing these and a program for causing a computer to realize a voice evaluation function. Such a program may be provided in the form of a recording medium such as an optical disk storing the program, or may be provided in the form of being downloaded to a computer via the Internet or the like and installed and used. According to this aspect, the service according to the above-described embodiment can be provided by a home PC (Personal Computer), a mobile terminal (including a smartphone), a game device, or the like.

DESCRIPTION OF SYMBOLS 10 ... Control part, 20 ... Storage part, 21 ... Accompaniment sound data storage area, 22 ... Image | video data storage area, 23 ... GM data storage area, 24 ... Singing voice data storage area, 30 ... Operation part, 40 ... Display part, DESCRIPTION OF SYMBOLS 50 ... Communication control part, 60 ... Sound processing part, 61 ... Microphone, 62 ... Speaker, 70 ... Bus, 100 ... Karaoke apparatus, 111 ... Pitch acquisition part, 112 ... Volume acquisition part, 113 ... Voice quality acquisition part, 114 ... Pitch Score calculation unit, 115 ... Singing technique calculation unit, 116 ... Intonation calculation unit, 117 ... Rhythm calculation unit, 118 ... Voice quality calculation unit, 119 ... Scoring unit, 120 ... Accompaniment feature extraction unit, 121 ... Chord acquisition unit, 122 ... Key Acquisition unit, 123 ... key change acquisition unit, 124 ... simultaneous pronunciation number acquisition unit, 125 ... accompaniment rhythm acquisition unit, 126 ... tempo acquisition unit, 127 ... tempo change acquisition unit, 128 ... elapsed sound characteristics Department, 200 ... server, 210 ... network storage

Claims (6)

An evaluation data acquisition unit for acquiring evaluation data representing the sound that serves as an example of the music;
An accompaniment sound data acquisition unit for acquiring accompaniment sound data representing the accompaniment sound of the music;
A sound data acquisition unit for acquiring sound data representing a singing voice or a performance sound performed according to the accompaniment sound;
A sound data feature specifying unit for specifying a feature of the sound data acquired by the sound data acquiring unit;
An accompaniment data feature identifying unit that analyzes the accompaniment sound data acquired by the accompaniment sound data acquisition unit and identifies the characteristics of the accompaniment sound data;
When evaluating the sound data based on the feature specified by the sound data feature specifying unit and the evaluation data acquired by the evaluation data acquiring unit, the sound data is specified by the accompaniment data feature specifying unit. An evaluation unit that evaluates in an evaluation mode according to the characteristics
The accompaniment data feature specifying unit, the number of notes contained within a predetermined time in the obtained accompaniment sound accompaniment sound indicated by the data, a phrase of this該伴locations key changes in Soon, and the accompaniment tone An evaluation apparatus characterized by specifying at least one of the structures as a characteristic of the accompaniment sound. The evaluation unit is
An item-by-item evaluation unit that evaluates the sound data for a plurality of evaluation items based on the features specified by the sound data feature specifying unit;
A coefficient specifying unit for specifying a value of a weighting coefficient for each evaluation item according to the feature specified by the accompaniment data feature specifying unit;
The evaluation results for each of the evaluation items by the item-by-item evaluation unit, performs weighting using the weighting factor for each of the evaluation items specified by the coefficient specifying unit, evaluating the sound data according to the result of said weighting The evaluation apparatus according to claim 1, further comprising: a sound data evaluation unit. The item-based evaluation unit evaluates the sound data for a plurality of evaluation items including a pitch based on the features specified by the sound data feature specifying unit,
The accompaniment data feature specifying unit specifies the number of notes included in a predetermined time in the accompaniment sound indicated by the accompaniment sound data as a feature of the accompaniment sound data,
The coefficient specifying unit adjusts the value of a weighting coefficient for one or a plurality of evaluation items including the pitch according to the number of notes specified by the accompaniment data feature specifying unit. The evaluation device described. The item-based evaluation unit evaluates the sound data for a plurality of evaluation items including a pitch based on the features specified by the sound data feature specifying unit,
The accompaniment data feature specifying unit analyzes the accompaniment sound data and specifies the chord progression of the accompaniment sound,
The coefficient specifying unit increases the value of the weighting coefficient for one or more evaluation items including the pitch when the chord progression specified by the accompaniment data feature specifying unit is a predetermined pattern. The evaluation apparatus according to claim 2, wherein: The evaluation unit for each item evaluates the sound data for a plurality of evaluation items including a rhythm based on the features specified by the sound data feature specifying unit,
The accompaniment data feature specifying unit specifies, as a feature of the accompaniment sound data, a location where the rhythm represented by the accompaniment sound data differs from the rhythm represented by the evaluation data,
The said coefficient specific | specification part enlarges the value of the weighting coefficient about the one or some evaluation item containing the said rhythm in the location specified by the said accompaniment data characteristic specific | specification part. Evaluation device. On the computer,
Obtaining evaluation data representing the sound that serves as an example of the music;
Obtaining accompaniment sound data representing the accompaniment sound of the music;
Obtaining sound data representing a singing voice or a performance sound performed according to the accompaniment sound;
Identifying the characteristics of the acquired sound data;
Analyzing the acquired accompaniment sound data to identify the characteristics of the accompaniment sound data, the number of notes included within a predetermined time in the accompaniment sound indicated by the acquired accompaniment sound data ; a step portion where the key is changed, and that at least one of phrase structure of the accompaniment tone, to identify as a feature of the accompaniment tone in those該伴Soon,
A step of evaluating the sound data in an evaluation manner according to the characteristics of the specified accompaniment sound data when evaluating the sound data based on the characteristics of the specified sound data and the acquired evaluation data; A program to execute and.

Images