JP4862413B2 - Karaoke equipment - Google Patents

Description

  The present invention relates to a technique for scoring a singer's singing ability.

Some karaoke apparatuses that perform automatically based on music data analyze a singer's voice input to a microphone and score the singer's singing ability. For example, the karaoke apparatus disclosed in Patent Document 1 recognizes the words of the singer's voice input to the microphone and evaluates how much the words match the words of the lyrics of the music. According to this karaoke apparatus, it is possible to evaluate whether or not the singer correctly remembers the lyrics.
JP-A-10-91172

  By the way, in order to recognize the wording of a voice | voice like the karaoke apparatus currently disclosed by patent document 1, it is necessary to perform voice recognition. When performing speech recognition, the input speech is analyzed and the acoustic features of the speech are extracted. Then, from the words stored in the dictionary, the word whose acoustic feature is closest to the acoustic feature of the input speech is searched for and output as a speech recognition result. Here, in order to correctly recognize words, words stored in the dictionary are important, and in order to correctly recognize words, it is necessary to store many words in the dictionary. However, if many words are stored in the dictionary, it takes time to find the closest word from many words, and the evaluation result cannot be immediately displayed. There are many songs sung in karaoke that are not only in Japanese but also in foreign languages. When performing speech recognition for a large number of languages, it is necessary to prepare a dictionary for each language, and when adding songs in a new language to a karaoke device, a dictionary must also be newly prepared. The problem becomes that it becomes difficult to add music easily due to complexity.

  The present invention has been made under the background described above, and its purpose is to enable a singer to evaluate whether or not he / she correctly remembers lyrics without complicating the system.

In order to solve the above-described problems, the present invention includes a storage unit that stores model voice data representing a model voice when a song is sung according to lyrics, and a voice input unit that inputs a singing voice of a singer. Specifying means for dividing the model voice represented by the model voice data into a plurality of voice sections, and identifying voice sections corresponding to the divided voice sections in the singing voice input to the voice input means; The evaluation means for evaluating the correctness of the lyrics by comparing the singing voice of the voice section specified by the specifying means and the model voice corresponding to the singing voice of the voice section, and the evaluation result of the evaluation means A karaoke apparatus having display means for displaying is provided.

In this aspect, the evaluation means obtains the degree of coincidence between the singing voice of the voice section specified by the specifying means and the model voice corresponding to the singing voice of the voice section, and based on the obtained degree of coincidence , You may make it evaluate correctness .
In addition, the storage unit stores lyrics data representing the lyrics of the music, and when the degree of coincidence obtained by the evaluation unit is less than a predetermined value, the voice of the voice section in which the degree of coincidence is less than the predetermined value The lyrics specifying means for specifying the lyrics corresponding to the lyrics from the lyrics represented by the lyrics data stored in the storage means, and the display means displays the lyrics specified by the lyrics specifying means Good.
Further, the evaluation means may obtain a degree of coincidence between the formant frequency of the singing voice and the formant frequency of the model voice.

  According to the present invention, it is possible to evaluate whether a singer correctly remembers lyrics without complicating the system.

[Configuration of the embodiment]
FIG. 1 is an external view of a karaoke apparatus according to an embodiment of the present invention. As shown in the figure, the karaoke apparatus 1 is connected with a monitor 2, a speaker 3L, a speaker 3R, and a microphone 4. Karaoke device 1 is remotely operated by an infrared signal transmitted from remote control device 5.

  FIG. 2 is a block diagram showing a hardware configuration of the karaoke apparatus 1. Each unit connected to the bus 101 communicates with each other via the bus 101. A CPU (Central Processing Unit) 102 uses a RAM (Random Access Memory) 104 as a work area and executes various programs stored in a ROM (Read Only Memory) 103 to control each unit of the karaoke apparatus 1. . The RAM 104 has a music storage area for temporarily storing music data. The storage unit 105 includes a hard disk device, and stores various data such as music data described later and digital data of singing voice input from the microphone 4.

The communication unit 108 receives music data from a host computer (not shown), which is a music data distribution source, via a communication network (not shown) such as the Internet, and the received music data is controlled by the CPU 102. Transfer to the storage unit 105. In the present embodiment, the music data may be stored in the storage unit 105 in advance. Further, the karaoke apparatus 1 is provided with a reading device that reads various recording media such as CD-ROM and DVD, and the music data recorded on the various recording media is read by the reading device and transferred to the storage unit 105 to be stored. It may be.
Here, the structure of music data used in the present embodiment will be described. As shown in FIG. 3, the music data in the present embodiment includes a header, musical sound data that is WAVE data representing the contents of the karaoke performance sound, and a waveform of a model voice when the lyrics of the music are correctly sung. Model audio data in the WAVE format, and a lyrics table storing lyrics data representing the lyrics of the music.

FIG. 4 is a diagram illustrating the format of the lyrics table. In the lyrics table, the lyrics data representing the lyrics of the music to be played is associated with the time interval data indicating the time interval in which the lyrics represented by the lyrics data should be pronounced when the musical sound is output according to the musical sound data. Stored.
For example, in the lyrics table shown in FIG. 4, the lyrics data on the first line represents the lyrics “Kamere-onga”, and the time interval data “01: 00-01: 02” associated with the lyrics data. "Indicates that the lyrics" Kamere-onga "is pronounced in the model voice from the time when 1 minute has passed since the start of the performance of the music to the time when 1 minute and 2 seconds have passed. The lyric data on the second line represents the lyrics “I have come”, and the time section data “01: 03-01: 06” associated with the lyric data is the music in the model voice. This means that the lyrics “I have come” will be pronounced between the time when 1 minute 3 seconds have passed and the time 1 minute 6 seconds have passed.

  The microphone 4 converts the singing voice of the input singer into a voice signal and outputs it. The audio signal output from the microphone 4 is input to an audio processing DSP (Digital Signal Processor) 111 and an amplifier 112. The voice processing DSP 111 performs A / D conversion on the input voice signal, and generates singing voice data representing the singing voice. This singing voice data is stored in the storage unit 105, compared with the model voice data, and used for scoring the singing ability of the singer.

  The input unit 106 detects a signal generated by an input operation to the operation panel or the remote control device 5 in the karaoke device 1 and outputs the detection result to the CPU 102. The display control unit 107 displays the video and the singer's singing score on the monitor 2 under the control of the CPU 102.

  The tone generator 109 generates a tone signal corresponding to the supplied tone data, and outputs the generated tone signal to the effect DSP 110 as a karaoke performance sound. The effect DSP 110 gives effects such as reverberation and echo to the musical sound signal generated by the sound source device 109. The effected tone signal is D / A converted by the effect DSP 110 and output to the amplifier 112. The amplifier 112 synthesizes and amplifies the musical tone signal output from the effect DSP 110 and the audio signal output from the microphone 4 and outputs the resultant signal to the speakers 3L and 3R. Thereby, the melody of music and the voice of the singer are output from the speakers 3L and 3R.

[Operation of the embodiment]
Next, the operation of this embodiment will be described. First, when the user operates the remote controller 5 to designate a music piece, the music data of the designated music piece is transferred from the storage unit 105 to the music storage area of the RAM 104 by the CPU 102. The CPU 102 executes karaoke accompaniment processing by sequentially reading various data included in the music data stored in the music storage area.

  Specifically, the CPU 102 reads out musical tone data included in the music data and outputs the read musical tone data to the sound source device 109. The tone generator 109 generates a tone signal of a predetermined tone color based on the supplied music data, and outputs the generated tone signal to the effect DSP 110. In the effect DSP 110, effects such as reverb and echo are applied to the musical sound signal output from the sound source device 109. The effected tone signal is D / A converted by the effect DSP 110 and output to the amplifier 112. The amplifier 112 amplifies the musical tone signal output from the effect DSP 110 and outputs it to the speakers 3L and 3R. Thereby, the melody of a music is output from the speakers 3L and 3R. In addition, when the music data is supplied to the sound source device 109 and the output of the musical sound is started, the CPU 102 starts counting the elapsed time that has elapsed since the output of the music is started.

  On the other hand, when the singer sings according to the reproduction of the music, the singer's voice is input to the microphone 4 and an audio signal is output from the microphone 4. The voice processing DSP 111 performs A / D conversion on the voice signal output from the microphone 4 and generates singing voice data representing the singing voice. This singing voice data is stored in the storage unit 105.

  The CPU 102 continues counting the elapsed time, and searches the lyrics table for a time interval including the counted time as the start time of the time interval. Then, the retrieved time interval and the lyrics data stored in association with the retrieved time interval are read out. For example, if the counted elapsed time is 01:00, the time section “01: 00-01: 02” and the lyrics data “Kamere-onga” on the first line are read out in the lyrics table shown in FIG. .

  When the CPU 102 reads out the time section, the CPU 102 compares the voice input to the microphone 4 in this time section with the model voice in this time section, and determines whether or not the singer sang the lyrics correctly. Specifically, the CPU 102 analyzes the voice represented by the model voice data and, as shown in FIG. 5, reads the time interval (01: 00-01) on the time axis of the voice waveform represented by the model voice data. : The speech waveform A between 02) is extracted. In addition, the CPU 102 analyzes the stored singing voice data, and extracts the voice waveform B between the read time sections on the time axis represented by the singing voice data, as shown in FIG. Then, as shown in FIG. 6A, the extracted speech waveform A is divided at a predetermined time interval (for example, 10 ms) and divided into a plurality of frames. Further, as shown in FIG. 6B, the extracted speech waveform B is divided at a predetermined time interval (for example, 10 ms) and divided into a plurality of frames.

  Next, the CPU 102 associates the speech waveform of each frame of the model speech with the speech waveform of each frame of the singing speech using a DP (Dynamic Programming) matching method. For example, in the waveform illustrated in FIG. 6, when the voice waveform of the frame A1 of the model voice corresponds to the voice waveform of the frame B1 of the singing voice, the frame A1 and the frame B1 are associated with each other. Further, when the voice waveform of the frame A2 of the model voice and the voice waveform of the frames B2 to B3 of the singing voice correspond to each other, the frame A2 and the frames B2 to B3 are associated with each other.

Next, the CPU 102 compares the characteristics of the speech waveform between corresponding frames. Specifically, the CPU 102 Fourier transforms the speech waveform for each speech waveform of each frame of the model speech. Then, the CPU 102 obtains the logarithm of the amplitude spectrum obtained by the Fourier transform, and inversely transforms it to generate a spectrum envelope for each frame. Then, the CPU 102 extracts the frequency f11 of the first formant, the frequency f12 of the second formant, and the frequency f13 of the third formant from the obtained spectrum envelope.
In addition, the CPU 102 performs a Fourier transform on the speech waveform for each speech waveform of the singer's speech frame associated with each frame of the model speech. Then, the CPU 102 obtains the logarithm of the amplitude spectrum obtained by the Fourier transform, and inversely transforms it to generate a spectrum envelope for each frame. Then, the CPU 102 extracts the frequency f21 of the first formant, the frequency f22 of the second formant, and the frequency 23 of the third formant from the obtained spectrum envelope.

For example, the CPU 102 generates a spectrum envelope of the frame A1 of the model voice, and extracts formant frequencies f11 to f13 of the first to third formants from the spectrum envelope. And CPU102 produces | generates the spectrum envelope of the audio | voice waveform of the flame | frame B1 matched with the flame | frame A1, and extracts the formant frequencies f21-f23 of the 1st-3rd formants from this spectrum envelope.
Further, the CPU 102 generates a spectrum envelope of the frame A2 of the sample voice, and extracts the first to third formant formant frequencies f11 to f13 from the spectrum envelope. Then, the CPU 102 generates a spectrum envelope of the speech waveforms of the frames B2 to B3 associated with the frame A2, and extracts the first to third formant formant frequencies f21 to f23 from the spectrum envelope.

Next, the CPU 102 compares the formant frequencies f11 to f13 extracted from each frame of the model voice with the formant frequencies f21 to f23 extracted from the frame associated with each frame of the model voice. Then, the CPU 102 determines that the difference between the formant frequency f11 and the formant frequency f21, the difference between the formant frequency f12 and the formant frequency f22, and the difference between the formant frequency f13 and the formant frequency f23 are equal to or greater than a predetermined value. Adds mismatch information D indicating that the formant frequencies do not match to the frame of the model voice.
For example, if the formant frequencies f11 to f13 of the speech waveform of the frame A1 match the formant frequencies of the speech waveform of the frame B1, the CPU 102 determines that the speech is matched between the corresponding frames, and does not match. Information D is not added to the frame A1.
On the other hand, when the difference between the formant frequencies f11 to f13 of the frame A2 and the formant frequencies f21 to f23 of the speech waveforms of the frames B2 to B3 is equal to or greater than a predetermined value, the formant frequencies do not match. Is added to the frame A2.

When determining whether the voice waveform of each frame of the model voice matches or does not match the formant frequency of the singer's voice waveform, the CPU 102 counts the number N of frames to which the mismatch information D is added. Next, the CPU 102 compares the total number M of frames of the divided sample voice data with the value of the number N. If the value of the number N is more than half of the total number M of frames, the read lyrics data is When it is determined that the lyric of the singer is different from the lyric of the model voice and the value of the number N is less than half of the total number M of frames, the singing is performed on the lyrics represented by the read lyric data. It is determined that the lyrics that the person pronounces are the same as the lyrics of the model voice. For example, if the number N of mismatch information is less than half of the total number M of frames for the voice “Kamere-onga” represented by the model voice data, the CPU 102 determines the lyrics of the singer and the model voice. Judge that the lyrics are the same.
In the present embodiment, when the value of the number N is half or more of the total number M of frames, the lyrics expressed by the read lyrics data are different from the lyrics of the singer's pronunciation and the lyrics of the model voice. If the ratio of the number N with respect to the total number M of frames is equal to or greater than a predetermined ratio other than 50%, the lyrics expressed by the lyrics data read out are the lyrics of the singer and the lyrics of the model voice. You may make it judge that it is different.

  The CPU 102 continues to count the elapsed time in parallel with the comparison between the model voice and the singing voice. When the counted elapsed time reaches 01:03, the time interval “01:03” in the second row of the lyrics table shown in FIG. “-01: 06” and the lyrics data “I came over” are read out. Further, when the singer sings in the read time interval according to the reproduction of the music, the singing voice data is stored in the storage unit 105. Here, for example, when the singer sings with a lyric “I will come” different from the lyric “I came” expressed by the read lyric data 2 when the singer mistakes the lyric, “I will come” Is generated and stored in the storage unit 105.

  Next, the CPU 102 divides the waveform of the sound input to the microphone 4 in this time interval and the waveform of the model sound in this time interval into a plurality of frames. Then, the speech waveform of each frame of the model speech is associated with the speech waveform of each frame of the singing speech, and the formant frequencies of the speech waveforms are compared between the associated frames. Then, the CPU 102 determines whether the voice waveform of each frame of the model voice matches or does not match the formant frequency of the singer's voice waveform, adds the mismatch information D, and then adds the total number of frames of the sample voice data divided. M is compared with the value of the number N of frames to which the mismatch information is added, and it is determined whether or not the singer sang the lyrics correctly.

  Here, because the singer sang with the lyrics different from “I come” against the lyrics “I came”, comparing the formant frequency of the voice waveform of the model voice with the formant frequency of the voice waveform of the singer The formant frequencies do not match, and the number N of mismatch information is equal to or greater than the total number M of frames. When the value of the number N is more than half of the total number M of frames, the CPU 102 determines that the lyrics expressed by the read lyrics data are different from the lyrics of the singer's pronunciation and the lyrics of the model voice. The display unit 107 controls the display 2 to display the lyrics “I have come” represented by the lyrics data to notify that the lyrics are wrong.

  Hereinafter, as described above, the CPU 102 repeats the reading of the lyrics data and the model voice data and the determination of the correctness of the lyrics sung by the singer along with the reproduction of the music. Then, when all performance event data is read, the karaoke accompaniment process is terminated.

  As described above, according to the present embodiment, it is possible to determine whether or not the singer has sung according to the lyrics without performing voice recognition using a dictionary. In addition, in this embodiment, if there is data of voice sung correctly according to the lyrics, it is possible to evaluate whether or not the singing was correctly performed according to the lyrics, so the system is configured like a mode of performing language recognition using a dictionary. Without being complicated, it is possible to evaluate whether or not the singer remembers the lyrics correctly for lyrics in various languages.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, For example, you may implement the present invention, changing the above-mentioned embodiment as follows.

  In the embodiment described above, the pitch of the voice represented by the singing voice data may be corrected so that the pitch of the voice waveform represented by the singing voice data becomes the pitch of the voice waveform represented by the model voice data.

In the embodiment described above, periodic fluctuations in the pitch of the voice waveform represented by the model voice data are detected to determine whether the model voice is vibrato, and the vibrato is applied. If it is determined, the degree of coincidence between the pitch fluctuation of the voice waveform represented by the model voice data and the pitch fluctuation of the voice waveform represented by the singing voice data is determined, and whether or not the singer is singing with vibrato correctly being sung. You may make it judge.
In addition, the pitch fluctuation of the voice waveform represented by the model voice data is detected to determine whether or not the voice serving as the model is screaming. The degree of coincidence between the pitch fluctuation and the pitch fluctuation of the voice waveform represented by the singing voice data may be determined, and it may be determined whether or not the singer sings correctly.

  In the above-described embodiment, the voice waveform represented by the model voice data and the voice waveform represented by the singing voice data are divided into a plurality of frequency bands by a plurality of bandpass filters, and the feature amount of the voice for each frequency band. The correctness of the lyrics may be determined by determining the degree of coincidence.

  In the above-described embodiment, model voice data representing a model voice waveform is stored, and the voice waveform represented by the model voice data is analyzed to analyze the formant frequency. The formant frequency for each frame when the frame is divided is stored in the storage unit 105 in advance, and the degree of coincidence is determined by comparing the stored formant frequency with the formant frequency of each frame of the singer's speech waveform. It may be.

  In the above-described embodiment, after the singer has finished singing the music, the correctness of the lyrics sung by the singer may be determined. Further, in the above-described embodiment, when it is determined that the lyrics of the singer's pronunciation and the lyrics of the model voice are different from each other, a message or an image notifying that the lyrics are wrong is displayed instead of displaying the lyrics. May be displayed.

1 is an external view of a karaoke apparatus according to an embodiment of the present invention. It is the block diagram which showed the hardware constitutions of the karaoke apparatus. It is the figure which illustrated the format of the music data in the embodiment. It is the figure which illustrated the format of the lyrics table. It is the figure which illustrated the waveform of the model voice and the waveform of the singing voice. It is a figure when the waveform of a model voice and the waveform of a singing voice are divided into a plurality of frames.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Karaoke device, 2 ... Monitor, 3L, 3R ... Speaker, 4 ... Microphone, 5 ... Remote control device, 101 ... Bus, 102 ... CPU, 103 ... ROM, 104 ... RAM, 105 ... storage unit, 106 ... input unit, 107 ... display control unit, 108 ... communication unit, 109 ... sound source device, 110 ... for effect DSP, 111 ... DSP for voice processing, 112 ... amplifier

Claims (4)

Storage means for storing example voice data representing a model voice when a song is sung according to lyrics;
Voice input means for inputting the singing voice of the singer;
A specifying unit that divides a sample voice represented by the sample voice data into a plurality of voice sections, and specifies a voice section corresponding to each divided voice section in the singing voice input to the voice input unit;
An evaluation means for evaluating the correctness of the lyrics by comparing the singing voice of the voice section specified by the specifying means and the model voice corresponding to the singing voice of the voice section;
A karaoke apparatus comprising: display means for displaying an evaluation result of the evaluation means. The evaluation means obtains the degree of coincidence between the singing voice of the voice section specified by the specifying means and the model voice corresponding to the singing voice of the voice section, and evaluates the correctness of the lyrics based on the obtained degree of coincidence. The karaoke apparatus according to claim 1, wherein: The storage means stores lyric data representing the lyrics of the music,
If the degree of coincidence obtained by the evaluation means is less than a predetermined value, the lyrics corresponding to the speech of the voice section where the degree of coincidence is less than the predetermined value are included in the lyrics represented by the lyrics data stored in the storage means There is a lyrics identification means to identify from
The karaoke apparatus according to claim 2, wherein the display means displays the lyrics specified by the lyrics specifying means.   The karaoke apparatus according to claim 2, wherein the evaluation unit obtains a degree of coincidence between a formant frequency of the singing voice and a formant frequency of the model voice.

Images