JP2005107087A - Singing voice evaluating device, karaoke scoring device and program for these devices - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a karaoke device in which a pleasant singing having vibrato is detected in a highly precise manner and the detection result is properly reflected into the evaluation of the singing capability. <P>SOLUTION: The daraoke device is provided with a singing pitch information detecting means which detects the pitch of singing voice and outputs the pitches as singing pitch data and a discrimination means which determines whether or not the pitch indicated by the singing pitch data is fluctuating in an approximate sine wave shape and when the pitch is fluctuating in an approximate sine wave shape, it is judged that the singing has vibrato. When the singing is determined to have vibrato by the discrimination means, a bonus point is added to the score that is obtained by comparing a model pitch data that is the model of singing with the singing pitch data. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a singing voice evaluation device and a karaoke scoring device.

  Many karaoke apparatuses equipped with a function for evaluating singing voices use a difference value between pitch data and volume data extracted from a singing voice signal input from a microphone, and pitch data and volume data of an exemplary guide melody. It is common to score the skill of a song by calculating each. On the other hand, there has been proposed a karaoke apparatus that performs evaluation in consideration of not only the pitch and volume but also the singing technique of “vibrato” (Patent Document 1). Note that “vibrato” is a singing method in which the voice is extended and the melody is tasted while minutely shaking the voice.

  The karaoke apparatus disclosed in Patent Document 1 specifies a period during which vibrato is preferred for singing from a guide melody, and vibrato if the frequency of the singing voice signal corresponding to this period varies with respect to the guide melody. It is assumed that the song is sung by adding a score.

Japanese Patent Laid-Open No. 10-78750

  However, the above-mentioned karaoke apparatus that makes a judgment that depends only on the presence or absence of frequency fluctuations is not only vibrato-savory singing, but also vibrato singing uncomfortable singing with unstable pitches. There was a risk that it could be judged that it was applied, and there was a problem that the skill of singing could not be properly evaluated. There was also a problem that the pitch of vibrato singing could not be correctly evaluated.

  The present invention has been devised under such a background, and provides a karaoke device that detects a vibrato-sounding comfortable song with high accuracy and appropriately reflects this in the evaluation of singing ability. The purpose is to do.

  The singing voice evaluation apparatus according to the preferred embodiment of the present invention includes a singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data, and the pitch indicated by the singing pitch data varies in a substantially sine wave shape. And determining means for determining that vibrato is applied to the singing voice when it is fluctuating in a substantially sine wave shape.

  The karaoke scoring device which is another preferred embodiment of the present invention includes a singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data, and the pitch indicated by the singing pitch data varies in a substantially sine wave shape. Singing model data that is a model of singing and is included in the music data, and a judging means for judging that the singing voice is vibratoed when it is fluctuating substantially sinusoidally The model pitch data output means for outputting the model pitch data according to the read singing model data, and the model pitch data output by the model pitch data output means and the singing pitch data are compared. Calculating means for calculating a score indicating the evaluation of the singing voice, and when the determining means determines that vibrato is applied An adding means for performing point addition to serial score.

  The karaoke scoring device according to another preferred aspect of the present invention is the karaoke scoring device, wherein the music data includes vibrato period instruction information indicating a period during which vibrato is to be sung, and the adding means is vibrato period instruction information. Is characterized in that point addition is performed when indicates the period of vibrato.

  According to another preferred embodiment of the present invention, a program includes a computer device that detects a pitch of singing voice and outputs the singing pitch information as singing pitch data, and the pitch indicated by the singing pitch data has a substantially sinusoidal shape. Whether or not it is fluctuating is detected, and when it fluctuates in a substantially sine wave shape, it functions as a judging means for judging that vibrato is applied to the singing voice.

  According to another preferred embodiment of the present invention, a program includes a computer device that detects a pitch of singing voice and outputs the singing pitch information as singing pitch data, and the pitch indicated by the singing pitch data has a substantially sinusoidal shape. Detecting whether or not it is fluctuating, and determining that vibrato is applied to the singing voice when fluctuating substantially sinusoidally, and singing as an example of singing included in the music data The model pitch data output means for reading the model data according to the progress of the music and outputting the model pitch data according to the read singing model data, and the model pitch data output by the model pitch data output means and the singing pitch data are compared. The calculation means for calculating the score indicating the evaluation of the singing voice and the determination means are vibrato To function as addition means for performing point addition to the score when it is determined.

  ADVANTAGE OF THE INVENTION According to this invention, the comfortable singing with vibrato can be detected with high precision, and this can be reflected in the evaluation of singing ability in the aptitude.

(A: 1st Embodiment)
Embodiments of the present invention will be described below with reference to the drawings.
<Configuration of Embodiment>
FIG. 1 is a block diagram showing a configuration of a karaoke apparatus main body 1 according to an embodiment of the present invention, and each part will be described below.
The CPU 11 controls each part of the apparatus by using the RAM 13 as a work area and executing various programs stored in the ROM 12. A communication I / F (interface) 15 receives music data from the host computer 6 that is a music data distribution source, and transfers the music data to an HDD (Hard Disk Drive) 14 under the control of the CPU 11. Data transfer to the HDD 14 by DMA (Direct Memory Access) is also possible. The music data includes performance data indicating the contents of the accompaniment and lyrics data indicating the lyrics of the music, and also includes music number data specifying the music and data indicating the genre of the music (details will be described later). ).
When the user performs a music designation operation, the designated music data is read from the HDD 14 based on the music number data, and is stored in a MIDI (Musical Instrument Digital Interface: registered trademark) storage area A1 in the RAM 13 (see FIG. 4). ). The CPU 11 sequentially reads and processes the music data in the MIDI storage area A1, thereby progressing the music performance.

  The voice of the singer inputted to the microphone 4 becomes a singing voice signal S1, which is outputted from the speakers 3 and 3 through the amplifier 21, and is inputted to the voice processing DSP 20. The voice processing DSP 20 performs A / D conversion on the singing voice signal S1, then extracts the pitch and volume of the singing voice, and outputs them as singing pitch data SP and singing volume data SV, respectively. In this case, the extraction processing of the audio processing DSP 20 is performed approximately every 30 ms.

The operation unit 16 is an operation panel provided on the front surface of the karaoke apparatus body 1 and has a number of keys such as a numeric keypad and key control keys. The operation unit 16 has a receiving unit that receives signals (infrared signals, wireless signals, etc.) output from the remote control terminal 5, and the signals received by the receiving unit are transferred to the CPU 11.
The display control unit 17 performs control for displaying video data, lyrics, and the like on the monitor 2. Note that the video data is stored in a video data storage unit (DVD playback device or the like) (not shown), and a video corresponding to the genre of the music is read out. The lyrics are displayed using the lyrics data in the music data, and a color change (so-called wipe) process is performed as the music progresses.
The tone generator 18 generates a musical sound signal corresponding to music data (specifically, performance data in the music data) sequentially read from the CPU 11 as the music progresses, and outputs it to the effect DSP 19.
The effect DSP 19 gives effects such as reverberation and echo to the musical sound signal generated by the sound source device 18. The musical sound signal to which the effect is applied is D / A converted by the effect DSP 19 and then amplified by the amplifier 21 and emitted from the speakers 3 and 3 as a karaoke performance sound. Since the amplifier 21 also amplifies the singing voice signal S1, the singing voice inputted from the microphone 4 and the karaoke performance sound are mixed here.

Here, the structure of music data used in this embodiment will be described.
The music data in the present embodiment has a header and a plurality of tracks, and the header portion indicates music number data for specifying the music, music title data indicating the music title, and genre as shown in FIG. Genre data, performance time data indicating the performance time of music, and the like are included. On the other hand, a plurality of tracks following the header include a guide melody track describing guide melody data representing the content of the melody to be sung by the user, a performance track describing the performance data representing the content of the karaoke performance sound, and lyrics There is a lyric track in which lyric data representing the contents of is described. In the case of this embodiment, the above music data is described according to the MIDI format.

Next, guide melody data described in the guide melody track will be described. The guide melody data of the present embodiment is the same as general guide melody data except that a portion to be sung can be recognized using vibrato.
Since FIG. 3 is a matrix of rows and columns, first the columns will be described.
The delta time in the first column indicates the time interval between events, and is represented by the number of tempo clocks. When the delta time is “0”, it is executed simultaneously with the immediately preceding event. The second column describes the contents of the message that each event of the performance data has. This message includes a control change message and the like in addition to a note-on message (NoteOn) indicating a sound generation event and a note-off message (NoteOff) indicating a mute event.

The third column describes channel numbers. Here, for simplicity of explanation, the channel number of the guide melody track is “1”.
In the fourth column, a note number (NoteNum) or a control number (CtrlNum) is described. Which is described depends on the content of the message. For example, in the case of a note-on message or a note-off message, a note number indicating a scale is described here, and in the case of a control change message, a control number indicating the type is described.
The fifth column describes specific values (data) of the MIDI message. For example, in the case of a note-on message or a note-off message, a velocity value representing the sound intensity is described here, and in the case of a control change message, a parameter value corresponding to a control number is described.

Next, each row shown in FIG. 3 is a musical tone parameter indicating the attribute of each note of the guide melody, and is composed of a note-on event, a note-off event, and a control change message.
In the example shown in FIG. 3, the length of the delta time 480 is the length of a quarter note. In this case, the C4 sound is generated along the length of the quarter note by the event processing of the first and second rows, and the G4 sound is generated over the length of the quarter note by the event processing of the third and fourth rows. Pronounced. Then, the F4 sound is pronounced over the length of the half note by the processes in the sixth and seventh lines.

  Here, a control change message with a control number “1” is described in the fifth line, and a control change message with a control number “2” is described in the eighth line. In this embodiment, the control number “1” is used for an instruction of “start vibrato” indicating the start of singing with vibrato, and the control number “2” is “vibrato end” indicating the end of vibrato. Used for instructions. Therefore, in the example shown in FIG. 3, it is instructed that the F4 sound generated over the length of a half note is a sound to be sung with vibrato. When a plurality of notes are located between the control number “1” and the control number “2”, all of these notes become notes to be sung with vibrato. As described above, the location to be sung by vibrato is instructed in advance.

  Next, a storage area set in the RAM 13 will be described with reference to FIG. The MIDI storage area A1 is an area for storing music data transferred from the HDD. In the flag area A2, a flag having a value of “1” is set in a period during which vibrato should be applied (hereinafter referred to as “vibrato period”). In the difference value storage area A3, a difference value between the singing pitch data SP and the guide melody pitch data GP and a difference value between the singing volume data SV and the guide melody volume data GV are stored.

<Operation of Embodiment>
Next, the operation of the karaoke apparatus having the above configuration will be described.
When the user performs a music specifying operation using the numeric keypad of the operation unit 16 or the remote control terminal 5, the music data of the specified music is transferred from the HDD 14 to the MIDI storage area A1 of the RAM 13. The CPU 11 executes karaoke accompaniment and lyrics display processing by sequentially reading the music data events. Specifically, the event data described in the performance track of the musical sound data is output to the tone generator circuit 16 and the lyrics data of the lyrics track is output to the display control unit 17. As a result, the karaoke accompaniment sound is output from the speakers 3 and 3, while the lyrics generated by the display control unit 17 are displayed on the monitor 2.

  On the other hand, the CPU 11 generates the guide melody pitch data GP and the guide melody volume data GV from the note number and velocity of the note-on event (NoteOn) of the guide melody data read according to the progress of the music, respectively, and the singing pitch. The scoring process is performed by comparing the data SP and the singing volume data SV. Since this scoring process is different between the period in which the vibrato is designated and the other period, each will be described below.

  First, the CPU 11 performs the process shown in FIG. 5 in order to recognize the period when the vibrato is designated. In step Sa1 shown in FIG. 5, it is determined whether or not the control message is “1”. If “1”, it is determined that the start of vibrato is instructed, and “1” is stored in the flag area A2 in the RAM 13. (Step Sa2). If the control message is “2”, it is determined that the end of the vibrato is instructed, and the flag area A2 in the RAM 13 is reset (steps Sa3 and 4). With the above processing, it can be determined whether or not vibrato is designated by looking at the contents of the flag area A2.

Next, the scoring process according to the contents of the flag area A2 will be described. The process shown in FIG. 6 is a process for preparing for scoring, and accumulates the results of the sequential evaluation when the singer sings. First, the CPU 11 refers to the contents of the flag area A2 of the RAM 13 and determines whether or not the value is “1” (steps Sb1 and Sb2). If the value of the flag area A2 is not “1”, the process proceeds to step Sb3 to calculate difference value data regarding pitch, volume, and rhythm. Then, the calculated difference value data is accumulated and stored in the difference value storage area A3 of the RAM 13. Details of the processing in step Sb3 are as follows.
1. The difference between the guide melody pitch data GP and the singing pitch data SP is detected, and stored in the difference value storage area A3 of the RAM 13 as pitch difference value data PD.
2. The difference between the volume represented by the guide melody volume data GV and the singing volume data SV is accumulated and stored in the difference value storage area A3 as volume difference value data VD.
3. The time difference between the sounding timing (or mute timing) of the guide melody and the rising (or falling) timing of the singing volume data SV is accumulated and stored in the difference value storage area A3 as rhythm difference value data RD. The above processing is performed for each note.

On the other hand, when the content of the flag area A2 is “1”, the CPU 11 stores the singing pitch data SP together with the time in the vibrato data storage area A4 of the RAM 13 (step Sb4). That is, for the singing voice during the vibrato period, the difference value is not calculated, and the singing pitch data SP is accumulated in time series.
As described above, if the period is not the vibrato period, difference value data regarding the pitch, volume, and rhythm is sequentially accumulated for the user's song. If the period is the vibrato period, the value of the singing pitch data SP is set every 30 ms. It is recorded and the state of the pitch change is saved.

FIG. 7 is a flowchart showing the score calculation process.
First, the CPU 11 reads out the pitch difference value data PD, the volume difference value data VD, and the rhythm difference value data RD accumulated in the difference value storage area A3 of the RAM 13, sums up each, and a deduction point corresponding to the summation result. Is calculated (step Sc1). In this case, the point of deduction is calculated so that the user's singing is shifted from the guide melody. That is, the deduction point increases as the total value of the difference value data PD, BD, RD increases.
Then, the CPU 11 subtracts this deduction point from the full score (100 points) (step Sc2). Further, the CPU 11 extracts the singing pitch data SP corresponding to each vibrato period from the vibrato data storage area A4 of the RAM 13 (step Sc3), and determines whether or not the change in the extracted singing pitch data SP matches the substantially sine wave shape. (Step Sc4).
Here, the determination process will be described.
FIG. 8A is a graph plotting the transition of the singing pitch data SP extracted during a certain vibrato period. CPU11 calculates | requires a period from the space | interval of a zero crossing point by making the center of the amplitude of song pitch data SP into a zero point. If the obtained cycle is constant, it is determined that good vibrato is applied. Vibrato is evaluated depending on how constant the cycle is. If the variation of the cycle is more than a certain value, it is determined that the vibrato is not applied. In the case of FIG. 8A, the singing pitch data SP transitions with about 240 ms (8 samplings: 30 ms × 8) as one cycle, which is an example of a good vibrato.
FIG. 8B is a graph plotting transitions of the singing pitch data SP extracted during another vibrato period. According to this graph, the period of the singing pitch data SP varies and does not transition to a sine wave shape. Therefore, when the singing pitch data changes following the trajectory shown in this graph, the CPU 11 determines that the vibrato is not applied.
Moreover, FIG.8 (c) is the graph which plotted the transition of the song audio | voice data SP extracted in another vibrato period. According to this graph, the cycle of the singing pitch data SP is substantially constant, but the amplitude varies depending on the cycle. In this embodiment, it is evaluated that the vibrato is applied when the cycle of the singing pitch data SP is constant. However, the change in the amplitude of the singing pitch data SP is also detected, and if this is not constant, the quality of the vibrato May be determined to be inferior and the evaluation of bonus points may be lowered. If the change in amplitude is significant, it may be determined that no vibrad is applied even if the period is constant.

  Next, the process proceeds to step Sc5, where the bonus points calculated in step Sc4 are added to the subtraction result in step Sc2, and the process proceeds to step Sc6, where the total score after the addition is output to the display control unit 17. As a result, the total score is displayed on the monitor 2 under the control of the display control unit 17.

As described above, the karaoke apparatus according to the present embodiment inserts data specifying the vibrato period into the guide melody track of the music data, and the singing pitch data SP in this period varies in a substantially sine wave shape. Assume that vibrato is applied and add bonus points. Thereby, vibrato which is an advanced singing technique can be accurately detected and reflected in the singing score.
In addition, the singing voice during the vibrato period is not evaluated by comparing it with the guide melody, so that the singing with the vibrato is not evaluated as a pitch shift or a volume shift, and it is suitable for the singing technique. It can be performed.

(B: Other embodiments)
Other embodiments of the present invention are shown below.
1. In the above-described embodiment, the period for singing with vibrato is specified by the control change information in the music data, but the method of specifying the vibrato period is not limited to this. For example, a note longer than a predetermined length (for example, a note longer than a half note) may be recognized as a vibrato period. Furthermore, the vibrato period may be recognized when the result connected by slur or tie exceeds a predetermined length. Such recognition can be easily realized by reading and analyzing events prior to performance when playing music data in MIDI format. Furthermore, when data indicating the break of a measure is inserted into the music data, it is possible to recognize the vicinity of the end of the measure, so if there is a note longer than a predetermined length near the end of the measure, You may be made to recognize that it is a vibrato period about a note. In the above case, the note length itself becomes the vibrato period instruction information.

2. You may display on the monitor 2 that it is a vibrato period. This display control may be performed by detecting control change information in the music data, or by detecting the length of a note. Further, in the above-described embodiment, it can be determined whether the period is the vibrato period by looking at the contents of the flag area A2 of the RAM 13, so the display may be controlled in accordance with the contents of the flag area A2. The content of the display may be a word indicating that it is a vibrato period or a modified display of lyrics. The modification of the lyric display is, for example, displaying the characters of the lyrics to be sung larger than other characters, or swaying the characters left and right. Further, the vibration device may be built in the microphone 4 and the vibration device may be vibrated during the vibrato period so that the singer can recognize that it is the vibrato period. The above is a modification regarding the announcement of the vibrato period. Conversely, when vibrato is applied to the singing voice input from the microphone 4, the display may be performed according to the state of the vibrato. . As a display mode, a predetermined moving image is displayed, or the display of the lyrics portion to which vibrato is applied is modified.

3. Data indicating the pitch fluctuation width and volume fluctuation width at the time of vibrato may be arranged in the music data, and the recommended vibrato state may be indicated for each music. As for scoring, the closer to the recommended vibrato state, the higher the score.

4). Depending on the state of the user's vibrato, the content may be displayed. For example, when the total number of bonus points to be added exceeds a predetermined score (for example, 10 points), a message such as “Nice Vibrato” may be displayed together with the total score. In addition, information on vibrato amplitude and period may be displayed together with the total score in numerical values and figures.

5. In the above-described embodiment, if the singing pitch data SP has changed to a substantially sine wave shape, it is determined that the song is a vibrato singing, but the singing volume data SV changes to a substantially sine waveform. You may decide that the song is vibrato. When a vibrato that is comfortable to listen to is applied, not only the pitch but also the volume transitions to a substantially sinusoidal shape, so even if the evaluation is based on the singing volume data SV, the vibrato can be accurately detected. Can do. Moreover, you may make it determine whether vibrato is applied based on the state of both singing pitch data SP and singing volume data SV.

6). In the above-described embodiment, the singing pitch data SP is stored and stored, and the presence or absence of vibrato is determined based on the singing pitch data SP after the performance is finished. However, the vibrato is not stored and stored. It may be possible to make a judgment on a singing song in real time.

7). The process of determining whether or not the change in the singing pitch data SP is a sine wave is not limited to the process of the above-described embodiment. In short, since it is only necessary to determine whether or not it is a sine wave, for example, a change in pitch may be subjected to Fourier analysis and determined from its spectrum. If it is a perfect sine wave, a single spectrum is detected, so the closer it is, the closer it is to a sine wave.

8). In the above-described embodiment, the vibrato period is recognized and the vibrato state detected in the vibrato period is determined. However, the vibrato period is not determined, and if any note is associated with vibrato, points are added. May be. Furthermore, the vibrato detection result can be used not only for adding points and display control, but also for arbitrary control. For example, sound effects can be controlled according to the vibrato state.

9. In the embodiment described above, the singing voice during the vibrato period was not evaluated by comparison with the guide melody. However, even during the vibrato period, difference value data with the guide melody was obtained, and this was used as a scoring target. It can also be added. In this case, the pitch of the singer to be scored may be the pitch at the time of singing each note within the vibrato period, or the average pitch for each note. What is necessary is just to compare the singer's pitch specified in this way with the guide melody pitch data GP. The singer's volume may be specified as the average volume for each note in the vibrato period. The rhythm can be compared by detecting the rising timing and falling timing of each note of the singer's voice and comparing the sound generation timing and the mute timing of the guide melody.

10. In the embodiment described above, the scoring is performed by comparing the guide melody pitch data GP generated from the note number of the note-on event (NoteOn) of the guide melody data with the singing pitch data SP. The GP itself may be stored from the beginning, and scoring may be performed by comparing the pitch data GP with the singing pitch data SP.

It is a block diagram which shows the electric constitution of a karaoke apparatus main body. It is a figure which shows the structure of music data. It is a figure which shows the content of the guide melody track | truck contained in music data. It is a figure which shows the memory area ensured by RAM. It is a flowchart which shows the detection process of a vibrato period. It is a flowchart which shows the accumulation | storage process of evaluation data. It is a flowchart which shows a score calculation process. It is a graph which shows transition of the singing pitch data extracted in the vibrato period.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Karaoke apparatus main body, 2 ... Monitor, 3 ... Speaker, 4 ... Microphone, 5 ... Remote control terminal, 6 ... Host computer, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... HDD, 15 ... Communication I / F , 16 ... operation unit, 17 ... display control unit, 18 ... sound source device, 19 ... DSP for effect, 20 ... DSP for sound processing, 21 ... amplifier

Claims (5)

Singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data;
Detecting means for detecting whether or not the pitch indicated by the singing pitch data changes in a substantially sine wave shape, and determining that the singing voice is vibratoed when changing in a substantially sine wave shape; A singing voice evaluation device. Singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data;
Detecting means for detecting whether or not the pitch indicated by the singing pitch data is changing in a substantially sine wave shape, and determining that vibrato is applied to the singing voice when changing in a substantially sine wave shape;
Model pitch data output means for reading out singing model data included in the music data as a model of singing according to the progress of the music, and outputting model pitch data according to the read singing model data;
Calculating means for calculating a score indicating evaluation of the singing voice by comparing the exemplary pitch data output by the exemplary pitch data output means and the singing pitch data;
A karaoke scoring device comprising: adding means for adding points to the score when the determining means determines that vibrato is applied.   The music data includes vibrato period instruction information indicating a period during which vibrato should be sung, and the adding means performs point addition when the vibrato period instruction information indicates a vibrato period. 2. The karaoke scoring device according to 2. Computer equipment,
Singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data;
Detecting means for detecting whether or not the pitch indicated by the singing pitch data varies in a substantially sinusoidal shape, and determining that vibrato is applied to the singing voice when the pitch varies in a substantially sinusoidal shape; Program to function. Computer equipment,
Singing pitch information detecting means for detecting the pitch of the singing voice and outputting it as singing pitch data;
Detecting means for detecting whether or not the pitch indicated by the singing pitch data is changing in a substantially sine wave shape, and determining that vibrato is applied to the singing voice when changing in a substantially sine wave shape;
Model pitch data output means for reading out singing model data included in the music data as a model of singing according to the progress of the music, and outputting model pitch data according to the read singing model data;
Calculating means for calculating a score indicating evaluation of the singing voice by comparing the exemplary pitch data output by the exemplary pitch data output means and the singing pitch data;
A program for functioning as addition means for adding points to the score when the determination means determines that vibrato is applied.

Images