JP2007033471A - Singing grading apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a singing grading apparatus which can perform grading by audibility more similar to human being's irrespective of pitches of voices uttered when singing. <P>SOLUTION: A karaoke system 11 performs karaoke performance by reproducing sequence data. During the performance, the karaoke system calculates a pitch ratio of a pitch of a voice input from a microphone 12 to that of a voice which is identified from the sequence data to be uttered, and uses the calculated pitch ratio to perform evaluation using two or more pitches ratios as a grading unit. After finishing the karaoke performance, the system performs grading of the user's singing using the evaluation resulting from the grading units, and displays the grading result on a display apparatus 14. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for evaluating and scoring a song sung by a user.

  Karaoke that sings along with accompaniment is now enjoyed by many people as an amusement. Among the karaoke devices for karaoke, many are equipped with a singing scoring device for scoring the singing ability so as to be able to cope with a person who wants to grasp his / her singing ability objectively.

  The scoring is usually performed by paying attention to at least one of the pitch (pitch) of the voice uttered by the user and the utterance timing of the voice. As a conventional singing scoring device that performs such scoring, for example, there are devices described in Patent Documents 1 and 2, respectively.

  In the conventional singing scoring devices described in Patent Documents 1 and 2, the evaluation focusing on the pitch of the voice (evaluation of pitch deviation) is the pitch of the voice (hereinafter, the voice and pitch are “user voice”, respectively). The pitch between the “user pitch” and the pitch of the voice to be uttered at the timing when the voice is uttered (hereinafter, the voice and the pitch are called “reference voice” and “reference pitch”, respectively) A (pitch) difference (difference) was obtained and fixed according to the pitch difference. In the conventional singing scoring device described in Patent Document 1, the average pitch of the portion sandwiched between the peak and the bottom of the pitch fluctuation is obtained so that the scoring result and the evaluation based on the audibility are more consistent, and the utterance period of the reference voice The average pitch that is closest to the reference pitch is selected for scoring. In the conventional singing scoring device described in Patent Document 2, a histogram of pitches detected in a frame is created with a fixed length of about 200 ms as one frame, and the class value of the class that is the maximum frequency is set in the frame. It is adopted as a user pitch.

  The human auditory system is extremely sensitive to pitch differences. However, it is known that the pitch (difference in pitch) is almost proportional to the frequency ratio. Thereby, for example, the difference between the pitches of two sounds at 440 Hz and 880 Hz and the difference between the two sounds at 880 Hz and 1760 Hz are both 1: 2, so that it is recognized that the difference is the same in terms of hearing.

Since humans have the auditory system as described above, the pitch difference that can recognize the pitch shift differs depending on the pitch. This is the result of the evaluation (scoring) according to the pitch in the fixed evaluation according to the difference between the user pitch and the reference pitch as employed in the conventional singing scoring devices described in Patent Documents 1 and 2, respectively. This means that the difference in evaluation on hearing is fluctuating. From this, in order to make the difference from the evaluation based on audibility smaller, in other words, in order to be able to score more suited to human audibility, the evaluation (scoring) result should not vary depending on the pitch. Is also considered important.
Japanese Patent Laid-Open No. 10-26995 Japanese Patent Laid-Open No. 11-224094

  An object of the present invention is to provide a singing scoring device capable of scoring more suited to human hearing regardless of the pitch of the voice to be uttered during singing.

  The singing scoring device of the present invention is based on the premise that the singing of the music by the user is evaluated and scored, and corresponds to the first pitch that is the pitch of the voice signal input by the user, and the voice signal. Using a pitch ratio calculation means for calculating a pitch ratio between second pitches that are pitches of voice to be uttered at the singing position of the music to be played, and a pitch ratio calculated by the pitch ratio calculation means And scoring means for scoring the song by the user.

  The scoring means performs scoring by evaluating the deviation of the pitch from the pitch ratio for each scoring unit, with the predetermined number of pitch ratios calculated by the pitch ratio calculating means as scoring units. Is desirable. In addition, it is desirable to evaluate the scoring unit by excluding the pitch ratio exceeding the predetermined range from the pitch ratio as the scoring unit. It is desirable that the evaluation results of scoring units can be notified to the user sequentially at a predetermined timing.

  The program of this invention is equipped with the function for implement | achieving each means with which the said song scoring apparatus comprises.

  The present invention includes a first pitch that is a pitch of a voice signal input by a user, and a second pitch that is a pitch of a voice to be uttered at a song singing position corresponding to the voice signal. The pitch ratio is calculated, and the song by the user is scored using the pitch ratio.

  In the human auditory system, it is known that the amount of pitch (difference in pitch) is almost proportional to the frequency ratio. From this, by calculating the pitch ratio and using it for singing, it becomes possible to perform more appropriate scoring that matches the characteristics of the auditory system, regardless of the pitch of the voice to be uttered. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram for explaining the configuration of a karaoke system constructed using a karaoke system equipped with a singing scoring device according to the present embodiment.

  As shown in FIG. 1, the karaoke system includes a karaoke system (karaoke device) 11, a microphone 12 for voice input, a plurality of speakers 13 for sound emission, and a display device such as a television or a liquid crystal display device. 14 are connected to each other. As a result, the karaoke system 11 is realized as displaying lyrics on the display device 14 and emitting the voice input from the microphone 12 and the performance for karaoke through the speaker 13.

  FIG. 2 is a configuration diagram of the karaoke system 11. As shown in FIG. 2, a CPU 201 that controls the entire system 11, a ROM 202 that stores programs executed by the CPU 201 and various control data, a RAM 203 that the CPU 201 uses for work, and a microphone 12 that outputs sound by input. An A / D converter (ADC) 204 for converting an analog audio signal into digital data (audio data), a switch (SW) unit 205 having various operators, and a sequencer 206 for reproducing a performance for karaoke Then, a sound source 207 that generates musical tone waveform data in accordance with instructions from the sequencer 206, audio data output from the CPU 201 and waveform data output from the sound source 207 are input, mixed and output as waveform data for sound emission. D / A conversion of mixer 208 and waveform data output by mixer 208 It includes D / A converter for outputting an analog audio signal (DAC), a Te.

  The sequence data for reproducing the performance for karaoke is stored in the ROM 202 or the sequencer 206 in the form of a standard MIDI file (SMF), for example. Here, the following description will be made on the assumption that the SMF is stored in the ROM 202. The ROM 202 also stores text data for displaying lyrics for each SMF (musical piece).

  The switch unit 205 includes, as switches to be operated, a numeric keypad used for music selection and the like, a reproduction start / end switch for instructing the reproduction start / end of karaoke performance, and the like. A plurality of adjustment knobs are provided for performing various adjustments such as mixing level adjustment, volume adjustment, and karaoke performance key (pitch) adjustment. In addition, a detection circuit for detecting a change in the state of these operators is provided, and the detection circuit indicates a switch that has been pressed with a switch, a knob with a changed state with a knob, and a state after the change. The CPU 201 is notified. The CPU 201 performs control for responding to the user's instruction based on the notification.

  The music selection by the numeric keypad is performed, for example, by inputting a music number. When the user inputs the music number, the CPU 201 recognizes the music number by a notification from the switch unit 205, reads SMF (sequence data) assigned with the music number from the ROM 202, and sends it to the sequencer 206. As a result, the sequencer 206 plays back the SMF received from the CPU 201.

  In the SMF, time data indicating the timing for processing the MIDI data indicating the contents of the performance event is added. Therefore, the reproduction is performed by sequentially processing the MIDI data according to the time data. By the processing of the MIDI data, the sound source 207 is instructed from the sequencer 206 to start and end the tone generation by designating the pitch.

  The sequencer 206 notifies the CPU 201 of the pitch (reference pitch: note number) of the musical sound that starts or ends with the MIDI data when processing the MIDI data related to the musical sound corresponding to the melody. By the notification, the CPU 201 recognizes the reference pitch of the voice to be uttered during the reproduction of the sequence data (during karaoke performance) and the period during which the voice is uttered. The MIDI data is stored as data of a predetermined channel so that the MIDI data related to the pronunciation of the musical sound corresponding to the melody can be discriminated. The reference pitch changes depending on the operation of the knob for adjusting the key. However, for the sake of convenience, the change is not assumed here.

  During the reproduction of the sequence data, the CPU 201 detects the pitch (user pitch) of the voice input through the microphone 12 from the voice data input from the ADC 204, and scores the user's song using the detection result. The scoring result is displayed on the display device 14 after the reproduction of the sequence data is completed.

  The ADC 204 samples an audio signal at a predetermined sampling period and outputs audio data. In order to perform pitch detection from the audio data, the CPU 201 stores audio data for a predetermined period in the past in an area secured in the RAM 203. Hereinafter, with reference to FIG. 3 and FIG. 4, reproduction of sequence data and scoring performed by the reproduction will be described in detail.

  FIG. 3 is a flowchart of the reproduction process. The flow of the process executed by the CPU 201 for reproducing sequence data is shown. First, the reproduction process will be described in detail with reference to FIG. The reproduction process is realized by the CPU 201 executing a program stored in the ROM 202.

  First, in step 301, music is selected in accordance with a user operation on the switch unit 205. The song number input by the user for designating the song is specified from the type of key notified from the switch unit 205 that the user has operated and the order thereof. After the user inputs the song number, the process proceeds to step 302 and waits for an instruction to reproduce the sequence data. Since the instruction is made by operating the reproduction start / end switch as described above, the operation proceeds to step 303 by operating the button.

  In step 303, sequence data (SMF) designated by the user is read from the ROM 202 and sent to the sequencer 206 to instruct the start of reproduction. The corresponding text data is read from the ROM 202 and displayed on the display device 14 as lyrics. Thereafter, the process proceeds to step 304.

  By instructing the sequencer 206 to start the reproduction of the sequence data, the karaoke performance is started, and the user starts singing at an appropriate timing. In step 304 and subsequent steps, processing for responding to the song is performed.

  First, in step 304, a scoring data collection process for collecting scoring data for scoring the user's song is executed. In subsequent step 305, a process for causing the display device 14 to display a pitch (pitch) deviation between the voice (user voice) input from the microphone 12 by the user's singing and the voice to be uttered (reference voice). I do. In the next step 306, an operation for correcting the pitch of the user voice to the reference pitch is performed on the voice data in accordance with the setting, and a process of outputting to the mixer 208 is performed. The process proceeds to step 307 after the execution.

  The display of the pitch shift display by the execution of the processing in step 305 is performed, for example, for each voice to be uttered. By notifying the deviation by display, the user can utter the sound of the correct pitch during singing.

  In step 307, it is determined whether or not the reproduction of the sequence data is instructed or the reproduction is finished. If the user instructs the end of playback of the sequence data by operating the playback start / end switch, or if the end of playback is notified from the sequencer 206, the determination is YES and the scoring result (score) is displayed in step 308. After being displayed on the device 14, the series of processing is terminated. When the user has instructed the end of reproduction of the sequence data, in step 308, the instruction to end the reproduction to the sequencer 206 is also performed. Otherwise, the determination is no and the process returns to step 304 above. Thereby, the processing loop formed in steps 304 to 307 is repeatedly executed until the reproduction of the sequence data is completed. The execution cycle is a sampling cycle.

FIG. 4 is a flowchart of the scoring data collection process executed as step 304 above. Next, the collection process will be described in detail with reference to FIG.
First, in step 401, the pitch of the voice input in the form of voice data via the ADC 204 is detected, and the pitch (user pitch) and the pitch of the voice to be uttered at this time point notified from the sequencer 206 ( The pitch ratio to the reference pitch is calculated. In this embodiment, a value obtained by dividing the reference pitch by the user pitch is calculated as the pitch ratio. After the calculation, the pitch ratio is substituted for the variable p_ratio in step 402, and further, a value obtained by subtracting 1 from the value of the variable p_ratio (= p_ratio−1.0) is substituted for the variable diff in step 403. Go to 404.

The method for detecting the user pitch is not particularly limited, but in this embodiment, the applicant adopts the method described in the specification attached to the application of Japanese Patent Application No. 2005-54481. By adopting this method, the pitch ratio is calculated according to the following flow.
(1) First, audio data of a predetermined size is extracted as a frame from an area secured in the RAM 203 and converted into a frequency domain by DFT (Discrete Fourier Transform). (2) Each frequency channel is converted into a frequency domain. From the obtained frequency components, two or more frequency channels in which harmonics or fundamentals exist are found. If such a channel is not found, it is determined that no sound is input or the input sound is an unvoiced sound, and the pitch ratio is not calculated. (3) Two or more frequency channels in (2) Is found, one of the two or more found channels is used as the reference channel. (4) The greatest common divisor of the frequency corresponding to the channel index of the two or more found frequency channels is calculated (5). It is determined how many harmonics the reference channel corresponds to from the calculated greatest common divisor. (6) Multiple determined by (5) the inter-frame phase difference in the frequency channel corresponding to the reference pitch notified from the sequencer 206. (7) The ratio of the phase difference between the frames of the reference channel to the target phase difference calculated in (6) is calculated. The ratio calculated in (7) is a pitch ratio obtained by dividing the reference pitch by the user pitch. For this reason, the pitch ratio becomes 1 when the user utters a sound having a correct pitch. The value of the variable diff is positive if the voice pitch is lower than the reference pitch, and negative if the voice pitch is higher than the reference pitch. When the pitch ratio is calculated as described above, it is possible to reliably calculate the pitch ratio even for a musical sound in which a fundamental frequency called missing fundamental is missing or very small compared to other frequencies.

  In step 404, it is determined whether or not the absolute value of the variable diff is greater than a predetermined constant OUTER_THRESH. If the absolute value is less than or equal to the constant OUTER_THRESH, the determination is no and the process moves to step 405. Otherwise, the determination is yes, and the series of processing ends here.

  An error may occur in the pitch ratio calculation (pitch detection). Therefore, in the present embodiment, if the pitch ratio is out of the range specified by the constant OUTER_THRESH by the determination process in step 404, it is considered that an error has occurred in the pitch ratio calculation, and scoring is performed. Excluded in doing. When two or more frequency channels with harmonics or fundamentals are not found, or when there is no voice to be uttered, although not particularly shown, a series of processing is performed after executing the processing of step 401. It is supposed to end. In this way, the evaluation can be performed more appropriately by excluding the pitch ratio and the period that should not be taken into consideration in the evaluation. The constant OUTER_THRESH is determined using, for example, a pitch of 5 degrees as a guide.

  The actual situation is that pitch fluctuation often occurs in the generated voice. For this reason, in consideration of the fluctuations, scoring data is collected for each scoring unit with a certain number of data as scoring units. Here, the scoring unit is referred to as a “scoring frame”. The variables accum, data_counter, and avg are variables prepared for performing an evaluation focusing on the pitch for each scoring frame. The fixed number of data is prepared as a constant frame_num.

  In step 405, the result (= accum + diff) obtained by adding the value of the variable diff to the previous value is substituted for the variable accum. In the next step 406, the value of the variable data_counter is incremented. In the next step 407, it is determined whether or not the value of the variable data_counter is equal to the constant frame_num. The value of the variable data_counter corresponds to the number of times that the pitch ratio that is valid in the current scoring frame is calculated. For this reason, when the effective pitch ratio for the scoring frame is calculated, the determination is yes and the process proceeds to step 408. Otherwise, the determination is no and the series of processing ends here.

  A determination of YES in step 407 means that there is no pitch ratio to be calculated in the scoring frame. For this reason, in step 408 and subsequent steps, an evaluation focusing on the pitch is performed on the scoring frame, and a process for storing the evaluation result is performed.

  First, in step 408, the value of the variable frame_counter for counting the number of scoring frames to be processed is incremented. In the subsequent step 409, a value obtained by dividing the value of the variable accum by the constant frame_num is substituted for the variable avg. After the substitution, the routine proceeds to step 410, where it is determined whether or not the absolute value of the variable avg is smaller than a predetermined constant PASS_THRESH. The constant PASS_THRESH is determined as a value indicating a pitch ratio range in which it is considered that no pitch deviation has occurred. The value of the variable avg is a value that relatively indicates the average value of the pitch ratio in the scoring frame, and the constant PASS_THRESH is determined assuming the value of the variable avg. From this, when the average value is within the range, the absolute value of the variable avg is equal to or smaller than the constant PASS_THRESH, that is, the determination is YES. Then, after incrementing the value of the variable pass_counter in step 411, step 412 is performed. Migrate to Otherwise, the determination is no and the process of step 412 is then executed.

  The human auditory system is extremely sensitive to pitch differences. As is well known, there is an individual difference in the minimum frequency difference (frequency discrimination threshold) at which the difference in pitch between two consecutive sounds can be heard, but even a normal person is said to be about 0.2%. ing. This value is 0.1% or less for an expert with a sharp ear. The constant PASS_THRESH is a value determined in consideration of such a situation, for example.

  In step 412, 0 is substituted into the variable accum. In the subsequent step 413, 0 is substituted into the variable data_counter. In this way, after initializing variables in order to evaluate the next scoring frame, a series of processing is terminated.

  The generation of audio data to be output to the mixer 208 is performed using an area separately secured in the RAM 203. In step 306 of FIG. 3, pitch scaling is performed according to the pitch ratio substituted in the variable p_ratio in step 402, and the audio data after the scaling is overlapped with the audio data stored in the area. The mixer 208 outputs audio data for one sampling after the overlap addition.

  In step 308 of FIG. 3, the score is calculated and displayed using the values of the variables pass_counter and frame_counter. If the score is expressed as grade, it is calculated by the following equation, for example.

grade = pass_counter / frame_counter × 100
In the human auditory system, the amount of pitch (difference in pitch) is known to be approximately proportional to the frequency ratio. Focusing on this, the user's singing is evaluated and scored using the pitch ratio between the user pitch and the reference pitch. For this reason, scoring suitable for human hearing can be performed irrespective of the tendency of the pitch of the voice to be uttered in the music (the number of voices to be uttered for each pitch). Thereby, the scoring result (score) can be evaluated as appropriate for more people.

  In the present embodiment, the singing is evaluated and scored by paying attention only to the pitch, but the utterance timing of voice (at least one timing of utterance start and end, or utterance period, etc.) You may make it perform the evaluation which paid attention to. A plurality of different constants PASS_THRESH may be selected so that scoring according to the user level can be performed. It is desirable that the selection (or setting) can be arbitrarily performed by the user. The pitch ratio calculation method and the scoring method using the pitch ratio are not limited to the present embodiment, and various modifications may be made. Audio input may be performed via a network or may be performed via a recording medium such as a CD. In the reference pitch, the pitch of a musical sound generated by a part other than the melody may be used as the reference pitch.

  All or a part of the singing scoring apparatus as described above or a program for realizing the modification may be stored in a recording medium and distributed. Or you may make it distribute via the communication medium which comprises a network. In such a case, the user can apply the present invention to the device by acquiring the program and loading it into the data processing device (computer).

It is a figure explaining the structure of the system for karaoke constructed | assembled using the karaoke system carrying the singing scoring apparatus by this Embodiment. It is a lineblock diagram of a karaoke system carrying a song scoring device by this embodiment. It is a flowchart of a reproduction process. It is a flowchart of scoring data collection processing.

Explanation of symbols

1 CPU
5 Sound Generator 5-1 Integer Address Calculation Block 5-2 Decimal Address Calculation Block 5-3 ENV Generation Block 5-4 Waveform ROM Data Latch Block 5-5 Multiply and Accumulate Block 5-6 Accumulation Block 6 Waveform ROM
501 to 504, 509 to 511, 521 to 523 Register 505, 506, 516, 517 Multiplexer 507 Adder / Subtractor 514 Right 2-bit shifter 515 Operation unit 519 Multiplier 520 Adder

Claims (5)

In the singing scoring device that evaluates and scores the singing of music by the user,
Between the first pitch that is the pitch of the voice signal input by the user and the second pitch that is the pitch of the voice to be uttered at the singing position of the music corresponding to the voice signal. A pitch ratio calculating means for calculating a pitch ratio;
Using the pitch ratio calculated by the pitch ratio calculation means, scoring means for scoring the song by the user;
A singing scoring device comprising: The scoring means performs the scoring by evaluating a deviation in pitch from the pitch ratio for each scoring unit, with the predetermined number of pitch ratios calculated by the pitch ratio calculating means as scoring units.
The singing scoring device according to claim 1. The scoring means performs evaluation of the scoring unit by excluding a pitch ratio that exceeds a predetermined range from the pitch ratio as the scoring unit.
The singing scoring device according to claim 2. The scoring means can sequentially notify the user of the evaluation result of the scoring unit at a predetermined timing.
The singing scoring device according to claim 2 or 3, wherein It is a program that is executed by a singing scoring device that evaluates and scores a song of a song by a user,
Between the first pitch that is the pitch of the voice signal input by the user and the second pitch that is the pitch of the voice to be uttered at the singing position of the music corresponding to the voice signal. A function to calculate the pitch ratio;
Using the pitch ratio calculated by the function to calculate, the function of scoring the song by the user,
A program to realize