JP4204941B2 - Karaoke equipment - Google Patents

Description

  The present invention relates to an improvement in scoring accuracy in a karaoke apparatus having a singing scoring function.

  Conventionally, some karaoke apparatuses have a scoring function for scoring the skill of a singer. The scoring function that has been practically used in the past compares the pitch extracted from the reference such as a guide melody with the frequency extracted from the singing voice (for example, Patent Document 1), and adds the evaluation of the volume change to this. (Patent Document 2).

Japanese Patent Laid-Open No. 10-49183 JP-A-10-161673

  However, since the guide melody is MIDI data in which notes (note-on event data and note-off event data) are mechanically arranged, the pitch changes mechanically at an accurate beat timing. Therefore, if you sing exactly according to the so-called score, you can get a high score with the scoring function, but such singing is never good. On the other hand, when singing with a singing technique such as “for” or “rubbert”, the length of the note is freely changed after accompaniment (karaoke performance), the above mechanically accurate notes Since the length of the song and the singing do not match, there was a problem that the score was rather low.

  An object of the present invention is to provide a karaoke apparatus with a scoring function that can accurately determine the skill of actual singing.

According to the first aspect of the present invention, performance means for playing karaoke music and supplying reference data including pitch information and delimiter information of each note of the singing melody in synchronization with the performance of the karaoke music, and sampling input of the singing voice A singing voice input means that compares the singing voice and the reference data for each note, and, based on the frequency difference, determines the pass / fail for each note , and performs scoring The singing voice and reference data of a predetermined section including the scoring target note are shifted in the time axis direction to determine the position where the cross-correlation is maximum, and the singing voice of the scoring target note and the reference data are determined at that position. and a scoring unit for comparing, said scoring means, 3 degrees, 5 degrees, or the song St. number of pitch difference of pitch difference chord constituent notes of the song being played It makes a comparison between the reference data by adding or subtracting a frequency of the audio, when the resulting determination pass either has a feature to make a scoring by the decision of acceptance for the scoring target note To do.

  By shifting the singing voice and the reference data in the time axis direction, even if the timing of the singing and the timing of the reference are shifted, they can be matched by taking the cross-correlation, and can be scored at the matching position. Even when singing with “for” or “rubbert”, the singing timing can be compared and graded according to the reference.

  The invention of claim 2 is characterized in that the scoring target note and a note immediately after the scoring target note are used as the predetermined section.

  In the present invention, not only the note (note) to be compared but also the section including the immediately following note are mutually shifted to detect the position at which the frequency difference between the singing voice and the reference is minimized, thereby making the sound change timing more accurate. It is possible to detect singing of “for” and “rubbert” more accurately.

  According to a third aspect of the present invention, there is provided filter means for performing low-pass filtering on one or both of the pitch information of the singing melody and the inputted singing voice.

  In the present invention, one or both of the pitch information of the singing melody and the singing voice are low-pass filtered. By subjecting the pitch information of the singing melody to LPF processing, the pitch information of the singing melody becomes discontinuous and the mechanical pitch change becomes gentle, and the pitch change of the actual singer can be approached. Further, by subjecting the singing voice to LPF processing, fine frequency fluctuations such as vibrato can be removed, and it is possible to prevent the skillful singing from becoming a low evaluation.

  As described above, according to the present invention, even when a singer sings using the technique of “for” or “rubbert”, it is possible to accurately score based on reference data such as guide melody data.

A karaoke apparatus according to an embodiment of the present invention will be described with reference to the drawings.
In a karaoke device, the performance of a karaoke song is the operation of outputting the background video / lyric telop to the monitor while generating the music of the karaoke song. The singing voice is scored by comparing it with a reference to calculate a score, and a scoring operation is performed to display the score after the end of the song.

  FIG. 1 is a block diagram of the karaoke apparatus. The karaoke apparatus is composed of a CPU 10 that controls the operation of the entire apparatus and various devices connected thereto. Connected to the CPU 10 are a hard disk 11, a RAM 12, a sound source 13, a mixer (effector) 14, a vocal adapter 19, an MPEG decoder 20, a synthesis circuit 21, and an operation unit 23. The hard disk 11 stores music data for playing karaoke music, video data for displaying a background video on a monitor, and the like. In the RAM 12, an area for reading out programs and music data, a scoring log area for recording scoring results and the like in the scoring mode are set.

  The sound source 13 forms a musical sound signal in accordance with music data (note event data or the like) input by the music sequencer 31 executed by the CPU 10. The formed tone signal is input to the mixer 14. The mixer 14 gives an effect such as echo to the plurality of musical sound signals generated by the sound source 13 and the singing voice signal of the singer input via the microphone 17 -A / D converter 18. Mix the signal with an appropriate balance. The mixed digital audio signal is input to the sound system 15. The sound system 15 includes a D / A converter and a power amplifier. The input digital signal is converted into an analog signal, amplified, and emitted from the speaker 16. The effect of the mixer 14 on each audio signal and the balance of mixing are controlled by the CPU 10.

  The singing voice signal converted into a digital signal by the A / D converter 18 is also input to the vocal adapter 19. The vocal adapter 19 calculates the singing frequency from the input singing voice signal and also calculates the reference frequency input from the music sequencer 31 of the CPU 10. And this singing frequency and a reference frequency are synchronized and it inputs into CPU10 (scoring mode process 34) every 30 ms. As the reference, guide melody data included in the song data is used. The determined frequency is expressed as a cent value from C0.

  The background video data 41 stored in the HDD 11 is encoded in the MPEG2 format, and the background video reproduction program 33 executed by the CPU 10 reads it and inputs it to the MPEG decoder 20. The MPEG decoder 20 converts the input MPEG data into an NTSC video signal and inputs it to the synthesis circuit 21. The synthesizing circuit 21 is a circuit that synthesizes an OSD such as a lyrics telop or a scoring result display on the video signal of the background video. The synthesized video signal is displayed on the monitor display 22.

  The operation unit 23 includes a panel switch interface, a remote control receiving circuit, and the like, and inputs an operation signal to the CPU 10 according to the operation of the panel switch and the remote control device by the user. The CPU 10 detects this operation signal by the operation input processing program 35 and executes a corresponding process. The operation input processing program 35 is included in the system program.

  The panel switch and the remote control device are provided with various key switches for selecting a song number and selecting a mode such as a scoring mode.

  When a song number is input with a panel switch or a remote control device, the operation input processing program 35 detects this and transmits it to the sequencer 30 as a request for a karaoke song. In response, the sequencer 30 reads the song data of the karaoke song identified by this song number from the song data storage area 40 of the hard disk 11. The sequencer 30 includes a song sequencer 31 and a lyrics sequencer 32, and the lyrics sequencer 32 includes a character pattern creation program 32a. The music sequencer 31 reads the data of tracks such as performance data tracks and guide melody tracks in the music data, and controls the sound source 13 with this data to generate performance sounds of karaoke music. The lyrics sequencer 32 reads the data of the lyrics track in the song data, creates a lyrics telop image pattern based on the data, and outputs the image pattern to the synthesis circuit 21. Further, the background video reproduction program 33 reads predetermined background video data in accordance with an instruction from the sequencer 30 and inputs it to the MPEG decoder 20.

  Here, the music data stored in the hard disk 11 will be described with reference to FIG. As shown in FIG. 5A, the song data includes a musical sound track for playing a karaoke song, a guide melody track for generating a guide melody, a lyrics track for displaying a lyrics telop, an accompaniment chord of a song ( It consists of a chord track etc. The song data has a header, audio data, audio data control track, etc. in addition to this, but it is omitted in this figure for the sake of simplicity.

  Each track is described according to the MIDI format. For example, the guide melody track includes event data such as note-on event data and note-off event data and timing data indicating the read timing of each event data, as shown in FIG. Note-on event data includes pitch data, and designates the pitch of a musical tone (guide melody) generated by note-on. This musical tone continues until the next note-off event data is read out.

  The timing data can be composed of duration data indicating the time interval between the event data, absolute time data indicating the absolute time from the start time of the music, and the like.

  The event data of the musical tone track and the guide melody track is composed of the note event data indicating the pitch, volume, on / off, etc. of the musical tone as described above, and by inputting this note event data to the sound source 13, the sound source Reference numeral 13 sounds or mutes the musical sound corresponding to the event data. The musical sound track is composed of a plurality of tracks (parts) for generating musical sounds of a large number of musical instruments, and the guide melody track is composed of single melody MIDI data for guiding the singing melody. The chord track is a track for designating accompaniment chords (C, G7, etc.) in synchronization with the performance track and the guide melody track, and is described as a system exclusive message.

  On the other hand, the event data of the lyrics track is sequence data in which the lyrics telop of the karaoke song is implemented by system exclusive data, and has event data different from the musical tone track and the guide melody track. The event data is page break data, lyrics display data, and the like.

  In the performance of karaoke music in the normal mode, the sequencer 30 generates the karaoke performance sound and displays the lyrics telop as described above. In the scoring mode, the scoring process is performed by the scoring mode processing program 34 in addition to this. The action is executed.

  First, with reference to the functional block of FIG. 3, the process of each part at the time of scoring mode is demonstrated. The singing voice signal input from the microphone 17 is converted into a digital voice signal by the A / D converter 18 and input to the vocal adapter 19 (at the same time input to the mixer 14, but only the operation in the scoring mode is described here. To do). In the vocal adapter 19, this digital audio signal is input to the singing frequency detector 102 to detect the singing frequency (cent value).

  On the other hand, reference data is input from the song sequencer 31 to the reference frequency detector 101 in synchronization with the performance of the karaoke song. As described above, guide melody data is used as the reference data. The reference frequency detection unit 101 extracts pitch information from note-on event data of the input MIDI data, and outputs the pitch cent value as a reference frequency.

  The detection of the singing frequency by the singing voice detection unit 102 and the detection of the reference frequency by the reference frequency detection unit 101 are performed in synchronization every 30 ms, and the detection result is input to the scoring mode processing program 34 every 30 ms.

  Further, when note-on / off event data is input from the music sequencer 31, the reference frequency detection unit 101 notifies the scoring mode processing program 34 of note-on information and note-off information at the timing.

  In the scoring mode processing program 34, low-pass filter (LPF) processing (105, 106) is performed on the input singing frequency and reference frequency. The LPF process for the reference frequency is a process for smoothening the pitch change of the reference (see FIG. 4A), which is a mechanical pitch sequence, and bringing it closer to a human song. Further, the LPF processing for the singing frequency is processing for obtaining flat singing frequency information by removing techniques such as vibrato.

  FIG. 4A shows an example of guide melody data used as a reference. The reference data is mechanical data in which the pitch changes discontinuously at an accurate beat timing even in a legato section in which notes are continuous. By performing LPF processing on such a discontinuous reference, the pitch gradually changes between notes as shown in FIG. It is possible to make the pitch change similar to that of singing. Note that rest sections where notes are interrupted, places where singing with non-legato, etc. are excluded from the LPF processing. As a result, it is possible to prevent the LPF processing from becoming unnatural due to the data of the section without sound.

  FIG. 6C is a diagram showing an example of singing voice frequency data. The singing voice frequency has a gentle transition (so-called “scribbling”) at the transition of notes (pitch), and has a periodic frequency change such as vibrato in the stretched part of the sound. . By performing LPF processing on this singing voice frequency data, it is possible to remove fine frequency changes such as overshoot and vibrato in the squeaky part, as shown in Fig. 4 (D), and accurately extract the frequency that was sung. Will be able to.

  The audio signal input from the microphone 17 includes various noises as well as the singing audio signal. When the level of the noise component is large, the frequency detection unit 102 may detect the frequency by regarding the noise component as a singing voice signal. If such a noise component is input to the LPF processing unit 106, not only the one sample but also erroneous data will be output after that. Therefore, if there is a sudden pitch change of 150 cents or more, which is difficult to consider as a frequency change of the singing voice, ignore the data (adopt the previous sample data again) and perform LPF processing. , It can prevent adverse effects due to noise.

  Since the data sequence of the singing frequency and the reference frequency is discrete data every 30 ms, the LPF processing unit 106 for the singing frequency uses a secondary filter with a cut-off frequency of 5.5 Hz in order to achieve the above processing appropriately. The LPF processing unit 105 for the reference frequency uses a secondary filter having a cutoff frequency of 5 Hz.

  In this embodiment, LPF processing is performed on both the singing frequency and the reference frequency, but this processing is not essential. Moreover, even if it carries out only with respect to either one, the said each effect can be acquired.

  The singing frequency and the reference frequency that have been subjected to the LPF process are input to the scoring unit 107. The scoring unit 107 compares the singing frequency with the reference frequency, calculates a difference (cent value), and determines a pass note and a fail note for each note (note) based on the difference. Since the note-on information / note-off information is input from the reference frequency detection unit 101 to the scoring unit 107, the singing frequency matches the pitch of the singing melody for a predetermined (one or more) samples ( If the number of times the singing frequency matches the pitch of the singing melody is less than the above-mentioned predetermined sample, the “failed note” is determined. It is determined that

  In the above comparison, instead of simply comparing the singing frequency and the reference frequency at the same timing, the reference frequency is moved back and forth on the time axis, and the corresponding singing voice is at a position where the cross-correlation between both sample sequences is maximum. The frequency is compared with the reference frequency. Cross correlation is

  However, it is also possible to take the difference between samples corresponding to each other when shifted and to set the position where the integrated value of the difference is minimum as the maximum correlation point.

  FIG. 5 shows an example of processing for detecting the point where the cross-correlation is maximized by shifting the reference data back and forth. In this example, the cross-correlation of two notes of the pass / fail decision and the next note is taken, and the pass / fail decision is made for the previous note at the position where the cross-correlation is maximized. Yes. By taking the cross-correlation for a plurality of notes in this way, the transition of the sound becomes clear, and it becomes easy to associate the singing voice with the reference note.

  FIG. 5A shows a sample data sequence of reference frequency stored in the reference memory and a sample data sequence of singing voice frequency stored in the singing voice memory. When this is compared at the same timing, it becomes as shown in FIG. From this state, the reference data is shifted in the delay direction as shown in FIGS. This is the direction in which the correlation increases when sung late. Conversely, as shown in FIG. 5E, the reference data is shifted in the advance direction. This is the direction in which the correlation increases when the song is sung. A predetermined amount is shifted by one sample or several samples in both directions, and the cross-correlation is calculated for each shift. Then, pass / fail is determined at the position where the cross-correlation is the highest. In this example, since the cross-correlation is largest at the time of FIG. 4D, the reference frequency corresponding to the previous note is compared with the singing frequency at this position.

  In the pass / fail judgment process by detecting and comparing the cross-correlation between the singing voice and the reference, it is a matter of course that the singing voice frequency is directly compared with the reference frequency, but when the frequency difference is larger than 200 cents. May be arranging, ad-lib or harmony singing, so from the singing frequency octave (± 1200 cents), 3 degrees (± 400 cents, ± 300 cents), 5 degrees (± 700 cents) or The above comparison is performed by adding or subtracting the number of cents of the pitch (pitch difference) of the constituent sounds of the chord of the song at that time. If it is determined that the pass is acceptable, it is determined that the song is arranged, ad-lib or harmony is being performed.

  When the music is finished, the scoring unit 107 sums up the final score. The final score is calculated by dividing the number of accepted notes by the total number of notes and adding 50 to the value obtained by multiplying it by 50 to obtain a maximum score of 100 points. That is, in consideration of the entertainment function in the karaoke apparatus, 50 points are given at a minimum.

  In the final score calculation, only the passing notes are targeted for scoring, but the number of rejected notes and their ranks may be subject to scoring.

The processing of the scoring mode processing program 34 will be described with reference to the flowchart.
FIG. 6 is a flowchart showing a main routine of scoring processing operation. In this operation, input from the vocal adapter 19 is monitored, and processing corresponding to the input data is executed. The input buffer is checked every 30 ms to capture the singing frequency and the reference frequency and confirm whether note-on / off information is input. When the singing frequency and the reference frequency are taken in (s1), LPF processing is executed for both of them (s5, s7). Then, the LPF processed reference data is written in the reference list of the list memory (see FIG. 7) (s6), and the LPF processed singing voice frequency data is written in the singing voice list of the list memory (s8). As described above, only one of the LPF processes may be performed.

  When note-on information is sent from the reference frequency detector 101 (s2), this note-on information is written into the control data list of the list memory (s10). Thus, it can be seen that the sample data written after this is sample data after the note-on. In this operation, the sample data is stored in the memory even during the note-off period, but the sample data may not be stored during the note-off period. Further, when the pitch is changed in the MIDI running status, note-on information and note-off information may be written continuously.

  Here, the list memory is a memory area set in the hard disk 11 or the RAM 12 and, as shown in FIG. 7, a predetermined amount of reference frequency, singing frequency sample data and note-on / note-off event information are sequentially stored in the order of input. It is set to be.

  When note-off information is sent from the reference frequency detection unit 101 (s3), the note-off information is written to the reference memory (and the singing voice memory) (s11), and the note that was note-off this time and the immediately preceding note are written. The cross-correlation of the two notes of the note is taken, and the above-described pass / fail determination process is performed on the immediately preceding note at the position where the correlation is maximum (s12).

  On the other hand, when a music end message is sent from the music sequencer 31 or the reference frequency detection unit 101 (s4), the score totaling processing operation is executed (s14).

  FIG. 8 is a flowchart showing the score totaling process. This operation is executed in s14 of the main routine. First, the total number of notes and the number of accepted notes are totaled (s25). Next, the number of accepted notes is divided by the total number of notes, this point is multiplied by 50, and then 50 is added to calculate the final score (s26). The calculated final score is displayed on the monitor 22 (s27).

  In this embodiment, the cross-correlation between the note (target note) for determining pass / fail and the next note of the next note is detected, but the interval for detecting the cross-correlation is not limited to this. The cross-correlation may be detected only for the target note, or the cross-correlation may be detected for the entire song or phrase unit.

  In addition, when the cross-correlation between the target note and the next note is detected and the pass / fail of the target note is determined, there is no next note for the last note of the song or phrase. For this note, the cross-correlation may be detected only by the target note.

The block diagram of the karaoke apparatus which is embodiment of this invention The figure which shows the structural example of the song data used with the karaoke apparatus The figure which shows the functional block of the scoring process of the karaoke apparatus The figure explaining LPF processing in the karaoke apparatus The figure explaining the system which calculates | requires the pass / fail by calculating | requiring the cross correlation of the karaoke apparatus The flowchart which shows the main routine of the scoring part of the karaoke apparatus The figure which shows the list memory of the same karaoke device The flowchart which shows the score totaling process of the same karaoke device

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... CPU, 11 ... Hard disk, 12 ... RAM, 13 ... Sound source, 14 ... Mixer, 15 ... Sound system, 16 ... Speaker, 17 ... Microphone, 18 ... A / D converter, 19 ... Vocal adapter, 20 ... MPEG decoder, 21 ... Synthesis circuit, 22 ... Monitor, 23 ... Operation part,
30 ... Sequencer, 31 ... Song sequencer, 32 ... Lyric sequencer, 32a ... Character pattern creation program, 33 ... Background video reproduction program, 34 ... Scoring mode processing program, 35 ... Operation input processing program,
40 ... song data storage area, 41 ... background video storage area, 43 ... scoring log,
DESCRIPTION OF SYMBOLS 101 ... Reference frequency detection part, 102 ... Singing frequency detection part, 105, 106 ... Low pass filter process part, 107 ... Scoring part

Claims (3)

A performance means for playing karaoke songs and supplying reference data including pitch information and break information of each note of the singing melody in synchronization with the performance of the karaoke songs,
Singing voice input means for sampling and inputting singing voice;
The singing voice is compared with the reference data for each note, and based on the frequency difference, a pass or fail is determined for each note , and a scoring unit for scoring, which includes a scoring target note The singing voice of the section and the reference data are shifted relative to each other in the time axis direction to determine the position where the cross-correlation is maximum, and the scoring means for comparing the singing voice of the scoring target note with the reference data at the position,
And the scoring means adds or subtracts the cent number of the pitch difference of the chord component sound of the song being played three times, five times, or the performance of the singing voice to the reference data and A karaoke apparatus characterized in that if a comparison is obtained and a pass determination is obtained, the pass is determined for the target musical score and the scoring is performed.   The karaoke apparatus according to claim 1, wherein the predetermined section is the scoring target note and a note immediately after the scoring target note.   3. The karaoke apparatus according to claim 1, further comprising a filter unit that performs low-pass filtering on one or both of the pitch information of the singing melody and the input singing voice.

Images