JP2008015388A - Singing skill evaluation method and karaoke machine - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To score only a singing part by judging whether a sound input from a microphone is accompaniment information or human singing voice. <P>SOLUTION: A score processing part 12 incorporated in a Karaoke machine 1 is provided with an accompaniment/singing voice discrimination part 21 for judging whether a sound signal input from the microphone 2 is a Karaoke accompaniment or human singing voice. The accompaniment/singing voice discrimination part 21 calculates a spectrum by analyzing the frequency of the sound signal input from the microphone 2 and decides that the input sound signal is accompaniment information when the concentration of energy of the spectrum into a high frequency band is detected. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a singing ability evaluation method and a karaoke apparatus.

Almost all karaoke devices currently on the market have a karaoke scoring function. As a conventional karaoke scoring device, for example, a pitch (pitch) extracted from a singing voice of a singer input from a microphone as shown in Patent Document 1 is compared with a pitch of a guide melody, and a score is calculated based on the degree of coincidence. Is known.
Japanese Patent No. 2925759

  In these karaoke scoring devices, the pitch is detected from the singing voice of the singer input from the microphone, and the singer's song is scored based on the pitch. However, the singer's singing voice is not always input to the microphone. For example, in the case of a karaoke apparatus installed in a karaoke BOX or the like, an audio signal in which a singer's singing voice and accompaniment information are mixed is often input to the microphone. When the singer sings away from the microphone, when the singer's voice is low, or when the accompaniment volume is high, the karaoke accompaniment wraps around the singer's microphone and affects the karaoke scoring results (See FIG. 13).

  The present invention has been made to solve the above problem, and a singing ability evaluation method and singing ability capable of determining whether a voice signal input from a voice signal input means is a singer's singing voice information or accompaniment information. An object is to provide a karaoke apparatus having an evaluation function.

  In order to achieve the above object, a singing ability evaluation method according to claim 1 of the present invention is a spectrum calculation step for causing a computer to perform frequency analysis on an input voice signal input from voice signal input means and calculate a spectrum; A spectrum storage step for storing the spectrum obtained by executing the spectrum calculation step in the spectrum storage means, and detecting that the energy of the spectrum read from the spectrum storage means is concentrated in a high frequency band. And an accompaniment / singing voice determination step for determining that the input audio signal is accompaniment information.

  Further, in the singing ability evaluation method according to claim 2 of the present invention, in addition to the configuration of the invention according to claim 1, in the accompaniment / singing voice determination step, the pitch is calculated from the spectrum read from the spectrum storage means. When the pitch is higher than a certain threshold value, it is determined that the input audio signal is accompaniment information.

  Further, in the singing ability evaluation method according to claim 3 of the present invention, in addition to the configuration of the invention according to claim 1, in the accompaniment / singing voice determination step, the spectrum is calculated from the spectrum read from the spectrum storage means. An inclination is calculated, and when the calculated spectrum inclination is larger than a certain threshold value, it is determined that the input sound signal is accompaniment information.

  Moreover, the singing ability evaluation method according to claim 4 of the present invention is the singing ability evaluation method according to any one of claims 1 to 3, wherein the accompaniment / singing voice determination step further includes the spectrum storage means. From the read spectrum, the amount of harmonics included in the spectrum is measured, and when the amount of harmonics exceeds a certain threshold, it is determined that the input audio signal is accompaniment information.

  Further, the singing ability evaluation method according to claim 5 of the present invention is the singing ability evaluation method according to any one of claims 1 to 3, wherein the accompaniment / singing voice determination step further includes the spectrum storage means. The number of extreme values included in the spectrum is counted from the read spectrum, and when the number of extreme values is greater than a certain threshold value, it is determined that the input audio signal is accompaniment information.

  Further, a karaoke apparatus equipped with the singing ability evaluation function according to claim 6 of the present invention includes a spectrum calculation means for performing frequency analysis on an input voice signal input from the voice signal input means and calculating a spectrum, and the spectrum calculation. The spectrum storage means for storing the spectrum obtained by the means, and the input audio signal is accompaniment information when it is detected that the energy of the spectrum read by the spectrum storage means is concentrated in the high frequency band And accompaniment / singing voice judging means.

  Further, in the karaoke apparatus equipped with the singing ability evaluation function according to claim 7 of the present invention, in addition to the configuration of the invention according to claim 6, the accompaniment / singing voice determination means is read by the spectrum storage means. The pitch is calculated from the obtained spectrum, and when the pitch is higher than a certain threshold value, it is determined that the input voice signal is accompaniment information.

  Further, in the karaoke apparatus equipped with the singing ability evaluation function according to claim 8 of the present invention, in addition to the configuration of the invention according to claim 6, the accompaniment / singing voice determination means is read by the spectrum storage means. The inclination of the spectrum is calculated from the obtained spectrum, and when the calculated inclination of the spectrum is larger than a certain threshold value, it is determined that the input voice signal is accompaniment information.

  Moreover, the karaoke apparatus equipped with the singing ability evaluation function according to claim 9 of the present invention is the karaoke apparatus according to any one of claims 6 to 8, wherein the accompaniment / singing voice determination means further includes the spectrum. The amount of overtones included in the spectrum is measured from the spectrum read out by the storage means, and when the amount of overtones exceeds a certain threshold, it is determined that the input sound signal is accompaniment information. .

  Moreover, the karaoke apparatus equipped with the singing ability evaluation function according to claim 10 of the present invention is the karaoke apparatus according to any one of claims 6 to 8, wherein the accompaniment / singing voice determination means further includes the spectrum. The number of extreme values included in the spectrum is counted from the spectrum read out by the storage means, and the input audio signal is determined to be accompaniment information when the number of extreme values is greater than a certain threshold value. To do.

  The singing ability evaluation method according to claim 1 of the present invention can determine whether the input voice signal is accompaniment information or singing voice information by using the frequency characteristic of the spectrum calculated from the input voice signal. According to the present invention, since accompaniment information input from a microphone or the like can be prevented from being erroneously scored as singing voice information, highly accurate singing ability evaluation can be performed.

  The singing ability evaluation method according to claim 2 of the present invention can determine whether the input voice signal is accompaniment information or singing voice information using the pitch calculated from the input voice signal. According to the present invention, since accompaniment information input from a microphone or the like can be prevented from being erroneously scored as singing voice information, highly accurate singing ability evaluation can be performed.

  Moreover, the singing ability evaluation method according to claim 3 of the present invention can determine whether the input voice signal is accompaniment information or singing voice information by using the inclination of the spectrum calculated from the input voice signal. According to the present invention, since accompaniment information input from a microphone or the like can be prevented from being erroneously scored as singing voice information, highly accurate singing ability evaluation can be performed.

  Further, the singing ability evaluation method according to claim 4 of the present invention, in addition to the effect of the invention according to any one of claims 1 to 3, further determines the accompaniment / singing voice using the amount of overtones included in the spectrum. By performing this, it becomes possible to evaluate the singing ability with higher accuracy.

  Further, the singing ability evaluation method according to claim 5 of the present invention, in addition to the effect of the invention according to any one of claims 1 to 3, further performs accompaniment / singing voice determination using the number of extreme points of the spectrum. By performing, more accurate singing ability evaluation becomes possible.

  Moreover, the karaoke apparatus equipped with the singing ability evaluation function according to claim 6 of the present invention uses the frequency characteristic of the spectrum calculated from the input voice signal to determine whether the input voice signal is accompaniment information or singing voice information. Can be determined. In karaoke BOX and the like, it is possible to prevent a problem that scoring accompaniment information that has entered the microphone as singing voice information, thereby providing a highly accurate singing ability evaluation function.

  A karaoke apparatus equipped with the singing ability evaluation function according to claim 7 of the present invention determines whether the input voice signal is accompaniment information or singing voice information using a pitch calculated from the input voice signal. Can do. In karaoke BOX and the like, it is possible to prevent a problem that scoring accompaniment information that has entered the microphone as singing voice information, thereby providing a highly accurate singing ability evaluation function.

  Further, a karaoke apparatus equipped with the singing ability evaluation function according to claim 8 of the present invention determines whether the input voice signal is accompaniment information or singing voice information by using the slope of the spectrum calculated from the input voice signal. can do. In karaoke BOX and the like, it is possible to prevent a problem that scoring accompaniment information that has entered the microphone as singing voice information, thereby providing a highly accurate singing ability evaluation function.

  Moreover, the karaoke apparatus equipped with the singing ability evaluation function according to claim 9 of the present invention uses the amount of overtones included in the spectrum in addition to the effect of the invention according to any of claims 6 to 8. Since accompaniment / singing voice determination is performed, a more accurate singing ability evaluation function can be provided.

  Moreover, in addition to the effect of the invention according to any one of claims 6 to 8, the karaoke apparatus equipped with the singing ability evaluation function according to claim 10 of the present invention further utilizes the number of extreme points of the spectrum. Since accompaniment / singing voice determination is performed, a more accurate singing ability evaluation function can be provided.

  Next, embodiments to which the present invention is applied will be described in detail with reference to the drawings. As an embodiment of the present invention, a karaoke apparatus equipped with a singing ability evaluation apparatus will be described. FIG. 1 is an external view of a karaoke scoring device in the present embodiment. As shown in FIG. 1, a microphone 2, a display 3, and an AMP 4 are connected to the karaoke apparatus 1, and a speaker 5 is connected to the AMP 4.

  FIG. 2 is a block diagram illustrating the internal structure of the karaoke apparatus 1. As shown in FIG. 2, the karaoke apparatus is composed of an electronic circuit centered on a CPU (Central Processing Unit) 19. The CPU 19 is connected to an HDD (Hard Disk Drive) 18, a video controller 6, a mixer 7, a RAM (Random Access Memory) 9, and a scoring processing unit 12, and controls the operation of each device. The microphone 2 is connected to the A / D conversion unit 17 inside the karaoke apparatus 1. The A / D converter 17 is connected to the RAM 9 and the mixer 7. The mixer 7 is connected to the performance device 8 and transmits the output of the mixer 7 to the external AMP 4 via the performance device 8. The RAM 9 is connected to an A / D conversion unit 17, a scoring processing unit 12, and a CPU 19. The RAM 9 stores the audio signal A / D converted by the A / D converter 17, the karaoke scoring result calculated by the scoring processor 12, and the like.

  The scoring processing unit 12 includes a pitch extraction unit 13, a vibrato detection unit 14, and a score calculation unit 15. The score calculation unit 15 calculates a score based on outputs from the pitch extraction unit 13 and the vibrato detection unit 14. It has become.

  The HDD 18 stores background images, performance data, lyrics telop, and other information of a large number of karaoke songs. The operation unit 16 includes a panel switch and a remote control receiving circuit, and transmits an operation signal from the user to the CPU 19. When the singer selects a specific song from a large number of karaoke songs and inputs it from the operation unit 16, the CPU 19 receives the signal and reads the performance data of the corresponding karaoke song from the HDD 18 and outputs it to the mixer.

  On the other hand, the singing voice of the singer input from the microphone 2 is sampled by the A / D converter 17 and sent to the mixer 7. The mixer 7 synthesizes the singing voice of the singer inputted from the microphone 2 and the performance data read from the HDD 18 and outputs the synthesized data to the performance device 8. The synthesized performance data is output from the speaker 5 via the AMP 4. At the same time, the CPU 19 sends the background video and the lyrics telop to the video controller 6. The lyrics telop is displayed on the display 3 in synchronization with the performance, and the color of the currently performed lyrics telop changes. The singer sings according to the accompaniment while watching the lyrics telop. The CPU 19 is in charge of this series of operation control. The video controller 6 corresponds to “display control means” in the claims, and the display 3 corresponds to “display means” in the claims.

  Next, the operation of the scoring processing unit 12 will be described. Whether or not to use the karaoke scoring function attached to the karaoke device is left to the singer's intention. A singer who desires karaoke scoring operates the operation unit 16 to turn on the scoring function. When the singer's karaoke song starts, the CPU 19 gives a scoring start instruction to the scoring unit 12. The scoring unit 12 given the scoring start instruction starts karaoke scoring. When the karaoke scoring is started, the CPU 19 reads out the karaoke song data sung by the singer from the HDD 18 and starts writing the guide melody included in the read song data into the RAM 9. On the other hand, the singing voice of the singer input from the microphone 2 is sampled by the A / D conversion unit 17 and recorded as an audio signal in the RAM 9 by DMA (Direct Memory Access). The pitch extraction unit 13 reads an audio signal from the RAM 9 and calculates a pitch. At the same time, the vibrato detection unit 14 reads an audio signal from the RAM 9 and detects vibrato. The score calculation unit 15 compares the pitch information detected by the pitch extraction unit 13 with the guide melody read from the RAM 9, and the vibrato information detected by the vibrato detection unit 14 and the accompaniment / singing voice determination unit 21 compare with the comparison result. The score is calculated in consideration of the output accompaniment / singing voice determination result. The calculated score is written in the RAM 9 as a scoring result. The A / D converter 17 corresponds to the voice signal input means described in the claims.

  When the CPU 19 gives a scoring end instruction to the scoring processing unit 12 after the performance is finished, the karaoke scoring process ends. The CPU 19 sends the scoring result read from the RAM 9 to the video controller 6. The scoring result is displayed on the display 3, and the singer is glad to see the scoring result of the song. In the present embodiment, the score is displayed on the display 3 after the performance is finished. However, the present invention is not limited to this, and the intermediate score from the time the singer starts to sing until the present time is sequentially displayed on the display, and the score is obtained while the singer sings. A structure that can be confirmed may be used.

  The scoring processing unit 12 includes a DSP (Digital Signal Processor) and scoring dedicated firmware that performs scoring processing. In a normal karaoke scoring device, the DSP is used only for pitch calculation, and the scoring process is often performed by the CPU. However, in this embodiment, almost all processing related to scoring is handled by the DSP (scoring processing unit). With this configuration, the degree of freedom in scoring circuit design is greatly improved and a very detailed analysis is possible.

  Next, the storage area set in the RAM 9 will be described with reference to FIG. The audio signal storage memory 9 </ b> A is an area for storing the audio signal that has been A / D converted by the A / D conversion unit 17. The autocorrelation function storage memory 9B is an area for storing the autocorrelation function calculated by the pitch extraction unit 13 in the scoring processing unit 12. The Fourier series storage memory 9 </ b> C is an area for storing a Fourier series calculated by the pitch extraction unit 13 in the scoring processing unit 12. In the voiced / unvoiced determination result storage memory 9D, is the cut out voice frame read from the RAM 9 a voiced sound? Is it silent? This is an area for storing the determined result. In the accompaniment / singing voice determination result memory 9E, is the cut out voice frame read from the RAM 9 the singing voice information? Accompaniment information? This is an area for storing the determined result. The pitch storage memory 9 </ b> F is an area for storing a pitch calculated by the pitch extraction unit 13. The pitch change amount storage memory 9G is an area for storing the pitch change amount calculated by the vibrato detection unit 14. The vibrato information storage memory 9H is an area for storing the vibrato information calculated by the vibrato detection unit 14. The instantaneous score storage memory 9I is an area for storing the instantaneous score calculated by the score calculation unit 15. The cumulative score storage memory 9J is an area for storing the cumulative score calculated by the score calculation unit 15. The work memory 9W is an area used by the scoring processing unit for temporary storage of scoring processing. The autocorrelation function storage memory 9B and the Fourier series storage memory 9C correspond to “spectrum storage means” in the claims.

  FIG. 4 is a flowchart illustrating the scoring operation procedure performed by the scoring processing unit 12. The operation of the scoring unit 12 will be described with reference to FIGS. The scoring unit 12 includes a pitch extraction unit 13, a vibrato detection unit 14, an accompaniment / singing voice determination unit 21, and a score calculation unit 15.

  First, the operation of the pitch extraction unit 13 will be described with reference to the flowchart shown in FIG. In the pitch extraction process, first, an autocorrelation function is calculated based on the audio signal read from the audio signal storage memory 9A of the RAM 9 and written to the autocorrelation function storage memory 9B of the RAM 9 (S10). Next, fast Fourier transform is performed on the audio signal read from the audio signal storage memory 9A of the RAM 9, and the obtained Fourier series is written in the Fourier series storage memory 9C of the RAM 9 (S11). Next, based on the autocorrelation function read from the autocorrelation function storage memory 9B of the RAM 9, it is determined whether the input speech is voiced or unvoiced? The determination result is stored in the voiced / unvoiced determination result storage area of the RAM 9. Write to 9D (S12). Next, the autocorrelation function and the Fourier series are read from the autocorrelation function storage memory 9B of the RAM 9 and the Fourier series storage memory 9C of the RAM 9, and accompaniment / singing voice determination is performed based on these, and the obtained accompaniment / singing voice determination result is obtained. The accompaniment / singing voice determination result storage memory 9E of the RAM 9 is written (S13). Next, the autocorrelation function and the Fourier series are read out from the autocorrelation function storage memory 9B of the RAM 9 and the Fourier series storage memory 9C of the RAM 9, the pitch is extracted based on these, and the detected pitch is stored in the pitch storage memory 9F of the RAM 9. (S14). The pitch extraction unit 13 takes charge of the processes from S10 to S14. The processes of S10 and S11 correspond to the “spectrum calculation step” and “spectrum storage step” in the claims, and the scoring processing unit (DSP) 12 that executes the processes of S10 and S11 is used as the “spectrum calculation means”. Equivalent to. The process of S13 corresponds to the “accompaniment / singing voice determination step” described in the claims, and the scoring processing unit (DSP) 12 that executes the processes of S10 and S11 corresponds to “accompaniment / singing voice determination means”.

  Next, processing of the vibrato detection unit 14 will be described. The vibrato detection unit 14 calculates a pitch change amount based on the pitch read from the pitch storage memory 9F of the RAM 9 and writes it to the pitch change amount storage memory 9G of the RAM 9 (S15). Next, vibrato detection is performed based on the pitch change amount read from the pitch change amount storage memory 9G of the RAM 9, and the vibrato detection result is written in the vibrato information storage memory of the RAM 9 (S16). The vibrato detection unit 14 takes charge of the processes from S15 to S16.

  The score calculation unit 15 includes a pitch storage memory 9F, a guide melody storage memory 9M, a voiced / unvoiced determination result storage memory 9D, an accompaniment / singing voice determination result storage memory 9E, a pitch change amount storage memory 9G, and a vibrato information storage memory 9H. The pitch, guide melody, voiced / unvoiced determination result, accompaniment / singing voice determination result, pitch change amount, and vibrato detection result are read out, and the score is calculated based on these (S17). The score calculation result obtained by the process S17 is written into the instantaneous score storage memory 9I and the cumulative score storage memory 9J of the RAM 9. The instantaneous score storage memory 9I of the RAM 9 records the instantaneous score analyzed for a short time, and the cumulative score storage memory 9J of the RAM 9 averages by accumulating the instantaneous scores from the start to the present. A score is recorded.

  The instantaneous score is compared with the guide melody and the pitch, and the similarity is scored. However, score calculation is not performed for an unvoiced section in which the determination result read from the voiced / unvoiced determination result storage memory 9D is determined to be “unvoiced sound”. Further, no score calculation is performed for the accompaniment section in which the determination result read from the accompaniment / singing voice determination result storage memory 9E is determined to be “accompaniment information”. Further, the score calculation is not performed for the section where the pitch change amount read from the pitch change amount storage memory 9G is intense. In addition, for the section determined as the “vibrato section” from the information read from the vibrato information storage memory 9H, the beauty of the vibrato is calculated and used as a score. The series of scoring processes ends when a scoring end instruction is received from the CPU 19 (S18). The average score written in the cumulative score storage memory 9J becomes the score of the singer's song.

Hereinafter, operations of the pitch extraction unit 13, the vibrato detection unit 14, and the score calculation unit 15 will be described in detail. The pitch extraction unit 13 performs autocorrelation analysis on the input speech as preprocessing, and obtains an autocorrelation function. In this embodiment, the singing voice input from the microphone 2 is sampled by the A / D converter 17 at a sampling frequency of 48 [kHz] and written to the voice signal storage memory 9 </ b> A of the RAM 9. The scoring unit 12 cuts out an analysis frame of 1440 [points] per analysis from the audio signal storage memory 9A of the RAM 9 and performs an analysis. In the autocorrelation function method, “analysis frame: F ₀ = {x (1), x (2),..., X (N)}” and “analysis frame shifted by i [point]: F _i = {x (I), x (1 + i),..., X (N + i)} ”. An example of an equation for calculating the correlation value R (0, i) is shown in Equation 1.

In the autocorrelation function method, the deviation amount i between F ₀ and F _i is changed from 1 to N [point], and the correlation value (similarity) R (0, i) is sequentially calculated. The autocorrelation function R (0, i) calculated in this way is written into the autocorrelation function storage memory 9B of the memory 9.

In the pitch extraction method using the autocorrelation function, the pitch (fundamental frequency) f is calculated by Equation 2 using the correlation value R (0, i). Formula 2 detects the shift amount argmax _i {R (0, i)} that maximizes the correlation value R (0, i) when the shift amount i is sequentially changed, as the fundamental period of the audio signal, This means that the sampling frequency 48000 [Hz] divided by the basic period is calculated as the pitch. The pitch extracted in this way is written into the pitch storage memory 9F of the RAM 9.

Here, a method for determining whether a singer's singing voice is voiced sound or unvoiced sound will be briefly described. Human voices include voiced and unvoiced sounds, but it is generally known that the pitch cannot be calculated from unvoiced sounds. For this reason, it is necessary to determine voiced / unvoiced sound before calculating the pitch. Voiced / unvoiced can be easily determined using the ratio R (0, i _max ) / R (0, 0) of the autocorrelation function used in Equation 1. If R (0, i _max ) / R (0, 0) is greater than a certain threshold value, it is determined as a voiced sound, and if it is smaller than a certain threshold value, it is determined as an unvoiced sound. In the present embodiment, unvoiced sounds are not used for singing ability evaluation because of their low pitch reliability. The voiced / unvoiced determination method is not limited to the autocorrelation function ratio, and other known voiced / unvoiced determination techniques such as a zero-cross method may be used.

In this embodiment, pitch extraction using an autocorrelation function and pitch extraction using fast Fourier transform (FFT) are used together to realize more reliable pitch extraction. In pitch extraction using the fast Fourier transform, an FFT spectrum is calculated using the fast Fourier transform for the audio signal read from the audio signal storage memory of the memory 9 and written into the Fourier series storage memory 9C. The pitch is detected as a frequency when the FFT spectrum takes the maximum value. The pitch extraction method using the autocorrelation function is effective for pitch extraction from male bass singing voices, and the pitch extraction method using fast Fourier transform is effective for pitch extraction from female treble singing voices. The pitch f used for the scoring process is selected from the pitch f1 calculated from the autocorrelation function and the pitch f2 calculated by the fast Fourier transform, for example, according to the selection criteria shown in Table 1. When f1 and f2 are larger than the fixed threshold value FTH, f = f2 is selected. Otherwise, f = f1 is selected. By this method, a highly reliable pitch f can be selected. FTH is a preset threshold value for determining high and low sounds. Here, for example, FTH = 400 [Hz].

Next, the operation of the vibrato detection unit 14 will be described. The vibrato detection unit 14 first calculates a pitch change amount. Pitch change D (i), using an analysis frame past pitch f _i between the current pitch f _{i + 1} read from the pitch storage memory 9F in RAM 9, is calculated by equation 3. The calculated pitch change amount D (i) is written in the pitch change amount storage memory 9G.

In the pitch change amount storage memory 9G of the RAM 9, the pitch change amount signal calculated by Equation 3 is buffered for 500 [ms], for example. Assuming that the frame shift is 10 [ms], 500 [ms] corresponds to 50 frames, so that the pitch change amount signal for N = 50 points is buffered. An autocorrelation function e (τ) expressed by Equation 4 is calculated for the pitch change signal D (i) of 50 points. The autocorrelation function is a function suitable for examining the periodicity of a signal. When the autocorrelation function e (τ) exceeds a certain threshold value, the pitch variation signal is considered to have a certain degree of periodicity, and therefore it can be determined that the input audio signal is vibrato. The vibrato determination result thus determined is written in the vibrato information storage memory 9H.

Next, the operation of the score calculation unit 15 will be described in detail. The score calculation unit 15 classifies the input audio signal into four sections (i) to (iv) shown in Table 2. (I) An unvoiced section and (iii) a section with a large pitch change (a section with a large pitch change amount) are not used for score calculation, and (ii) a vibrato section and (iv) a normal singing section are scored. The section determined as accompaniment information by the accompaniment / singing voice determination unit 21 is classified into the silent section (i) and is not used for score calculation. (Iv) In the normal singing section, the pitch extracted from the input voice signal is compared with the guide melody, and a score proportional to the similarity is calculated. (Ii) The score of the vibrato section is calculated by, for example, multiplying the maximum value of the correlation strength e (τ) calculated by Equation 4 by a preset constant. Vibrato is considered to be more beautiful as the value of e (τ) is larger and the periodicity is stronger. Therefore, for example, the score of the vibrato section is calculated by multiplying the maximum value of e (τ) by a preset constant. The final score is calculated as the sum of (ii) the score of the vibrato section and (iv) the score of the normal singing section.

  FIG. 5 is a block diagram of the score calculation unit 15. The reliability calculation module 151 stores the voiced / unvoiced determination result, the accompaniment / singing voice determination result, and the pitch change amount from the voiced / unvoiced determination result memory 9D of the RAM 9, the accompaniment / singing voice determination result memory 9E, and the pitch change amount storage memory 9G, respectively. Reading, the pitch reliability is calculated based on these, and the calculated pitch reliability is written in the work area 9W of the RAM 9. The instantaneous score calculation module 152 reads the pitch, vibrato determination result, guide melody, and pitch reliability from the pitch storage memory 9F, the vibrato information storage memory 9H, the guide melody storage memory 9M, and the work area 9W of the RAM 9, respectively. The instantaneous score for one singing voice analysis frame is calculated, and the calculated instantaneous score is written in the instantaneous score storage memory 9I of the RAM 9. The score accumulating module 153 accumulates the instantaneous score read from the instantaneous score storage memory 9I of the RAM 9, calculates the cumulative score from the start of karaoke scoring until the present, and records it in the cumulative score storage memory 9J of the RAM 9 To do.

  After the karaoke scoring is completed, the cumulative score recorded in the cumulative score storage memory 9J is read from the RAM 9 and displayed on the display 3 via the video controller 6. The singer is anxious to see the scoring results displayed on the display 3. The accumulated score may be displayed on the display as it is, or may be displayed after conversion through a score conversion function or a conversion table. For these conversion tables and conversion functions, for example, a conversion table or conversion function designed so that the probability that 100 points are calculated is 5% or less after performing a statistical survey of the score distribution in advance.

  Next, the operation of the accompaniment / singing voice determination unit 21 will be described in detail. First, in order to design a circuit for determining whether the voice input from the dynamic microphone connected to the karaoke apparatus is a karaoke accompaniment or a human singing voice, the frequency characteristics of the dynamic microphone will be considered.

  FIG. 6 shows frequency characteristics of a dynamic microphone connected to a general karaoke apparatus. A dynamic microphone has a characteristic in which a frequency characteristic changes depending on a distance from a singer's mouth to the microphone. For example, (a) when the distance between the sound source and the microphone is 25 mm, a low-frequency emphasis characteristic in which a low tone near 200 [Hz] is most emphasized and a high tone above 1000 [Hz] is weakened. (B) When the distance between the sound source and the microphone is 50 mm, the characteristics are almost flat. (C) When the sound source is 600 mm away from the microphone, there is almost no bass in the vicinity of 100 [Hz], and the graph rises to the right (high frequency emphasis characteristic) in which trebles of 1000 [Hz] or higher are emphasized.

  FIG. 7 illustrates the microphone frequency characteristics using an example that is easier to understand. For example, even if a soprano singer sings from a distance, the dynamic microphone can pick up the singing voice. Pick up the high voice of soprano singer even if it is 1-2 m away. Even if the microphone is far away, pick it up if it is loud. On the other hand, a voice with a low bass vocal is not picked up after 10 cm away.

  Due to the nature of the dynamic microphone described above, when a karaoke accompaniment wraps around a microphone from a speaker farther away than the singer who has the microphone, the accompaniment entering the microphone is a high-frequency emphasis on the accompaniment coming out of the speaker. Become. That is, the sound that wraps around the microphone has a low specific gravity of the BASS line (bass), and is accompanied by a high specific gravity of the drum hi-hat sound and the electric guitar sound (high sound) (see FIG. 8). Therefore, if frequency analysis is performed on accompaniment information input from a dynamic microphone and a spectrum is calculated, the spectrum energy tends to concentrate in a high frequency band.

  Using this property, accompaniment information and human voice can be discriminated. One method for detecting a feature in which spectrum energy is concentrated in a high frequency band is a method using pitch extraction. When a pitch is extracted from an accompaniment that wraps around from a speaker farther away than the singer, the frequency often becomes higher than that of a human singing voice (however, the high frequency of a woman: the band may overlap in the vicinity of 800 Hz or higher). Hereinafter, the accompaniment / singing voice discrimination method will be described.

  In the embodiment of the present invention, first, the pitch detector 13 calculates a pitch in consideration of a band higher than a human voice. Specifically, the analysis band in the autocorrelation function calculation (S10) and Fourier series calculation (S11) in FIG. 4 is set higher than the human singing band (70 Hz to 1200 Hz), for example, 70 Hz to 7000 Hz. The pitch is extracted from the autocorrelation function and Fourier series having an analysis band of 70 Hz to 7000 Hz. When a pitch of 1200 Hz or higher is detected with this configuration, that portion is regarded as accompaniment information and is not used for score calculation. In this method, for example, when some female singers who are good at high frequencies sing, a part of the singing voice may be regarded as an accompaniment. However, since there is no singer who always sings at a pitch of 1200 Hz (D6) or higher, there is no practical problem in karaoke scoring. First, the accompaniment / singing voice determination result determined by the above-described method is written in the accompaniment / singing voice determination result storage memory 9E.

  However, since accompaniment information in the vicinity of the 800 Hz band may be input from the microphone, there is a problem that accompaniment information cannot be completely removed only by the above-described determination method. Next, a method for determining accompaniment using the characteristics of an autocorrelation function when a pitch of a band that is difficult to determine whether it is female voice or accompaniment information in the vicinity of the 800 Hz band (somewhat high) will be described. For example, when a woman sings at a high frequency around 800 Hz, the autocorrelation function extracted from the singing voice often has a relatively smooth waveform as shown in FIG. 9 (there is less harmonics). On the other hand, the autocorrelation function calculated from the accompaniment information has an extremely large number of extreme values (local peaks) included in the spectrum as shown in FIG. Male singing voices have many overtones, but the detected pitch has a low frequency.

  Using this feature, even when a “pitch that is difficult to determine whether it is female singing voice or accompaniment information” of 480 Hz to 1200 Hz, for example, the local peak (extreme value) of a spectrum such as an autocorrelation function is 100 or more, for example ( If the harmonics are abnormally high), the portion can be determined as accompaniment or noise. In the embodiment of the present invention, the number of local peaks (extreme values) of the autocorrelation function is counted from the autocorrelation function storage memory 9B of the RAM 9 using the above-described method. When the number of local peaks is larger than a preset threshold value TH (for example, 100), it is determined that the input voice is not human singing voice but accompaniment information (or noise). The accompaniment / singing voice determination result determined by this method is recorded in the accompaniment / singing voice determination result storage memory 9E of the RAM 9.

There are various other methods for detecting that the number of overtones is abnormally large. For example, it is possible to calculate the length L connecting adjacent elements of the spectrum and determine that “the number of overtones is abnormally large” when the scale L is larger than a certain threshold value. For example, when the autocorrelation function R (0, i) calculated by Equation 1 is “spectrum”, the length L can be calculated from this spectrum using Equation 5 or Equation 6. Equations 5 and 6 are essentially the same, and either one can be used to measure (quantify) “overtones”. Thus, for example, even when “a female singing voice or accompaniment information pitch that is difficult to determine” of 480 Hz to 1200 Hz is detected, if L is larger than a certain threshold value, the portion can be determined as accompaniment or noise. . Needless to say, an FFT spectrum or other spectrum can be used as the spectrum instead of the autocorrelation function.

  In this embodiment, in addition to the accompaniment / singing voice determination using the autocorrelation function described above, the singing voice / accompaniment discrimination is performed using the Fourier series (FFT spectrum) stored in the Fourier series storage memory 9C of the memory 9. This increases the reliability of accompaniment / singing voice determination. Hereinafter, an accompaniment / singing voice determination method using a Fourier series will be described. FIG. 11A shows an FFT spectrum obtained by applying a fast Fourier transform to a singing voice collected from a microphone, and FIG. 11B shows an FFT spectrum obtained from an accompaniment that wraps around the microphone. For example, the pitch can be detected as a frequency when the FFT spectrum takes the maximum value. The pitch extracted from the singing voice is distributed around 400 [Hz], and the FFT spectrum of the accompaniment that has come around the microphone is distributed around 1500 [Hz]. For example, when a pitch of 1200 [Hz] or more is detected, the portion is regarded as accompaniment information and is not used for score calculation, so that the accuracy of karaoke scoring can be improved. The accompaniment / singing voice determination result determined by this method is recorded in the accompaniment / singing voice determination result storage memory 9E of the RAM 9.

  Finally, in the embodiment of the present invention, in addition to the accompaniment / singing voice determination described above, the singing voice / accompaniment determination is performed from the slope of the FFT spectrum (Fourier series) read from the Fourier series storage memory 9C of the memory 9. Hereinafter, a method of performing accompaniment / singing voice determination from the inclination of the spectrum will be described. A spectrum between 200 [Hz] and 7000 [Hz] is cut out, and the slope of the spectrum when the spectrum is linearly approximated by the least square method is calculated. When the slope of the straight line is larger than a predetermined threshold value TH, it is determined that “the input voice is accompaniment information”. The accompaniment / singing voice determination result determined by this method is recorded in the accompaniment / singing voice determination result storage memory 9E of the RAM 9.

  FIG. 12A shows the slope of the FFT spectrum obtained by applying fast Fourier transform to the singing voice inputted from the microphone, and FIG. 12B shows the FFT spectrum obtained by applying fast Fourier transform to the accompaniment that wraps around the microphone. Is the slope of It can be observed that the slope of the FFT spectrum calculated from the singing voice increases and the slope of the FFT spectrum calculated from the accompaniment decreases. Using this feature, it is easy to discriminate between the singing voice and the accompaniment coming from the microphone. In this way, accompaniment / singing voice can be determined using the feature that “the sound coming from a sound source farther away from the singer is emphasized by the high frequency by the high frequency emphasis characteristic of the dynamic microphone”.

  In this embodiment, the autocorrelation function method and the Fourier transform method are used as the pitch detection method. However, the pitch detection method is not limited to this, and for example, the cross correlation method, the cepstrum method, the square root / fourth root spectrum self Other known pitch detection methods such as a correlation method, a logarithmic spectrum autocorrelation method, and a linear prediction method can also be used. In the present invention, the input sound is determined as accompaniment information when a high pitch that is less likely to be a human voice is detected, and the pitch detection method is not particularly limited. Needless to say, as a spectrum used for accompaniment discrimination, for example, an LPC spectrum, a cross-correlation function, a cepstrum, an LPC cepstrum, a square root / quarter root spectrum, etc. obtained by linear prediction analysis can be used.

  In addition, as the frequency feature used for obtaining the slope of the straight line, other known frequency features such as an LPC spectrum, a group delay spectrum, an LPC cepstrum, a cepstrum, an autocorrelation function, a cross correlation function, and the like can be used.

For example, fast Fourier transform is applied to the input speech to obtain an FFT spectrum. Next, when a fast inverse Fourier transform is applied to the FFT spectrum, a feature amount called a cepstrum is obtained. When the high-order coefficient of the cepstrum is replaced with 0 and further subjected to fast Fourier transform, a smoothed spectrum is obtained. In this smoothed spectrum, for example, an average value AH of higher-order spectral coefficients of 600 [Hz] or more and an average value AL of lower-order coefficients of less than 600 [Hz] are calculated. For example, the ratio RATE between the higher-order spectral coefficient and the lower-order spectral coefficient is obtained by Expression 7, and when RATE is larger than a certain threshold value, it can be determined that the input voice is accompaniment information.

  In the above example, the smoothed spectrum is obtained using the cepstrum after performing the inverse Fourier transform. However, the present invention is not limited to this, and AH, AL, and RATE are calculated from the Fourier transform coefficients that are not smoothed. Ratio of higher-order coefficient and lower-order coefficient: RATE may be calculated. Furthermore, other known frequency features such as LPC spectrum, group delay spectrum, LPC cepstrum, cepstrum, autocorrelation function, cross-correlation function, etc. can be used as the frequency feature used to determine the ratio RATE and the slope of the straight line. Needless to say.

  Moreover, in this embodiment, the example which calculated the score from the similarity by comparing the pitch extracted from the singing voice of the singer with the guide melody was shown. However, it is not limited to this, and a scoring system that does not refer to the guide melody can be constructed. For example, the minimum distance between the pitch extracted from the singing voice of the singer and the pitch of the average rate scale (or the pure rate scale) nearest to the pitch is calculated, and the smaller the minimum distance, the higher the score is calculated. A scoring system equipped with a scoring algorithm may be used. Further, a scoring system may be constructed that measures the amount of harmonics contained in a person's singing voice and calculates a higher score as the number of harmonics increases. Alternatively, an algorithm may be employed that estimates a pitch (pitch that does not become dissonant) between the guide melody and the harmonic sound, and adds a high score when the singer's singing voice matches the harmonic sound.

  The present invention can be used for a karaoke scoring device and a singing ability evaluation device mounted on a karaoke device.

It is an external appearance of the karaoke apparatus in 1st embodiment of this invention. It is a block diagram of the karaoke apparatus in the first embodiment of the present invention. 3 is a diagram showing a storage area secured in a RAM 9. FIG. It is a flowchart which shows operation | movement of the scoring process part of the karaoke apparatus. It is a block diagram which shows the score calculation procedure in embodiment of this invention. It is a figure which shows the frequency characteristic of a dynamic microphone. It is the figure which explained concretely the phenomenon caused by the frequency characteristic of a dynamic microphone. It is the figure explaining the property of the accompaniment information which goes around to a microphone in karaoke. It is the figure which showed the autocorrelation function (16 local peaks) computed from the singing voice of 880 [Hz] which a woman sings. It is the figure which showed the autocorrelation function calculated from the accompaniment information. It is a figure which shows the FFT spectrum extracted from the FFT spectrum extracted from the singing voice, and the accompaniment. It is the figure which compared the inclination of the spectrum calculated from the accompaniment which went around to the microphone and the inclination of the spectrum calculated from the singing voice. It is a figure explaining the wraparound of the accompaniment in karaoke scoring.

Explanation of symbols

1 Karaoke device 2 Microphone 3 Display 4 AMP
5 Speaker 6 Video controller 7 Mixer (effector)
8 Performance Equipment 9 RAM (Random Access Memory)
12 Scoring processor (scoring DSP)
DESCRIPTION OF SYMBOLS 13 Pitch extraction part 14 Vibrato detection part 15 Score calculation part 16 Operation part 17 A / D conversion part 18 HDD (Hard Disk Drive)
19 CPU (Central Processing Unit)
21 Accompaniment / Singing Voice Determination Unit 151 Reliability Calculation Module 152 Instantaneous Score Calculation Module 153 Score Accumulation Module

Claims (10)

On the computer,
A spectrum calculating step of performing frequency analysis on the input voice signal input from the voice signal input means and calculating a spectrum;
A spectrum storage step of storing the spectrum obtained by executing the spectrum calculation step in the spectrum storage means;
And performing an accompaniment / singing voice determination step for determining that the input audio signal is accompaniment information when it is detected that the spectrum energy read from the spectrum storage means is concentrated in a high frequency band. Characteristic singing ability evaluation method. In the accompaniment / singing voice determination step,
The pitch is calculated from the spectrum read from the spectrum storage means,
The singing ability evaluation method according to claim 1, wherein when the pitch is higher than a certain threshold value, the input voice signal is determined to be accompaniment information. In the accompaniment / singing voice determination step,
The slope of the spectrum is calculated from the spectrum read from the spectrum storage means,
2. The singing ability evaluation method according to claim 1, wherein the input voice signal is determined to be accompaniment information when the calculated slope of the spectrum is larger than a certain threshold value. A singing ability evaluation method according to any one of claims 1 to 3,
In the accompaniment / singing voice determination step, from the spectrum read out from the spectrum storage means, the amount of harmonics included in the spectrum is measured,
A singing ability evaluation method characterized in that an input audio signal is determined to be accompaniment information when the amount of overtones is greater than a certain threshold value. A singing ability evaluation method according to any one of claims 1 to 3,
In the accompaniment / singing voice determination step, from the spectrum read out from the spectrum storage means, the number of extreme values included in the spectrum is counted,
A singing ability evaluation method characterized in that an input audio signal is determined to be accompaniment information when the number of extreme values is greater than a certain threshold value. Spectrum calculation means for performing frequency analysis on the input voice signal input from the voice signal input means and calculating a spectrum;
Spectrum storage means for storing the spectrum obtained by the spectrum calculation means;
An accompaniment / singing voice judging means for judging that the input audio signal is accompaniment information when it is detected that the energy of the spectrum read out by the spectrum storing means is concentrated in a high frequency band. A karaoke device with a singing ability evaluation function. The accompaniment / singing voice judging means is:
Calculate the pitch from the spectrum read by the spectrum storage means,
The karaoke apparatus equipped with the singing ability evaluation function according to claim 6, wherein the input voice signal is determined to be accompaniment information when the pitch is higher than a certain threshold value. The accompaniment / singing voice judging means is:
Calculating the slope of the spectrum from the spectrum read by the spectrum storage means;
7. The karaoke apparatus equipped with the singing ability evaluation function according to claim 6, wherein the input audio signal is determined to be accompaniment information when the calculated inclination of the spectrum is larger than a certain threshold value. A karaoke apparatus according to any one of claims 6 to 8,
The accompaniment / singing voice determination means further measures the amount of harmonics contained in the spectrum from the spectrum read out by the spectrum storage means,
A karaoke apparatus equipped with a singing ability evaluation function, characterized in that an input voice signal is determined to be accompaniment information when the amount of overtones exceeds a certain threshold. A karaoke apparatus according to any one of claims 6 to 8,
The accompaniment / singing voice determination means further counts the number of extreme values included in the spectrum from the spectrum read out by the spectrum storage means,
A karaoke apparatus equipped with a singing ability evaluation function, characterized in that an input voice signal is determined to be accompaniment information when the number of extreme values is greater than a certain threshold value.

Images