JP2010134475A - Singing rating system, and program for singing rating processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately and promptly evaluate singing capability at low cost, without requiring a memory for storing a large amount of evaluation values, and calculation processing for calculating an average of the large amount of evaluation values after singing is finished. <P>SOLUTION: A CPU 1 sets a parameter t for defining an allowance range of a reference value in which the singing capability to be evaluated becomes 100 points, and a parameter a for defining a degree of singing capability outside of the allowance range from 100 to 0 points, and calculates the evaluation value of an input sound signal on the basis of the parameter t and the parameter a. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a singing scoring apparatus and a singing scoring program for scoring the singing ability of an input audio signal.

A karaoke scoring device is known as a device for scoring the singing ability of an input audio signal.
For example, in a karaoke scoring device of a certain patent document, in order to evaluate a singer's way of singing melody information given by a MIDI message, the first detection means uses its pitch data and level data based on the singer's voice. The second detecting means detects note on / off data, pitch data and level data corresponding to the singing melody to be generated by the singer in the MIDI message. Then, the detected pitch data and level data are individually compared by the pitch comparison means and the level comparison means, and data for evaluating the singing method is created based on the comparison result and the note on / off data. . (See Patent Document 1)
In addition, the karaoke scoring device of another patent document is shown in FIG. 1 in this patent document in order to automatically determine the degree of approximation between the model voice and the voice of the singer input from the microphone more accurately. As shown in the configuration, the audio signal of the model song reproduced on the laser disc 101 is supplied to the level detection unit A103 and the pitch detection unit A104, and the signal level data and pitch detection detected by the level detection unit A103. The pitch data detected in the part A104 is stored in the buffer A102. Further, the user's voice is input from the microphone 201, the voice signal is supplied to the level detection unit B203 and the pitch detection unit B204, and is detected by the signal level data detected by the level detection unit B203 and the pitch detection unit B204. The pitch data is stored in the buffer B202. Then, the comparison / determination unit 300 reads the data stored in each system, determines the timing of singing from the level signal, the pitch deviation from the pitch data, and outputs the comparison result as scoring data. (See Patent Document 2)

On the other hand, when analyzing an audio signal input from a microphone or the like, a time-domain audio signal converted into a digital signal is converted into a frequency-domain spectrum signal by Fast Fourier Transform (FFT) or the like, and the converted spectrum signal is converted into a spectrum signal. Analyzing is conventionally performed.
For example, an audio conversion apparatus of a patent document receives a first audio signal that has been digitally converted in order to analyze a sound signal by converting a pitch frequency with high quality by a simple phase control process, and a spectrum by Fourier transform. Fourier transform means for converting to a signal, selection means for receiving a spectrum signal from the Fourier transform means and selectively outputting a sound source information signal from the spectrum signal, a sound source information signal from the selection means, and a pitch of the sound source information signal The pitch frequency is converted in response to the frequency conversion means for converting the frequency and outputting the frequency-converted signal, and the spectrum envelope signal included in the spectrum signal from the Fourier transform means and the signal output from the frequency conversion means. The received spectrum signal is received every analysis frame, and this is converted into a second audio signal by inverse Fourier transform. A structure including a Fourier transform means and a phase control means for receiving the second sound signal from the inverse Fourier transform means and controlling the phase of the second sound signal by the shift width of the analysis frame in response to the conversion factor of the pitch frequency. It has become. (See Patent Document 3)

Japanese Patent Laid-Open No. 10-49183 Japanese Patent Laid-Open No. 11-224094 Japanese Patent No. 2753716

However, in Patent Document 1 and Patent Document 2, the singing ability is evaluated by simply comparing the reference value (melody information, model voice) and the input voice of the singer. Since the evaluation value varies finely from the start to the end, a memory that temporarily stores a large number of evaluation values is required, resulting in an increase in cost, and in the middle of the song or after the song ends There has been a problem that a large load is applied to control means such as a CPU due to arithmetic processing for calculating an average of a large number of evaluation values, particularly division processing.
Moreover, in the said patent document 1 and the patent document 2, when the senior singer generate | occur | produces fluctuation in a pitch by vibrato singing method, the subject that it judges incorrectly that it is pitch shift and evaluates singing ability low. there were.
In Patent Document 1 and Patent Document 2, the pitch and level of the input audio signal are compared by two systems of comparison means, and the singing ability is evaluated by detecting the pitch deviation and the pronunciation timing deviation. There has been a problem that the apparatus has a complicated configuration.
On the other hand, in Patent Document 3, a sound source information signal, that is, a pitch of a fundamental tone that is a fundamental frequency sound is directly detected and selected from a Fourier-transformed spectrum signal. Since the level of the harmonic overtone may be higher than the level, there has been a problem that the pitch of the fundamental tone of the audio signal input from the microphone cannot be reliably detected.

  In addition, the present invention reliably detects the pitch of the fundamental tone of the input singer's voice signal, and even when an advanced singer fluctuates in pitch due to the vibrato method, the singing ability is properly evaluated. With the goal.

  The singing scoring apparatus according to claim 1, wherein the singing ability to be evaluated is a singing out of the allowable range in the range of the first evaluation parameter and the maximum evaluation value to the minimum evaluation value that defines the allowable range of the reference value. Parameter setting means for setting a second parameter that defines the degree of force, signal analysis means for analyzing the number of errors in each frame with the number of samples of powers of 2 in the input speech signal as one frame, and the signal analysis When the number of errors analyzed by the means exceeds half of one frame, the frame is set as the lowest evaluation value, and when the number of errors analyzed does not exceed half of one frame, 2 of one frame The first parameter and the second parameter set by the parameter setting means with the evaluation value of the frame based on the number of samples other than the error corresponding to 1 / It has a configuration in which and an evaluation operation means for calculating based.

  2. The singing scoring apparatus according to claim 1, wherein the signal analysis means analyzes the pitch of the fundamental tone by detecting the greatest common divisor of at least two or more pitches from the input audio signal. The evaluation calculation means may be configured to calculate the evaluation value of the pitch of the fundamental tone detected by the signal analysis means.

  In the singing scoring device according to claim 2, as described in claim 3, the signal analysis means calculates a phase from the frequency component of the input audio signal, and uses the calculated phase from the audio signal. It may be configured to detect the greatest common divisor of at least two or more pitches.

  The singing scoring program according to claim 4 is out of an allowable range within a range of a first parameter and a maximum evaluation value to a minimum evaluation value that define an allowable range of a reference value at which a singing ability to be evaluated becomes a maximum evaluation value A step A for setting a second parameter that defines the degree of singing ability of the sound; a step B for analyzing the number of errors in each frame with the number of samples of powers of 2 in the input audio signal as one frame; and the step B If the number of errors analyzed by exceeds one half of one frame, that frame is regarded as the lowest evaluation value, and if the number of errors analyzed does not exceed one half of one frame, two minutes of one frame Based on the first parameter and the second parameter set in the step A, the evaluation value of the frame is determined by the number of samples other than the error corresponding to 1. It has a configuration to execute a step C of calculating and the computer.

  In the singing scoring program of claim 4, as described in claim 5, step B detects the greatest common divisor of at least two or more pitches from the input audio signal, and calculates the pitch of the fundamental tone. Analyzing, the step C may be configured to calculate the evaluation value of the pitch of the fundamental tone detected in the step B.

  In the singing scoring program of claim 5, as described in claim 6, the step B calculates the phase from the frequency component of the input audio signal, and uses the calculated phase to calculate the phase of the audio signal. Alternatively, the greatest common divisor of at least two or more pitches may be detected.

  According to the present invention, the pitch of the fundamental tone of the input singer's voice signal is reliably detected, and even when an advanced singer fluctuates in pitch due to the vibrato singing method, the singing ability is properly evaluated. An effect is obtained.

The block diagram which shows the structure of the karaoke apparatus in embodiment to which the singing scoring apparatus of this invention is applied. The function block diagram which represented the signal processing function of CPU of FIG. 1 as hardware. The figure which shows the structure of the counter inside CPU of FIG. 1 for a song scoring process. The flowchart of the main routine of CPU. 5 is a flowchart of switch processing in the main routine of FIG. The flowchart of CPU timer interrupt. The flowchart of the karaoke process in the main routine of FIG. The flowchart of the pitch difference calculation process in FIG. The flowchart of the pitch ratio calculation process in FIG. 10 is a flowchart of phase compensation processing in FIG. 9. The flowchart of the difference average calculation process in FIG. 8 is a flowchart of section score calculation processing in FIG. The figure which shows the specific example of the parameter in each level of a beginner level, an intermediate level, and an advanced level. The figure which shows the characteristic of the calculation formula of a section score. The figure which shows the screen which displays an area score and pitch deviation. The figure which shows the relationship between the utterance area of the audio | voice signal input, and the singing area of evaluation object.

Hereinafter, embodiments of the singing scoring device of the present invention will be described in detail with reference to the drawings, taking a karaoke device as an example.
FIG. 1 is a configuration diagram of a karaoke apparatus according to an embodiment. In FIG. 1, a CPU 1 controls the entire apparatus and has a small capacity ROM / RAM and a DSP (digital signal processor) function. A music memory 2, a switch unit 3, a ROM 4, a RAM 5, a display unit 6, an A / D converter 8, and a musical tone generation unit 9 are connected to the system bus of the CPU 1, and data and data are transmitted between the CPU 1 and each unit. Send and receive commands.

  The song memory 2 stores a plurality of accompaniment songs and lyrics for karaoke. The switch unit 3 includes a music selection switch, a start / stop switch, and other various switches. The ROM 4 stores a singing scoring program executed by the CPU 1 and various control data. The RAM 5 is a work area of the CPU 1 and has various registers. The display unit 6 includes, for example, a liquid crystal display (LCD) and a plurality of LEDs. A microphone 7 is connected to the A / D converter 8 wirelessly or by wire, performs A / D conversion of an analog audio signal input from the microphone 7 and outputs the audio data. For example, AD conversion is performed at a sampling frequency of 8021 Hz and 16 bits. Hereinafter, the voice data obtained by AD conversion will be referred to as “original voice data” or “original waveform data” for convenience, and the voice input to the microphone 7 will be referred to as “original voice”. . The musical tone generation unit 9 generates musical tone generation waveform data in accordance with an instruction from the CPU 1. The D / A converter 10 performs D / A conversion on the waveform data generated by the musical tone generation unit 9 and outputs an analog audio signal. The sound system 11 emits the audio signal.

  FIG. 2 shows the hardware from the signal processing of the CPU 1 to the audio signal input from the microphone 7 of FIG. 1 to the A / D converter 8 until it is output from the D / A converter 10. 2 is a functional configuration diagram represented as hardware. In FIG. 2, an input buffer 21 is a buffer that temporarily stores original audio data output from the A / D converter 8. The frame extraction unit 22 extracts a frame that is audio data of a predetermined size from the original audio data stored in the input buffer 21. The size, that is, the number of audio data (samples) is, for example, 256. In order to perform accurate phase expansion, it is necessary to extract the frames by overlapping them. Therefore, the frames are cut out by overlapping with the overlap factor OVL. The factor OVL is set to 4. In this case, the hop size is 64 (256/64 = 4). The range of the pitch scaling value from the pitch of the original audio data (hereinafter referred to as “original pitch”) to the target pitch is premised on the range of 0.5 to 2.0.

  The frame extracted by the frame extraction unit 22 is output to a low pass filter (LPF) 23. The LPF 23 removes high frequency components in order to prevent the frequency components from exceeding the Nyquist frequency due to pitch shift. The FFT unit 24 performs fast Fourier transform (FFT) on the frame output by the LPF 23. The FFT is executed by setting the FFT size (number of points) to twice the frame size (256 × 2 = 512).

  The phase compensation unit 25 expands / contracts the size so as to compensate for the expansion / contraction of the frame due to the pitch shift, with respect to the frequency component of each frequency channel obtained by performing the FFT. For example, if the pitch scaling value is 2, which is the maximum value in the premised range, the frame size is reduced to ½ by the pitch shift, so that the frame is doubled to compensate (maintain) the size. For this reason, the FFT size is set to twice the frame size. Details of the pitch scaling value calculation method will be described later.

  The FFT unit 24 receives a 256-sample frame from the LPF 23 and sets it in the first half of the FFT-sized frame. Set all 0s in the second half. The reason why 0 is set in the latter half is to bring about an interpolation effect in the frequency domain after executing FFT. The frequency resolution is improved to provide the interpolation effect. The FFT unit 24 performs FFT on the frame in which such a set has been performed.

  The IFFT unit 26 returns the frequency component of each frequency channel after the phase compensation unit 25 has expanded / contracted size to data in the time domain by performing IFFT (inverse FFT), and generates audio data for one frame. And output. The pitch shifter 27 performs interpolation or thinning for the frame generated by the IFFT unit 26 in accordance with the pitch scaling value input from the phase compensation unit 25, and shifts the pitch. A general Lagrangian function, a sinc function, or the like can be used for interpolation and decimation, but in this embodiment, pitch shift (pitch scaling) is performed by Neville interpolation. The frame size becomes the original size (256 samples) by the above interpolation or thinning. The audio data of the frame is hereinafter referred to as “synthetic audio data”, and the sound produced by the audio data is referred to as “synthesized audio”.

  The output buffer 29 is a buffer for storing synthesized voice data to be emitted from the sound system 11 as voice. The frame adder 28 adds the synthesized voice data for one frame input from the pitch shifter 27 to the synthesized voice data stored in the output buffer 29 by overlapping with the overlap factor OVL. The synthesized speech data stored in the output buffer 29 is output to the D / A converter 10 and D / A converted.

  The input buffer 21 and the output buffer 29 are areas secured in the RAM 5, for example. The units 22 to 28 except the A / D converter 8, the D / A converter 10, the input buffer 21, and the output buffer 29 execute the program stored in the ROM 4 by the CPU 1 using, for example, the RAM 5 as a work. It is realized with. Although a detailed description is omitted, the target pitch is instructed by operating the keyboard 2, for example. The target pitch may be specified by performance data such as a standard MIDI file or data received via a communication network.

Next, the calculation method of the pitch scaling value by the phase compensation unit 25 will be described in detail. Hereinafter, the scaling value is expressed as ρ.
By performing the FFT, a frequency component having a real component and an imaginary component is extracted for each frequency channel having a different frequency. When the real component is expressed as real and the imaginary component is expressed as img, the frequency amplitude mag and the phase phase of each frequency channel can be calculated as follows.

mag = (real ² + img ² ) ^1/2 (1)
phase = arctan (img / real) (2)
The phase phase calculated using arctan is limited to between −π and π. However, since the phase phase is an integral value of the angular velocity, it needs to be expanded. In order to easily distinguish the presence or absence of expansion, when the folded phase is expressed by a lowercase θ and the expanded phase is expressed by an uppercase Θ, it is originally Θ _{k, t} = θ _{k, t} + 2nπ n = 0 , 1, 2, ... (3)
It becomes. Therefore, the phase phase (= θ) needs to be developed by obtaining n. Here, k and t added as subscripts to Θ in the equation (3) represent the frequency channel index and time, respectively.

The expansion can be performed by the following procedure.
First, the phase difference Δθ between frames is calculated as follows.
Δθ _{i, k} = θ _{i, k} −θ _{i−1, k} (4)
Here, Δθ _{i, k} represents the phase difference between the previous frame and the current frame in the frequency channel k of the original speech waveform, and the subscript i represents the frame. The current frame (current frame) is represented by i, and the immediately preceding frame is represented by i-1.

Δθ _{i, k} in equation (4) is in a folded state. On the other hand, the central angular frequency Ω _{i, k} of the frequency channel k is represented by Ω _{i, k} = (2π · fs) · k / N (5) where fs is the sampling frequency and N is the number of FFT points (size).
Indicated by At the frequency Ω _{i, k} , assuming that the time difference from the previous frame is Δt, the phase difference ΔZ _{i, k} is ΔZ _{i, k} = Ω _{i, k} · Δt (6)
It can be calculated by The time difference Δt is Δt = N / (fs · OVL) (7)
It is. Since equation (6) is in a phase-expanded state, it can be described as follows.

ΔZ _{i, k} = Δζ _{i, k} + 2nπ (8)
If the difference between the phase difference Δθ _{i, k} calculated by equation (4) and the phase difference Δζ _{i, k} in equation (8) is δ (= Δθ _{i, k} −Δζ _{i, k} ), then Δθ _{i, k} − Ω _{i, k} · Δt = (Δζ _{i, k} + δ) − (Δζ _{i, k} + 2nπ)
= Δ-2nπ (9)
Can be derived. Therefore, if 2nπ on the right side of Equation (9) is deleted and the range is limited to between −π and π, δ can be calculated. The δ is a phase difference actually detected in the original speech waveform (hereinafter referred to as “actual phase difference”).

If the phase difference ΔZ _{i, k} (= Ω _{i, k} · Δt) is added to the actual phase difference δ calculated as described above, the phase difference ΔΘ _{i, k} that has been phase-expanded can be obtained as follows. .
ΔΘ _{i, k} = δ + Ω _{i, k} · Δt = δ + (Δζ _{i, k} + 2nπ) = Δθ _{i, k} + 2nπ
(10)
Ω _{i, k} · Δt in equation (10) can be modified as follows from equations (5) and (7).

Ω _{i, k} · Δt = {(2π · fs) / N} · k · {N / (fs · OVL)}
= (2π / OVL) · k (11)
In the discrete Fourier transform (DFT) including FFT, the frequency component leaks to all frequency channels (transition) except in a special case where the frequency of the frequency component included in the audio data (signal) is an integer multiple of the DFT point. Resulting in. Therefore, when analyzing the harmonic structure or the like of a signal, it is necessary to detect a frequency channel in which a frequency component actually exists from the result of DFT.

  For the detection, a method is generally adopted in which a peak of a frequency amplitude is detected and the peak is regarded as a channel in which a frequency component exists. The simplest method for that purpose is to regard a channel having a frequency amplitude larger than the frequency amplitudes of the two front and rear channels as a peak. However, in such a method, the peak due to the side lobe of the window function may be mistakenly recognized as a peak. For this reason, the channel having the smallest frequency amplitude is extracted from the found channels, and if the frequency amplitude is equal to or less than a predetermined value of the peak frequency amplitude (for example, −14 db of the peak frequency amplitude), it is regarded as a correct peak. Things are also done.

  Such peak detection can detect peaks with higher accuracy, but requires a two-step search and is complicated in processing. Therefore, in this embodiment, in order to reduce the processing load, peak detection is not performed, and a frequency channel in which a frequency component of harmonics of the original sound is present is detected in consideration of the phase as follows. .

The relationship between the developed phase difference and frequency is represented by a straight line graph. In this case, the vertical axis of the graph is the phase difference and the horizontal axis is the frequency, and the phase difference calculated from the center frequency of each channel, that is, ΔZ _{i, k} calculated by Equation (6) is represented by a straight line. . The line plotted along the straight line represents the phase difference ΔΘ _{i, k} calculated by the expression (10) of the voice having the harmonic structure, that is, the voiced sound. The phase difference ΔΘ _{i, k} corresponds to the first 128 points of 512 FFT points.

  In a voice having a harmonic structure, a line is stepped (flat) in the vicinity of a frequency channel having a frequency component of harmonics of the voice. This is because the frequency component of the frequency channel leaks to a nearby channel. For this reason, it is considered that the frequency component of the harmonic overtone exists in the frequency channel including the portion where the straight line intersects the stepped portion of the line. The frequency channel at the intersection (hereinafter referred to as “overtone channel”) can be calculated from Equation (10) and Equation (6), but the processing is somewhat complicated. Therefore, in the present embodiment, the harmonic channel is detected using the actual phase difference δ of Equation (9).

As described above, the actual phase difference δ is the difference between Δθ _{i, k in} equation (4) and Δζ _{i, k in} equation (8). This δ increases as the distance from the channel in which the frequency component actually exists increases, and decreases as the channel approaches. When crossing 0 when exceeding a channel and exceeding the frequency increasing direction, the absolute value increases toward the negative side as the channel is left.

By detecting a point where the actual phase difference δ crosses zero, a harmonic channel can be found. A zero crossing from positive to negative occurs even in a portion where adjacent harmonics intersect. For this reason, in the present embodiment, the frequency channel of the index k meeting the following condition (hereinafter referred to as “zero cross determination condition”) is adopted as the harmonic channel in which the harmonic frequency component exists. The frequency channel of index k is the frequency channel closest to the zero cross point.
δ [k−2]> δ [k−1]> δ [k]> δ [k + 1]> δ [k + 2]
By searching for the frequency channel k satisfying such a zero-cross determination condition, the frequency channel closest to the point where the zero-cross is greatly increased from positive to negative can be extracted with high accuracy as a harmonic channel. The extraction can be performed reliably even if the number of FFT points is not sufficient and it is difficult to extract a harmonic channel by frequency amplitude. When it is necessary to perform extraction with higher accuracy, peak detection may be performed together.

In the present embodiment, two frequency (overtone) channels k satisfying this determination condition are detected from the smaller frequency. This is because the higher the frequency, the greater the influence of errors and the lower the accuracy. The indices of the harmonic channels detected in this way are denoted as hm1 and hm2 from the lowest frequency. Hereinafter, hm1 is also referred to as a reference index, and a harmonic channel having the reference index hm1 is also referred to as a reference channel. The phase difference ΔΘ _{i, k} (k = hm1, hm2) of each harmonic channel is _obtained by adding Ω _{i, k} · Δt calculated by the equation (11) to the equation (10), that is, the actual phase difference δ of the channel. Calculated by

を再帰的に繰り返すことで算出することができる。式（１２）中の「ｘ ｍｏｄ ｙ」はｘをｙで割った余りを表している。最大公約数ｇｃｄ（ｘ、ｙ）は別の方法で算出してもよい。 The pitch scaling value ρ is calculated from the detection result of the harmonic channel as follows.
First, the greatest common divisor of the frequencies corresponding to the detected indexes hm1 and hm2 of the two overtone channels is obtained. The greatest common divisor can be calculated using the Euclidean algorithm. The greatest common divisor gcd (x, y) of two non-negative integers x and y is

Can be calculated by recursively repeating. “X mod y” in equation (12) represents the remainder of dividing x by y. The greatest common divisor gcd (x, y) may be calculated by another method.

In the present embodiment, human voice is assumed as the original voice. For this reason, the lower limit of the frequency that the original speech can take is 80 Hz, and the lower limit of the index value is 6 corresponding to the frequency. Accordingly, the condition of y = 0 in the formula (12) is y <6. The calculated greatest common divisor is expressed as x.
The greatest common divisor x can be obtained regardless of whether or not the frequency channel corresponding to the pitch (fundamental tone) can be extracted as a harmonic channel. For this reason, it is possible to reliably obtain even a musical sound in which the fundamental frequency called missing fundamental is missing or very small compared to other frequencies.
After calculating the greatest common divisor x, a multiple hmx is calculated that is a ratio of the common divisor x to the frequency corresponding to the reference index hm1. The multiple hmx is hmx = hm1 / x (13)
Is required. The multiple hmx obtained in this way corresponds to a value obtained by dividing the frequency corresponding to the reference channel by the fundamental frequency (frequency of the fundamental tone (pitch)).

The phase difference ΔΘ _d developed by the target pitch is calculated by multiplying the multiple hmx obtained by the equation (13). Assuming that the fundamental frequency of the target pitch is fd [Hz], their multiplication is ΔΘ _d · hmx = 2πfd · Δt · hmx
= (2πfd · hmx · N) / (fs · OVL) (14)
Can be performed. The pitch scaling value ρ for converting the pitch of the original voice into the target pitch is ρ = ΔΘ _d · hmx / ΔΘ _{i, hm1} (15)
It can be calculated by The phase compensation unit 25 in FIG. 2 calculates the scaling value ρ in this way and outputs it to the pitch shifter 27. Thereby, the pitch shifter 27 performs pitch scaling with the scaling value ρ, and shifts the pitch.

Further, the phase compensation unit 25 performs phase scaling by the following equation.
θ ′ _{i, k} = ΔΘ _{i, k} ((θ ′ _{i−1, hm1} −θ _{i−1, hm1} ) / ΔΘ _{i, hm1} + (ρ−1))
+ Θ _{i, k} (16)
In the equation (16), “′” is added to the phase difference obtained by scaling. By performing scaling according to the equation (16), the phase consistency on the time axis (HPC) and the phase relationship between channels, that is, between frequency components (VPC: Vertical Phase Coherence) are both preserved. (See Japanese Patent Application No. 2004-374090).

The phase compensator 25 calculates the real component real ′ and the imaginary component img ′ from the phase phase ′ after scaling according to the equation (16) and the frequency amplitude mag calculated from the equation (1) by the following Euler formula. Calculate and convert to complex frequency components.
real ′ = mag · cos (phase ′) (17)
img ′ = mag · sin (phase ′) (18)

  The IFFT unit 26 inputs the frequency component thus converted from the phase compensation unit 25 for each frequency channel, executes IFFT, and returns the data to the time domain. The pitch shifter 27 performs pitch scaling by interpolation or thinning on the frame generated by the IFFT unit 26 according to the pitch scaling value ρ input from the phase compensation unit 25. As a result, the data amount expands / contracts to 1 / ρ, but the phase compensator 25 performs phase scaling of ρ times (Equation (16)), so the expansion / contraction is canceled out, and the data amount becomes the original size. Will be maintained. Since the frame addition unit 28 overlaps the frames obtained in this way, the synthesized speech having the target pitch is emitted by the sound system 11.

Next, the operation of the karaoke apparatus of FIG. 1 will be described in detail with reference to various counters shown in FIG. 3, a flowchart of a singing scoring process by the CPU 1 shown in FIGS. 4 to 12, and FIGS. To do.
In the execution of the singing scoring processing program, 256 (= 2 ⁸ ) samples, which are powers of 2, are taken as one frame, and the pitch of the note-on of the accompaniment and the pitch of the voice signal of the singer from the microphone 7 are calculated. Singing ability is scored while detecting the pitch difference. Specifically, when karaoke starts, the difference between the pitch of the singer's voice signal and the pitch to be sung (reference value) is detected every 8 msec, and the detected pitch difference data is stored in the input buffer 21 of FIG. Accumulate. The accumulated pitch difference data is calculated every 256 msec, that is, an average pitch difference that is an average value of the accumulated values of the 32 pitch difference data. Next, the accumulated average pitch difference data is scored for each interval of about 4 sec (4096 msec).

  For this reason, as shown in FIG. 3, seven counters CNTA to CNTG prepared in the RAM inside the CPU 1 are used for the singing scoring process. CNTA is a counter that represents the number of times the pitch difference has been integrated. CNTB is a counter representing the number of times of pitch difference errors accumulated. CNTC is a counter that represents the number of times the pitch difference is calculated. CNTD is a counter that represents the number of times the average pitch difference has been integrated. CNTE is a counter that represents the number of times that the average pitch difference error has been integrated. CNTF is a counter indicating the number of times of calculating the average pitch difference. CNTG is a counter that represents the number of times the section scoring has been integrated.

FIG. 4 is a flowchart of the main routine of the CPU 1.
First, when the power is turned on, an initialization process is executed (step SA1). After step SA1, the loop processing from step SA2 to step SA4 is repeated. That is, a switch process corresponding to a user's operation on the switch constituting the switch unit 3 is executed (step SA2), a karaoke process is executed (step SA3), and a sound generation process, an effect process, a volume adjustment process, etc. Other processing is executed (step SA4).

  FIG. 5 is a flowchart of the switch process in step SA2 in the main routine. It is determined whether or not the song selection switch is turned on (step SB1). When this switch is turned on, the song number of the selected karaoke song is stored in the register SONG (step SB2). And the accompaniment music of the music number is searched from the music memory 2 (step SB3), the song section where the lyrics start is detected, and the time is set (step SB4). Specifically, the time from the start of the accompaniment to the start of the singing section, that is, the time of the intro is stored in the register and set. Then, the process returns to the main routine of FIG. When the accompaniment starts, the time set for each timer interrupt described later is decremented.

  If the song select switch is not on in step SB1, it is determined whether or not the start / stop switch is turned on (step SB5). If this switch is turned on, the flag STF is inverted (step SB6). Then, it is determined whether the STF is inverted to 1 (music start) or 0 (music stop) (step SB7). When STF is inverted to 1, the prohibition of the timer interrupt is canceled (step SB8). On the other hand, when STF is inverted to 0, timer interrupt is prohibited (step SB9). After the timer interrupt is canceled or prohibited, the process returns to the main routine of FIG.

  In step SB5, if the start / stop switch is not turned on, it is determined whether or not another switch is turned on (step SB10). For example, it is determined whether or not other switches such as an effect switch for adding echo and reverb sound effects and a volume switch for adjusting the volume are turned on. When another switch is turned on, processing corresponding to the switch is performed (step SB11), and the process returns to the main routine of FIG.

  FIG. 6 is a flowchart of the timer interrupt. It is determined whether or not it is a singing section to be uttered (step SD1), and if it is a singing section, it is determined whether or not the flag KF is 0 (step SD2). If KF is 0, KF is set to 1 (step SD3). In step SB8 of FIG. 5, after the prohibition of the timer interrupt is canceled, a timer interrupt is accepted at regular intervals. As a result, after the music starts, as described above, the intro time stored in the register is decremented for each timer interrupt. Therefore, KF is set to 1 when the first singing section is reached. If it is not a singing section in step SD1, it is determined whether or not KF is 1 (step SD4). When KF is 1, since the singing section has ended, KF is reset to 0 (step SD5). Then, the process returns to the main routine of FIG. Thereafter, KF is set to 1 every time a singing section is entered. If KF is 0 in step SD4, it means that the intro time has not yet elapsed, and the process returns to the main routine.

  After KF is set to 1 in step SD3 or when KF is 1 in step SD2, the value of the timer register T (initial value is 0) is incremented (step SD6). Then, it is determined whether or not the value of T has reached 8 msec (step SD7). When the value of T has reached 8 msec, the flag TF is set to 1 (step SD8). After setting TF to 1 or when the value of T has not reached 8 msec in step SD7, the process returns to the main routine of FIG.

FIG. 7 is a flowchart of the karaoke process in step SA3 in the main routine. First, it is determined whether or not KF is 1 (singing section) (step SC1). When KF is 0, the process returns to the main routine, but when KF is 1, a process of calculating a pitch difference is executed. (Step SC2).
FIG. 8 is a flowchart of a process for calculating the pitch difference. It is determined whether or not a flag TF indicating an elapsed time of 8 msec is 1 (step SE1). When TF is 0, this flowchart is ended. When TF is 1, a pitch ratio calculation every 8 msec is calculated. Processing is executed (step SE2).

  FIG. 9 is a flowchart of the pitch ratio calculation process. First, it is determined whether or not it is a sampling timing at which original audio data is output from the A / D converter 8 (step SF1). If it is the timing, the determination is YES, the original audio data is written to the input buffer 21 on the RAM 5 (step SF2), and it is determined whether or not it is the frame extraction timing (step SF3). If the time to sample the original voice data for the hop size has elapsed since the previous timing, the determination is YES, and the original voice data stored in the input buffer 21 is extracted for one frame, LPF (low-pass filter) processing for removing high-frequency components and FFT (fast Fourier transform) are sequentially performed on the extracted frame (step SF4).

Next, a phase compensation process is executed for the frequency components of each channel obtained by FFT (step SF5). After the phase compensation process, the IFFT (Fast Inverse Fourier Transform) for the frequency components of each channel subjected to the phase compensation process, and the pitch obtained by executing the time scaling process on the audio data for one frame obtained by the IFFT The shift is performed, and the synthesized voice data obtained by the pitch shift is overlap-added to the synthesized voice data stored in the output buffer 29 on the RAM 5 (step SF6).
The functions of the frame extraction unit 22, the LPF 23, and the FFT unit 24 shown in FIG. 2 can be realized as hardware, but in this embodiment, are realized by executing the process of step SF4. Similarly, the function of the phase compensation unit 25 is realized by executing the phase compensation process in step SF5. The functions of the IFFT unit 26, the pitch shifter 27, and the frame addition unit 28 are realized by executing the process of step SF6.

  Next, it is determined whether or not it is time to output synthesized audio data for one sampling (step SF7). If it is the timing, the determination is YES, and the synthesized voice data to be output is read from the output buffer 29 and sent to the D / A converter 10 via the musical tone generator 9 (step SF8). And this flowchart is complete | finished. Note that the tone generator 9 has a function of mixing the waveform data of the tone generated inside and the input data.

FIG. 10 is a flowchart of the phase compensation process of step SF5 in the pitch ratio calculation process of FIG. First, the frequency amplitude mag and the phase phase (= θ) are calculated from the frequency components of each frequency channel from the equations (1) and (2) (step SG1). Next, calculation of the developed phase difference ΔΘ _{i, k} according to equations (4) to (10) is started (step SG2), and before equation (10), which is the time when the actual phase difference δ is calculated, Two overtone channels are detected from the actual phase difference δ (step SG3). Next, it is determined whether or not the harmonic channel is 2 or more (step SG4). If it is 2 or more, the phase difference ΔΘ _{i, k} of each frequency channel is calculated by the equation (10), and the phase is calculated. The development is completed (step SG5). Next, a scaling value calculation process for calculating the scaling value ρ is performed on the two detected harmonic channels using the equations (12) to (15) (step SG6).

In the scaling value calculation process in step SG6 indicated by the dotted frame, the frequencies corresponding to the detected index values hm1 and hm2 of the two overtone channels are substituted into variables h1 and h2, respectively (step SG10). Here, the variables h1 and h2 correspond to x and y in Expression (12), respectively. Then, it is determined whether or not the index value corresponding to the value of the variable h2 is less than 6 (step SG11). When the index value is 6 or more, the remainder obtained by dividing the value of the variable h1 by the value of the variable h2 is assigned to the variable t, the value of the variable h2 is assigned to the variable h1, and further to the variable h2. The value of variable t is substituted (step SG12). In step SG11, it is determined again whether the index value is less than 6. That is, until the index value corresponding to the value of the variable h2 becomes less than 6, the greatest common divisor between the frequencies corresponding to the index values hm1 and hm2 is substituted into the variable h1 according to the equation (12). When the index value corresponding to the value of the variable h2 is less than 6, the value obtained by dividing the frequency corresponding to the index value hm1 by the value of the variable h1 (the greatest common divisor) is substituted into the variable hmx according to Equation (13). (Step SG13). Then, the equation (14), multiplied by the value of the variable hmx the phase difference .DELTA..theta _d, calculates the scaling value ρ by formula (15) using the result of the multiplication (step SG14).

In this case, two overtone channels are extracted, but three or more overtone channels may be extracted. When peak detection is also performed, two or more overtone channels may be extracted in consideration of the magnitude of the frequency amplitude from the overtone channels extracted by paying attention to the actual phase difference. Good.
The formant moves with the pitch shift. For this reason, the synthesized speech becomes unnatural as the shift amount (scaling value ρ) increases. In order to avoid this, formant compensation may be performed together.

Further, since the pitch shift to the target pitch can be realized without extracting the fundamental frequency of the original voice, the fundamental frequency is not extracted. However, the fundamental frequency can be extracted using a multiple hmx. In the extraction (calculation), when the fundamental frequency is expressed as fi, the equation (7) is used.
fi = ΔΘ _{i, hm1} / (2π · Δt · hmx)
= (ΔΘ _{i, hm1} · fs · OVL) / (2π · N · hmx) (19)
Can be performed. When the target pitch is specified by frequency, the scaling value ρ may be obtained by calculating the fundamental frequency fi and then taking the ratio with the frequency of the target pitch. Further, the calculated fundamental frequency fi may be notified to the user by the display unit 6 or the like. Another method may be employed for generating the synthesized speech waveform.

After the scaling value is calculated in step SG6, the phase scaling process according to equation (16) is performed using the phase difference ΔΘ _{i, k} (step SG7). Next, the real number component real ′ (formula (17)) and the imaginary number component img ′ (formula (18)) are calculated from the phase phase ′ and the frequency amplitude mag calculated from the formula (1). Conversion is performed (step SG8). In step SG4, when the harmonic channel is not 2 or more, it is determined as an error and the error flag is activated (step SG9). After performing complex number conversion in step SG8 or after determining an error in step SG9, the phase compensation process is terminated.

  After the pitch ratio calculation process of FIG. 9, the process proceeds to step SE3 of the pitch difference calculation process of FIG. 8, and the determination result of whether or not the harmonic channel is 2 or more is referred to in step SG4 of FIG. The process branches depending on whether it is determined that the error is not 2 or more, or the harmonic channel is not 2 or more and is determined to be an error. When the channel in which the fundamental tone exists is 2 or more and is not determined to be an error, a value obtained by subtracting “1.0” from the calculated pitch ratio is stored as a pitch difference (step SE4).

  The calculated pitch ratio is the ratio of the pitch of the reference value (reference pitch) to the pitch of the audio signal input from the microphone 7 (input audio pitch). Therefore, when the two pitches match, the pitch ratio ( Input voice pitch) / (reference pitch) is a value of “1.0”. Therefore, the value obtained by subtracting “1.0” from the pitch ratio is positive when the input voice pitch is higher than the reference pitch, is negative when the input voice pitch is lower than the reference pitch, and the pitch difference includes a positive / negative sign. become.

  After calculating the pitch difference in step SE4, it is determined whether the count value of CNTA, which is the number of times the pitch difference is integrated, is less than 16 or more than 16 (step SE5). If the count value of CNTA is less than 16, the pitch difference is stored in the buffer and integrated (step SE6). Then, the count value of CNTA is incremented (step SE7). Originally, the average value of pitch difference data is calculated every 256 msec. That is, the average value is calculated every time the pitch difference data of 32 times are integrated. However, in this embodiment, in order to reduce the load on the CPU 1, the number of frames is set to a power of 2, and the average value is calculated every time 16 pitch difference data up to half of the number of frames are integrated.

  When it is determined in step SE3 that the channel in which the fundamental tone is present is 1 or less and an error has occurred, the value of the counter CNTB of the pitch difference error integration count is incremented (step SE8). After incrementing the value of CNTB, or after incrementing the value of CNTA in step SE7, the value of the counter CNTC of the number of pitch difference calculations is incremented (step SE9). After the value of CNTA becomes 16 in step SE5, the value of CNTA is not incremented, but the value of the counter CNTC for calculating the number of pitch differences is incremented (step SE9). As a result, a maximum of half of the pitch difference data other than the error is discarded, but in the extremely short time of 8 msec, even if half of the pitch difference data is lost, there are many. There is no effect. Next, the flag TF is reset to 0 (step SE10), and the process proceeds to the difference average calculation process of step SC3 in FIG.

FIG. 11 is a flowchart of the difference average calculation process executed every 256 msec.
First, it is determined whether or not the value of the counter CNTC for calculating the number of pitch differences every 8 msec has reached 32, which is the number of frames (step SH1), and after the value of CNTC reaches 32, CNTB is 16 It is determined whether or not it is larger (step SH2). That is, it is determined whether or not the pitch difference error is greater than half the number of frames. If CNTB is 16 or less, the integrated value is shifted to the right to calculate the average value of the bit differences (step SH3). Since the number of frames is a power of 2, 32, half of the number of frames is also a power of 2. Therefore, the average value of the integrated 16 bit differences is calculated by a 4-bit right shift instead of dividing the integrated value. As a result, the calculation processing of the average value calculation of the CPU 1 can be reduced, and a bottleneck of sound generation processing such as sound interruption called “moke of performance” can be avoided.

After the shift process in step SH3, it is determined whether or not the value of the counter CNTD of the number of times of average pitch difference integration is less than 8 (step SH4). As described above, since the accumulated average pitch difference data every 256 msec is scored every 4096 msec, the average pitch difference per section is 16 (= 4096/256) data. However, also in this case, in order to reduce the load on the CPU 1, scoring is performed based on the data of eight average pitch differences. When the value of CNTD is less than 8, the average pitch difference is integrated (step SH5), and the value of CNTD is incremented (step SH6).
In step SH2, if the value of CNTB is larger than 16, that is, if the number of pitch difference errors exceeds half of 32 which is the number of frames, the value of counter CNTE of the average number of times of pitch difference error is incremented. (Step SH7). Then, the error value is set as the average value of the section (step SH8).

  After incrementing the value of CNTD in step SH6, incrementing the value of CNTE in step SH8, or after the value of CNTD reaches 8 in step SH4, the value of the counter CNTF of the number of times of calculating the average pitch difference is set. Increment (step SH9). And it transfers to step SC4 of the karaoke process of FIG. 7, a pitch shift | offset | difference is displayed, it transfers to the following step SC5, and a section score calculation process is performed.

FIG. 12 is a flowchart of the interval score calculation process executed about every 4 seconds.
First, it is determined whether or not the value of CNTF, which is the number of times the average pitch difference is calculated every 256 msec, has reached 16 (step SJ1). That is, it is determined whether or not the average pitch difference calculation count has reached the maximum count of about 4 sec (4096 msec), which is one section. When the CNTF value is less than 16, this flowchart is terminated. However, when the CNTF value is 16, the value of the average pitch difference error integration counter CNTE is half the average pitch difference calculation count ( It is determined whether or not the number of allowable errors is greater than 8 (step SJ2). If the value of CNTE is 8 or less, the integrated value of the average pitch difference is shifted to the right to calculate the average value (step SJ3). As shown in step SH4 in FIG. 11, the number of integrations, which is the value of CNTD, is 8, so an average value of 8 average pitch differences is calculated by right shifting by 3 bits.

  Next, parameters are set for each of the beginner, intermediate and advanced levels of the singer (step SJ4). The parameter to be set is a parameter t that defines a permissible range of the reference value with a singing ability of 100 points of the maximum evaluation value, and a parameter that defines a degree of singing ability outside the permissible range in the range of 100 points to 0 points. a. After setting the parameters, the section score is calculated (step SJ5). In step SJ2, when the value of CNTE exceeds 8 and exceeds the allowable number of errors, the section score is set to 0 (step SJ6). After calculating the section score in step SJ5, or after setting the section score to 0 in step SJ6, the section scores are integrated (step SJ7). Then, the value of the counter CNTG of the number of times of section scoring is incremented (step SJ8), the pitch deviation is displayed (step SJ9), and the process returns to the karaoke process of FIG.

最初にｘの絶対値をとるのは、図８のステップＳＥ５で求めた平均ピッチ差分が正負の符号を含んでいるので、正の値の領域だけで計算を行うためである。
また、ピッチの最小単位である半音のピッチが１００セントであり、１オクターブが１２００セントであるので、得点が１００点となるピッチ範囲をdiff_t[セント]とすると、パラメータｔは、下記の式（２１）で表される。

得点が０点となるピッチ差分値をdiff_a[セント]とすると、パラメータａは、下記の式（２２）で表される。

In the calculation of the section score in step SJ5, assuming that the average pitch difference from the reference value is x and the section score is grade, an arithmetic expression for the section score is expressed by the following formula (20).

The reason why the absolute value of x is first taken is that the average pitch difference obtained in step SE5 in FIG. 8 includes a positive / negative sign, so that the calculation is performed only in the positive value region.
Also, since the semitone pitch, which is the minimum unit of the pitch, is 100 cents and one octave is 1200 cents, if the pitch range where the score is 100 points is diff_t [cents], the parameter t is expressed by the following formula ( 21).

If the pitch difference value at which the score is 0 is diff_a [cent], the parameter a is represented by the following equation (22).

  FIG. 13 is a diagram illustrating specific examples of the parameter t and the parameter a at each level of the beginner level, intermediate level, and advanced level. As shown in FIG. 13, in the case of a beginner singer, the pitch difference from the reference value is 100 points even when the pitch difference is 40 cents. On the other hand, in the case of an advanced singer, it is not 100 points unless the pitch difference from the reference value is within 20 cents. In the case of a beginner singer, the pitch difference from the reference value is 240 cents and 0 points, whereas in the case of an advanced singer, the pitch difference from the reference value is 120 cents and 0 points. . In the case of an intermediate singer, the parameter t, which is 100 points, and the parameter a, which is 0 points, are approximately between the beginner level and the advanced level.

  FIG. 14 is a diagram illustrating the characteristics of the formulas (20) to (22) that are the calculation formulas of the section scores. In FIG. 14, the horizontal axis represents the pitch difference, and the vertical axis represents the section score. As shown in FIG. 14, the characteristic of the section score with respect to the pitch difference has a trapezoidal shape. The upper side of the trapezoid defines the allowable range of pitch difference with a singing ability of 100 points, and the slope of the trapezoid defines the degree of singing ability outside the allowable range in the range from 100 points to 0 points. For beginners, the upper side of the trapezoid is longer and the slope is gentler. On the other hand, in the case of an advanced singer, the upper side of the trapezoid becomes shorter and the slope becomes steeper. For intermediate singers, the trapezoid is an intermediate shape between beginner and advanced.

  In the karaoke process of FIG. 7, after the calculation of the section score in step SC5, the section score is displayed (step SC6). FIG. 15 is a diagram showing a screen that displays the section score and the pitch deviation. As shown in FIG. 15, the pitch deviation is displayed by lighting up the corresponding LED with three LEDs having different emission colors in a high pitch state, a pitch match state, and a low pitch state. In step SC6, after the section score is displayed, it is determined whether or not the karaoke is stopped (step SC7). If the karaoke is not stopped, the process proceeds to step SC2 and the processes up to step SC6 are performed. repeat.

  When the karaoke song ends or the start / stop switch is turned on and it is determined in step SC7 that the karaoke is stopped, a total score is calculated (step SC8). The total score is calculated by dividing the integrated value of the section score calculated in step SJ7 in FIG. 12 by the final CNTG value incremented in step SJ8, that is, the number of times of section score integration. At the time of calculating the total score, the karaoke performance is stopped and the CPU 1 is released from the sound generation process, so the total score is calculated not by the shift process but by the division process. After calculating the total score, the total score is displayed (step SC9), the flag KF is reset to 0 (step SC10), the flag STF is reset to 0 (step SC11), and the main routine of FIG. Return.

  FIG. 16 is a diagram illustrating the relationship between the utterance section of the audio signal input from the microphone 7 and the singing section to be evaluated. As shown in FIG. 16, when an utterance is made before reaching the singing section in the accompaniment, the utterance is not determined as a determination and is not added to the score. On the other hand, if the utterance is not performed in spite of the singing section, the score is 0.

As described above, according to the embodiment, the CPU 1 is out of the allowable range in the parameter t that defines the allowable range of the reference value at which the singing ability to be evaluated is 100 points, and in the range of 100 points to 0 points. The parameter a that defines the degree of singing ability is set, and the evaluation value of the input audio signal is calculated based on the parameter t and the parameter a. In this case, the CPU 1 is configured to calculate the evaluation value of the pitch of the audio signal and the utterance timing, but may be configured to calculate one of the evaluation values.
Therefore, without requiring a memory for storing a large number of evaluation values or a calculation process for calculating an average of a large number of evaluation values after the music is finished, singing ability can be quickly and accurately performed at low cost. Can be evaluated.

Further, according to the embodiment, the CPU 1 calculates the difference between the pitch of the input audio signal and the reference value, integrates the difference including the sign, and determines the evaluation value of the pitch of the audio signal as the parameter t and the parameter a. Calculate based on
Therefore, even when an advanced singer has a fluctuation in pitch due to the vibrato method, the singing ability can be properly evaluated without considering the fluctuation of the pitch as a pitch shift.

Moreover, according to the said embodiment, CPU1 searches the song area of the input accompaniment, calculates the evaluation value of the audio | voice signal input into a song object area, and is input outside a song area. Signals are not subject to evaluation.
Therefore, the singing ability can be scored accurately.

Further, according to the above embodiment, the CPU 1 analyzes the number of errors in each frame with the number of samples of powers of 2 in the input audio signal as one frame, and the analyzed number of errors is ½ of one frame. If it exceeds, the frame is regarded as 0 points. If the number of analyzed errors does not exceed half of one frame, the frame is evaluated based on the number of samples other than errors corresponding to half of one frame. Calculate the value.
Therefore, by calculating the average value of the pitch differences by shift processing instead of division processing, it is possible to prevent the CPU 1 from being heavily loaded. As a result, it is possible to avoid a bottleneck in sound generation processing such as sound interruption.

Further, according to the embodiment, the CPU 1 detects the greatest common divisor of at least two or more pitches from the input audio signal, analyzes the pitch of the fundamental tone, and calculates the evaluation value of the detected fundamental pitch. calculate. In this case, the CPU 1 calculates the phase from the frequency component of the input audio signal, and detects the greatest common divisor of at least two or more pitches from the audio signal using the calculated phase.
Therefore, the pitch of the fundamental tone of the input singer's voice signal can be reliably detected.

  Furthermore, the overtone has a frequency that is an integral multiple of the frequency of the fundamental tone (pitch). As a result, the greatest common divisor between frequencies corresponding to two or more frequency channels (harmonic channels) in which harmonic frequency components exist can be handled as information representing the frequency of the fundamental tone. For this reason, as shown in the scaling value calculation processing of FIG. 10, the fundamental tone of hm1 that is the first speech waveform is used as the target fundamental tone with high accuracy using the greatest common divisor of two or more frequency channels. It is possible to generate hm2, which is the converted (shifted) second speech waveform. Since the necessity of extracting (detecting) the fundamental tone of the first speech waveform is avoided, the first speech waveform in which the fundamental frequency called missing fundamental is missing or very small compared to other frequencies. However, the second speech waveform having the target fundamental tone can be reliably generated. Further, by using the greatest common divisor, the frequency of the fundamental tone of the first speech waveform can also be reliably extracted (detected).

Next, a modification of the above embodiment will be described.
In the above embodiment, scoring for one octave of 1200 cents is performed, and the same note name with different octaves is not considered, but the octave difference is detected and folded so that the difference falls within the octave range. You may make it the structure which scores.
In addition, when calculating the total score, in the case of a beginner or intermediate singer, a bonus point may be added and scored. For example, the highest score of the section score is held, and when the total score is calculated, the held highest score is multiplied by a coefficient corresponding to the level of beginner or intermediate and added to the total score.
In the above embodiment, the number of data accumulated at the time of average pitch difference and section scoring is halved of the number of frames, but the calculation for calculating the average value can be performed by shift processing instead of division. In this way, the number of frames is not necessarily half as long as it is a power of 2. It may be 1/4 or 1/8. As the denominator increases, the number of data to be discarded increases. However, any ratio that provides the reliability of scoring is acceptable. In general, in a short section such as 4 sec, the singing power fluctuation is very small regardless of the level of the singer, so that the reliability of the scoring can be obtained even with a data number of 1/8 or less.
Moreover, in the said embodiment, although it was set as the structure which calculates a pitch difference and scores a song power, you may make it the structure which calculates the difference of the timing of pronunciation and scores a song power.

  In the above-described embodiment, the invention of the apparatus in which the CPU 1 executes the singing scoring program stored in advance in the ROM 4 has been described. However, the program is stored in an external storage medium such as a flexible disk (FD), CD, or memory card. It is also possible to install the singing scoring processing program downloaded or the singing scoring processing program downloaded from a network such as the Internet in the RAM 5 or a non-volatile memory such as a separately provided flash ROM, and the CPU 1 can execute the program. It is. In this case, the invention of the program and the invention of the storage medium can be realized.

In other words, the singing scoring program of the present invention is:
A first parameter that defines an allowable range of a reference value in which the singing ability to be evaluated is a maximum evaluation value, and a second parameter that defines a degree of singing ability outside the allowable range in the range from the maximum evaluation value to the minimum evaluation value. The computer is caused to execute step A for setting and step B for calculating the evaluation value of the input audio signal based on the first parameter and the second parameter set in step A.

The step B is characterized in that at least one evaluation value is calculated from the pitch and utterance timing of the input audio signal.
In this case, the step B calculates the difference between the pitch of the input audio signal and the reference value, integrates the difference including the sign, and sets the evaluation value of the pitch of the audio signal as the set first value. And calculating based on the second parameter and the second parameter.

  It further has a step C for searching for the singing section of the input accompaniment, and the step B calculates an evaluation value of the audio signal input in the singing target section searched by the step C, and the singing section An audio signal input outside is not subject to evaluation.

  The method further includes a step D of analyzing the number of errors in each frame with the number of samples of powers of 2 in the input speech signal as one frame, and the step B includes the number of errors analyzed in the step D of 2 in one frame. If the number of errors exceeds one half, the corresponding frame is regarded as the lowest evaluation value. If the number of analyzed errors does not exceed one half of one frame, the number of samples other than errors corresponding to one half of one frame An evaluation value of the frame is calculated.

  The step D analyzes the pitch of the fundamental tone by detecting the greatest common divisor of at least two or more pitches from the input speech signal, and the step B is a step of analyzing the pitch of the fundamental tone detected by the step D. An evaluation value is calculated.

  The step D is characterized in that the phase is calculated from the frequency component of the input audio signal, and the greatest common divisor of at least two or more pitches is detected from the audio signal using the calculated phase. .

1 CPU
2 song memory 3 switch part 4 ROM
5 RAM
6 Display Unit 7 Microphone 8 A / D Converter 9 Musical Sound Generation Unit 10 D / A Converter 11 Sound System

Claims (6)

A first parameter that defines an allowable range of a reference value in which the singing ability to be evaluated is a maximum evaluation value, and a second parameter that defines a degree of singing ability outside the allowable range in the range from the maximum evaluation value to the minimum evaluation value. Parameter setting means to be set;
Signal analysis means for analyzing the number of errors in each frame with the number of samples of powers of 2 in the input speech signal as one frame;
When the number of errors analyzed by the signal analysis means exceeds half of one frame, the frame is set as the lowest evaluation value, and when the number of errors analyzed does not exceed half of one frame, 1 Evaluation calculation means for calculating the evaluation value of the frame based on the first parameter and the second parameter set by the parameter setting means by the number of samples other than the error corresponding to one half of the frame;
Singing scoring device with The signal analysis means detects the greatest common divisor of at least two or more pitches from the input audio signal and analyzes the pitch of the fundamental tone,
The singing scoring apparatus according to claim 1, wherein the evaluation calculation unit calculates an evaluation value of a pitch of a fundamental tone detected by the signal analysis unit.   The signal analysis means calculates a phase from a frequency component of an input audio signal, and uses the calculated phase to detect a greatest common divisor of at least two or more pitches from the audio signal. The singing scoring device according to claim 2. A first parameter that defines an allowable range of a reference value in which the singing ability to be evaluated is a maximum evaluation value, and a second parameter that defines a degree of singing ability outside the allowable range in a range from the maximum evaluation value to the minimum evaluation value Step A to set,
A step B of analyzing the number of errors in each frame by setting the number of power-of-two samples in the input audio signal as one frame;
When the number of errors analyzed in step B exceeds 1/2 of one frame, the frame is set as the lowest evaluation value, and when the number of errors analyzed does not exceed half of one frame, one frame. A step C for calculating an evaluation value of the frame based on the first parameter and the second parameter set in the step A by the number of samples other than the error corresponding to one half of
A singing scoring program that causes a computer to execute.   The step B analyzes the pitch of the fundamental tone by detecting the greatest common divisor of at least two or more pitches from the input audio signal, and the step C is a step of analyzing the pitch of the fundamental tone detected by the step B. An evaluation value is calculated, The singing scoring program according to claim 4.   The step B is characterized in that the phase is calculated from the frequency component of the input audio signal, and the greatest common divisor of at least two or more pitches is detected from the audio signal using the calculated phase. The singing scoring program according to claim 5.

Images