JPH08115097A - Acoustic reproduction device - Google Patents

Abstract

PURPOSE: To provide an acoustic reproduction device in which the tempo and the interval of an accompaniment are compensated in accordance with the interval of a singer even when there exists a tempo difference, which is within an allowable range, between the voice of the singer and the accompaniment. CONSTITUTION: The device consists of a first interval extracting means 3 which extracts the interval of a first inputted voice, a second interval extracting means 6 which extracts the interval of a second inputted voice, storage means 4 and 7 which store the time histories of each interval, a computing means 8 which computes the differences in tempo and interval between the first and the second inputted voices employing the time histories of the intervals and a nonlinear pattern matching method, compensating means 9 and 10 which compensate the tempo and the interval of the second inputted voice so that they are made approximately equal to the tempo and the interval of the first inputted voice and a means 13 which reproduces the compensated second inputted voice that is compensated by the means 9 and 10 or a third inputted voice that is compensated for its tempo and interval differences.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound reproducing apparatus for correcting the tempo and pitch of an accompaniment sound in accordance with the tempo and pitch of the voice of a singer, for example.

[0002]

2. Description of the Related Art Heretofore, a pitch adjuster has been connected to a sound reproducing device such as a karaoke device to change only the pitch without changing the tempo of the accompaniment tune.

Regarding the adjustment of the pitch of the accompaniment sound of the sound reproducing device, as shown in Japanese Patent Laid-Open No. 5-35286, the pitch of the voice of the singer within a preset time even during the accompaniment sound. There is a method of detecting a pitch difference between the pitch of the accompaniment sound and the pitch of the accompaniment sound, and then automatically correcting the pitch of the accompaniment sound so as to approximate the pitch of the singer.

[0004]

However, in the conventional method of automatically correcting the pitch of the accompaniment sound so as to approximate the pitch of the voice of the singer, the time difference between the voice of the singer and the accompaniment sound is increased. That is, it is premised that there is no tempo difference between the singer's voice and the accompaniment sound, and in this situation, if a tempo difference occurs between the singer's voice and the accompaniment sound, There was a problem that the pitch was erroneously corrected.

Therefore, the present invention has been made in view of the above-described problems, and even if there is a tempo difference within a permissible difference between the voice of the singer and the accompaniment sound,
An object of the present invention is to provide a sound reproducing device that corrects the tempo and pitch of the accompaniment sound.

[0006]

A sound reproducing apparatus according to the present invention comprises a first pitch extracting means for extracting a pitch of a first input voice, and a second pitch extracting means for extracting a pitch of a second input voice. , A tempo difference and a pitch difference between the first input voice and the second input voice by a non-linear pattern matching method using a storage means for storing the time history of each pitch and the time history of the pitch. A calculating means for calculating, a correcting means for correcting the tempo and pitch of the second input voice to approximate the tempo and pitch of the first input voice, and the second input voice or the second input voice corrected by the correcting means, Means for playing back the third input sound in which the calculated tempo difference and pitch difference are corrected. In addition, the nonlinear pattern matching method has a DP
Using the matching, the slope of the time normalization function obtained from the DP matching is used to detect the first input voice and the second input voice.
An average calculated from the time history of the pitch of the second input voice and the time history of the pitch of the first input voice corrected according to the tempo difference. It is characterized in that the pitch difference is calculated as the pitch difference between the first input voice and the second input voice.

Further, the slope of the time normalization function obtained from the DP matching is limited so that the tempo difference of the first input voice with respect to the tempo of the second input voice is not corrected to a set value or more. And

Further, a correction period for correcting the tempo and pitch of the second input voice to approximate the tempo and pitch of the first input voice is set, and after the correction period, the tempo difference and pitch are set. The second input voice or the third input voice is reproduced so that the difference correction amount is kept constant.

On the other hand, a hand switch for instructing the presence / absence of correction of the tempo and pitch of the second input voice or the third input voice is provided, and the operator can use the hand switch to perform the correction period. The feature is that you can specify.

[0010]

The sound reproducing apparatus of the present invention receives a command to start correction so as to correct the tempo and pitch of the second input voice or the third input voice according to the operator's instruction, and firstly, the first The pitches of the input voice and the second input voice are extracted, and the time history of the pitch of each of the voices for a certain period is stored.

Next, using the time history of the pitch, the tempo difference and the pitch difference between the first input voice and the second input voice are calculated by the non-linear pattern matching method.

D is applied to such a nonlinear pattern matching method.
When P matching is used, the tempo difference between the first input voice and the second input voice is calculated from the slope of the time normalization function obtained from the DP matching. At this time, if the slope of the time normalization function is within a certain allowable range set in advance, the slope is calculated as the tempo difference between the first input voice and the second input voice, and if it is outside the allowable range. For example, it is determined that the tempo difference cannot be corrected.

On the other hand, if the tempo difference is within the allowable range, the time history of the pitch of the second input voice is approximated to the time history of the pitch of the first input voice by the time normalization function. The average pitch difference calculated from the corrected time history of the pitch of the second input voice and the corrected time history of the pitch of the first input voice is corrected between the first input voice and the second input voice. It is calculated as the pitch difference of.

Finally, the second input voice is corrected by the calculated tempo difference and pitch difference so as to approximate the tempo and pitch of the second input voice to the tempo and pitch of the first input voice. The third input sound is reproduced or is corrected by the calculated tempo difference and pitch difference so that the tempo and pitch of the third input sound are approximated to the tempo and pitch of the first input sound. Reproduce.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a schematic diagram of the case where the sound reproducing device according to the present invention is applied to a karaoke device. Usually, the voice of the singer is converted into an analog voice signal via the microphone 1 and transmitted to the mixing circuit 12. On the other hand, the musical sound information recording medium 5 has a MIDI (Musical Instrument Dig.
A digital accompaniment sound signal called ital interface) information is stored. The digital accompaniment sound signal passes through the tempo difference correction circuit 9 and the pitch difference correction circuit 10, but is not corrected, and the D / A It is converted into an analog accompaniment sound signal by the converter 11 and transmitted to the mixing circuit 12. In the mixing circuit 12, the analog accompaniment sound signal and the analog audio signal are mixed and amplified, and reproduced by the speaker 13 as a sound in which the voice of the singer and the accompaniment sound are mixed.

Now, a case where the tempo and pitch of the accompaniment sound are corrected and reproduced by the operation of the singer will be described with reference to the processing procedure shown in FIG.

In this embodiment, the first input voice is the voice of the singer, the second input voice is the teacher sound signal serving as the model of the singer stored in the musical tone information storage medium 5, and the third input voice. Will be described as an accompaniment sound signal stored in the musical sound information storage medium 5.

In step S1, the singer operates the hand switch attached to the microphone 1, whereby the karaoke apparatus according to the present invention receives a correction start command for correcting the accompaniment sound.

Immediately thereafter, the analog voice signal of the singer (which is the first input voice) is converted into a digital voice signal by the A / D converter 2 separately from the above-mentioned normal signal path. The sampling frequency of the A / D converter 2 at the time of A / D conversion is 2 kHz, and the audio signal passes through a low-pass filter having a cutoff frequency of 1 kHz before sampling.

Then, in step S2, the digital voice signal is sequentially time-sequentially output by the pitch extraction circuit 3.
The pitch of the voice of the singer is calculated using a signal processing technique such as the autocorrelation method, and the time history of the pitch of the singer is stored in the buffer memory 4. In the embodiment, the pitch extraction circuit 3
Extracts a pitch averaged every 20 msec of the measurement time, and the buffer memory 4 stores a time history of the pitch of the singer having a time length of 6 sec.

On the other hand, in the tone information storage medium 5, as described above, a sample of a singer called MIDI information (which is the second input voice) in a state where signals such as pitch, tone color and volume are separated in advance. The teacher sound signal and the digital accompaniment sound signal (which is the third input sound) are stored. Therefore,
The pitch of the teacher sound is extracted by the pitch extraction circuit 6, and the time history of the pitch of the teacher sound is stored in the buffer memory 7 with a time length of 5 sec. The time history of the pitches of the teacher's notes stored in the buffer memory 7 is thinned out or linearly interpolated so as to have an average pitch every 20 msec of the measurement time, like the pitcher's time history of the singer.

Therefore, the buffer memory 4 stores the time history of the pitch of the singer for 6 seconds calculated from the correction start command, and the buffer memory 7 stores the teacher for 5 seconds calculated from the correction start command. The time history of the pitch of the sound is stored, and the non-linear pattern matching circuit 8 is used to calculate the tempo difference and the pitch difference between the voice of the singer and the teacher's sound by using the time history of each pitch. .

Next, a method of calculating the tempo difference and the pitch difference in step S3 will be described. In this embodiment,
The tempo difference and the pitch difference between the voice of the singer and the teacher's sound are calculated by the DP matching method. The DP matching method is described in Journal of Acoustical Society of Japan, Vol.27 No.9 pp483-487, and in the present embodiment, the pitches of the singer's pitch and the pitch of the teacher's pitch are both recorded. The absolute value of the difference is 5se calculated from the time when the correction start command is issued.
At each time point between c, a time axis normalization function that minimizes the cumulative pitch difference up to that time point is obtained.

In the embodiment, the interval time history of the teacher sound is 5
Since it is stored for sec, and the pitch measurement time is in 20 msec unit, it is represented as time series data of 250 pitches A ₁ , A ₂ , ..., A ₂₅₀ . Similarly, since the singer's pitch time history is stored for 6 seconds, it is represented as time series data of the pitches B ₁ , B ₂ , ..., B ₃₀₀ . FIG. 3 shows a calculation example of the time axis normalization function by the DP matching method.
The time-axis normalization function starts from the grid point (A ₁ , B ₁ ),
From this starting point, the minimum value of the cumulative pitch difference at all grid points is obtained as the cumulative distance, and the path from the grid point (A ₂₅₀ , B _tr ) at which the pattern matching is terminated becomes smaller in the reverse time direction. Ask. Incidentally, B
_tr means the pitch data of the singer corresponding to the pitch data A ₂₅₀ of the teacher sound.

In FIG. 3, the abscissa represents the pitch time history of the teacher's tone and the ordinate represents the pitch time history of the singer, which is composed of the coordinates called grid points. Here, the time axis normalization function 31 uses the DP matching method to calculate the line segment 3
Of the cumulative distance that minimizes the cumulative pitch difference from the grid point (A ₁ , B ₁ ) at each grid point from (A ₂₅₀ , B ₂₀₀ ) to (A ₂₅₀ , B ₃₀₀ ) in 5 above. , The grid point (A ₂₅₀ , B _tr ) having the smallest value of the cumulative distance is used as a starting point, and the route is calculated by tracing the path in which the cumulative distance becomes smaller in the time reverse direction. In the embodiment, the singer's pitch measurement data B ₂₀₀ is 4 seconds after the correction start command, and the singer's singing immediately after the start of the correction command is delayed with respect to the teacher sound by the obtained time axis normalization function 31. However, it can be seen that, despite the fact that the teacher's sound is five seconds after the start of the correction command, the song is finished singing quickly. In FIG.
The line segment 32 is a line segment having a slope of 1 passing through (A ₁ , B ₁ ) and (A ₂₅₀ , B ₂₅₀ ). If the time axis normalization function matches the line segment 32, the teacher sound and the voice of the singer are There is no time lag, that is, tempo difference. In contrast, line segment 33
_{((A 25 0, B 300} ) of a line segment of street slope 1) and, line segment 3
4 (a line segment that passes through (A ₂₅₀ , B ₂₀₀ ) and has a slope of 1) forms a coordinate area called a time-axis matching window, and the time-axis normalization function If it is not correct, it is considered unrealistic and the tempo difference is not corrected. However, in the embodiment, the time-axis normalizing function is forced to maximize the accumulation within the time-matching window if an attempt is made to exceed the range of the time-matching window when searching for a route in the reverse time direction as described above. Since another grid point having a small distance is selected, the time axis normalization function is always found within the time axis matching window range. In the embodiment, an approximate straight line 36 is obtained for the time axis normalization function 31 by the method of least squares, and the inclination of the straight line 36 is used as the tempo difference correction amount.

Next, the processing in the tempo difference correction circuit 9 in step S5 will be described. The tempo difference correction circuit 9 uses the tempo difference correction amount obtained by the non-linear pattern matching circuit 8 and sets the value obtained by dividing the normal reproduction speed of the accompaniment sound by the tempo difference correction amount as the corrected reproduction speed of the accompaniment sound. In the example, a tempo difference correction amount of 0.97 was obtained.

Finally, the pitch difference correction in step S6
The processing in the circuit 10 will be described. Pitch difference correction circuit 10
Then, using the non-linear pattern matching circuit 8 shown in FIG.
Along the path of the time axis normalization function 31
Corresponding to the pitch time history of the hand sequentially on the time axis
Average from the sum of the pitch differences of the teacher's sound, and the average pitch difference
Is used as the pitch difference correction amount to correct the pitch of the accompaniment sound. concrete
First, description will be made with reference to FIG. Buffer memo at the top of Figure 4
An example of the pitch time history stored in Li 7
And the teacher sounds stored in the buffer memory 4 at the bottom.
An example of the interval time history of is shown. Obtained time base normalization
By following the path of function 31, A ₂₅₀To B_tr,
A₂₁₅To B₂₀₅・ ・ ・, A₁ To B₁₇Such as
Thus, a non-linear response in time is obtained. Pitch difference correction amount
Is the average value of the pitch difference corresponding to each time axis
It is calculated from

The above-described correction processing of the tempo difference and the pitch difference is applied to the digital accompaniment sound signal obtained from the musical sound information storage medium 5, and thereafter, the digital accompaniment sound signal corrected by a constant correction amount is converted into a D / A converter. After passing through 11, it becomes an analog signal, is mixed with the voice signal of the singer in the mixing circuit 12, and is reproduced by the speaker 13.

On the other hand, when the correction start command is not executed, the digital accompaniment sound signal supplied from the musical tone information storage medium 5 is the tempo difference correction circuit 9 and the pitch difference correction circuit 1.
0 passes through without correction, passes through the D / A converter 11, becomes an analog signal, is mixed with the voice signal of the singer in the mixing circuit 12, and is reproduced by the speaker 13. In the embodiment, the sampling rate of the D / A converter 11 is set to 4
At 4.1 kHz, the accompaniment sound signal immediately after D / A conversion is passed through a low-pass filter with a cutoff frequency of 20 kHz.

Here, even when the musical tone information storage medium 5 does not store a teacher sound signal as a model for singing (which is the second input voice), the accompaniment (which is the third input voice) By using the interval time history of the sound signal instead of the interval time history of the teacher sound signal (the first input voice)
Needless to say, the tempo difference and pitch of the singer's voice can be calculated to correct the tempo and pitch of the accompaniment sound.

[0031]

According to the sound reproducing apparatus of the present invention, the second input sound or the third input sound is matched with the tempo and pitch of the first input sound at the timing desired by the operator. Since the correction can be performed, for example, when the first input sound is applied to a karaoke device in which the correction target is an accompaniment sound as the voice of the singer, the tempo and pitch of the singer at the desired timing of the singer. Since the accompaniment sound corrected according to is reproduced, the singer can comfortably sing a song, and the entertainment of karaoke is enhanced.

Further, since the singer can instruct the karaoke apparatus to start the above-mentioned correction by operating the hand switch, very good operability can be obtained. In addition, since the correction amount of the tempo difference and the pitch difference is limited, the accompaniment sound is overcorrected, and the karaoke can be enjoyed without causing the singer to feel uncomfortable.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a sound reproducing device of the present invention.

FIG. 2 is a diagram showing a schematic processing procedure of the sound reproducing device of the present invention.

FIG. 3 is an example of a DP matching calculation result by a non-linear pattern matching circuit that constitutes the configuration of the sound reproducing device of the present invention.

FIG. 4 is a diagram showing a pitch time history of a teacher sound and a time history of a singer's pitch.

[Explanation of symbols]

 1 ... Microphone 2 ... A / D converter 3, 6 ... Pitch extraction circuit 4, 7 ... Buffer memory 8 ... Non-linear pattern matching circuit 9 ... Tempo difference correction circuit 10 ...・ Pitch difference correction circuit 11 ・ ・ ・ D / A converter 12 ・ ・ ・ Mixing circuit 13 ・ ・ ・ Speaker

Claims (5)

[Claims] 1. A first pitch extracting means for extracting a pitch of a first input voice, a second pitch extracting means for extracting a pitch of a second input voice, and a memory for storing a time history of each pitch. And a calculating means for calculating a tempo difference and a pitch difference between the first input voice and the second input voice by a non-linear pattern matching method using the time history of the pitch.
Correction means for correcting the tempo and pitch of the second input sound so as to approximate the tempo and pitch of the first input sound; and the second input sound or the second input sound corrected by the correction means,
Means for playing back the third input sound in which the calculated tempo difference and pitch difference are corrected, and a sound reproducing device. 2. The non-linear pattern matching method includes D
Using P matching, the tempo difference between the first input voice and the second input voice is calculated from the slope of the time normalization function obtained from the DP matching, and then corrected according to the tempo difference. Calculating an average pitch difference calculated from the recorded time history of the pitch of the second input voice and the time history of the pitch of the first input voice as the pitch difference between the first input voice and the second input voice. The sound reproducing device according to claim 1, wherein 3. The slope of the time normalization function obtained from the DP matching is limited, and the tempo difference of the first input voice with respect to the tempo of the second input voice is not corrected to a set value or more. The sound reproducing device according to claim 2. 4. A correction period for correcting the tempo and pitch of the second input voice to approximate the tempo and pitch of the first input voice, and after the correction period, the tempo difference and pitch. Second to keep the difference correction amount constant
2. The sound reproducing apparatus according to claim 1, wherein the input sound of the above or the third input sound is reproduced. 5. A hand switch for instructing correction of the tempo and pitch of the second input voice or the third input voice is provided, and the operator specifies the correction period by the hand switch. The sound reproducing device according to claim 4, which can be performed.