JP2612869B2 - Voice conversion method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to an audio information processing technique for processing human voice in broadcasting, movie, music, and the like. The present invention relates to a voice quality conversion method such as enhancing clarity or changing a sound to have a special effect.

[Summary of the Invention] The present invention relates to a technique for temporarily recording a human voice, changing the quality of the voice, and outputting the voice again as a voice. First, a linear prediction coefficient is calculated, a desired formant frequency and bandwidth are changed based on the linear prediction coefficient, and a linear prediction coefficient corresponding to the changed formant frequency and bandwidth is obtained. Change the envelope. Next, transform the original voice to the frequency domain by Fourier transform, change the original voice to the desired shape by the spectral envelope before and after the formant change, return it to the time domain by inverse Fourier transform, and then D / A convert it With this, the original sound has
It is a way to transform personality and improve verbal clarity.

[Prior art] Conventionally, at a broadcast site or the like, an analog filter is used to remove a specific frequency band of audio to eliminate personality, or a skilled engineer uses a graphic equalizer to reduce energy in a specific frequency band. The quality of the voice was corrected by augmentation and attenuation.

In recent years, a method using digital technology has been developed. If a voice waveform written at a sampling frequency F is read at a sampling frequency of FXR, all spectrum information including a pitch frequency becomes R times in frequency. At this time, if the waveform is thinned out or repeated using an appropriate time window and cycle, the quality of the voice can be converted while maintaining the same generation speed as the original voice. Such a device is called a “harmonizer” or the like, and has begun to be generally used as a sound effect device.

In addition, a method of controlling the quality and clarity of speech by performing filtering adapted to a speech generation model using a digital filter based on linear predictive analysis has been proposed.
61-206777.

[Problems to be Solved by the Invention] However, the method using an analog filter or a graphic equalizer basically only increases or decreases the energy of a specific frequency band of voice, and fundamentally changes the voice quality. Can not. In other words, only some of the features included in the original sound are emphasized and suppressed in terms of level, and it is not possible to decisively change individuality or improve clarity.

Further, in a device such as a "harmonizer", the whole spectrum information including the pitch moves on the frequency axis, so that the level of the voice and the personality greatly change. However, such changes are mechanical and different from the differences in the spectral information that exists between actual personal sounds. Therefore, the output sound tends to be inhuman. Also, the pitch frequency of the original voice cannot be stored.

Furthermore, since the method using a digital filter based on linear predictive analysis is adapted to a speech generation model,
Although it is possible to fundamentally control the quality and clarity of voice, there was a problem in sound quality due to factors such as filter stability.

Therefore, an object of the present invention is to solve the above-mentioned conventional problems and perform high-quality voice conversion using characteristics unique to voice by using analysis adapted to a voice generation model and fast Fourier transform together. It is an object of the present invention to provide a voice conversion method capable of performing the above.

[Means for Solving the Problems] For this purpose, in the present invention, a voiced sound section is extracted from the input voice, and in the voiced sound section, each voiced sound section is set within each short time section set by the analysis window width and the sliding period of the window width. By calculating the formant frequency and bandwidth of each of the short-term sections, and calculating the linear prediction coefficients within each short-term section, the spectrum envelope is calculated, and the voiced section is subjected to Fourier transform to be converted into the frequency domain, and the time trajectory of the formant frequency is calculated. Is calculated, a change is made to the formant frequency or bandwidth at each time point of the time trajectory, a spectrum envelope is calculated based on the changed formant frequency and bandwidth, and the spectrum envelope calculated after the formant frequency change is converted to the formant frequency. The quotient divided by the spectrum envelope calculated before the change is used as the change component, and the frequency transformed by the Fourier transform is used. After multiplying the wave number component by the change component and making a spectrum change other than the formant frequency change, the waveform is returned to the time domain by the inverse Fourier transform, and connected to the unvoiced sound section, the unvoiced section or the preceding and following voiced sound sections, and the new voice It is characterized by having a waveform.

[Operation] According to the above configuration, it is possible to change the frequency spectrum envelope using the fast Fourier transform, and to convert the voice quality of speech.

EXAMPLES Hereinafter, the present invention will be described in detail based on examples shown in the drawings.

FIG. 1 is a block diagram showing a voice conversion system according to an embodiment of the present invention. In the figure, reference numeral 2 denotes an analysis unit, 4 denotes a formant frequency control unit, and 6 denotes a spectrum control unit. Each unit is configured in a computer and performs voice quality conversion processing while using memories such as a ROM, a RAM, and a memory disk. Is executed.

The A / D-converted and sampled speech waveform is input to the analysis unit 2 to determine voiced and unvoiced and voiced and unvoiced sounds, and for voiced sounds, a resonance frequency is obtained.

Next, in the formant frequency control unit 4, the analysis unit 2
A formant frequency is obtained based on the resonance frequency obtained in step (1), and a desired change is made.

The spectrum controller 6 changes the spectrum envelope according to the formant frequency changed by the formant frequency controller 4.

When the voice quality conversion processing for a series of voiced sounds is completed, the unvoiced sound section and the unvoiced sound section are connected, and the process proceeds to the next voiced sound section. D /
A converted to output audio.

The details of the processing in each section will be described with reference to the flowchart shown in FIG.

First, in the analysis unit 2, a speech section and a non-speech section are determined in the analysis unit 2 based on the presence or absence of the speech power in the analysis unit 2 at a conversion bit number of 12 bits and a sampling frequency of 15 kHz. Next, in step S2, the sample value of the sound interval is
PARCOR analysis and zero-crossing analysis are performed to discriminate between unvoiced consonant sections and voiced sound sections. This is performed by checking the ratio of the high frequency component of the input frequency with reference to the first-order PARCOR coefficient or checking the number of zero crossings. That is, the energy of unvoiced consonants is distributed up to the high frequency region, and unvoiced consonants and voiced sounds are determined by examining the proportion of high frequency components and the number of zero crossings that increase at higher frequencies. In addition, PA
The reason for performing the discrimination using both the RCOR analysis and the zero-crossing analysis is to ensure the discrimination.

The time of the silent section and the waveform of the unvoiced consonant section determined in steps S1 and S2 are stored as they are in the RAM or the memory disk in steps S14 and S15, respectively.

Next, in step S3, a linear prediction analysis is performed by passing a sample value of the voice waveform in the voiced sound section through a so-called vocal tract inverse filter based on a voice generation model. That is, first, windowing is performed with a window width of about 20 msec, and a correlation function is obtained based on data obtained by windowing the sample values, thereby calculating linear prediction coefficients α _{1 to} α _p in step S4.
Here, p is an order of linear prediction, and p = 14 is used for a male voice and p = about 10 is used for a female voice. Further, in step S5, α _{1 to} α _p obtained above are used as coefficients, and roots z _{1 to} z _p of the complex number z satisfying the following equation (1).
Ask for.

1 + α ₁ z ^-1 + α ₂ z ^-2 +... + Α _p z ^-p = 0 (1) z _{1 to} z _p include conjugate complex roots, and one resonance point is represented by a pair of conjugate complex roots. Therefore, only for z _i whose imaginary part is positive, the resonance frequency F _i and its bandwidth B _i are obtained by the following equations (2) and (3), and the RAM and the linear prediction coefficient are obtained.
Alternatively, it is recorded on a memory disk or the like.

F _i = Fs / (2π) · arg (z _i ) [Hz] (2) B _i = Fs / π · log (| z _i |) | [Hz] (3) where Fs is the sampling frequency of the voice It is.

These series of operations are repeated until the voice section ends, while sliding the analysis start position backward by about 10 msec.

The formant frequency control unit 4, in step S6, a series of resonant frequency F _i obtained in the analysis unit 2 calculates the time trajectory of the formant frequency in consideration of the continuity and its bandwidth. In general, the roots that provide non-formant frequencies have wider bandwidths than those that provide formants. The formant frequencies are the first formant, the second
Called formants,… the phonology of vowels and voiced consonants
The first to third formants are important, and their trajectories are determined particularly accurately.

Next, in step S7, a desired change is made to the locus of the formant frequency obtained in step S6, and a new formant frequency and bandwidth are determined.

For example, as shown in FIG.
It is effective to enhance the movement of the formant frequency over the time axis of the third to third formants.

In order to change the personality, it is effective to shift all the formant frequencies uniformly as shown in FIG. As for the bandwidth, narrowing it gives a clear voice, and widening it gives a smooth voice.

When the new formant frequency trajectory and bandwidth are determined, in step S8, a new linear prediction coefficient at each time point is calculated as follows.

The new resonance frequency including the changed formant, the unchanged formant, and the resonance frequency not recognized as a formant is denoted by F _i ′, and its bandwidth is denoted by B _i ′. For each pair of F _i ′ and B _i ′, a new root z _i ′ of z is obtained by using the following equation (4) and generally considering that | Z _i ′ | <1.

z _i ′ = exp (−πB _i ′ / Fs + j2πF _i ′ / Fs) (4) In addition to these Z _i ′, each conjugate complex root and (1)
If there is a root of z obtained by solving the equation and the imaginary part is zero, a new polynomial is expressed as shown in the following equation (5) by using a total of p z _i ′ s including the imaginary part. create.

^{_{(1-z 1 'z -1}} ) (1-z 2' z -1) ...... ... (1-z p 'z -1) = 1 + α 1' z -1 + α 2 'z -2 + ... + α p 'Z- ^p (5) α ₁ ' to α _p 'satisfying the right side of the equation (5) give a new linear prediction coefficient.

The spectrum control unit 6 obtains the result obtained by the formant control unit 4, and changes the spectrum envelope of the original voice at each time point to a desired spectrum envelope.

Here, the number of samples corresponding to the window width of the linear prediction analysis in the analysis unit 2 is M, and the number of samples corresponding to the period of the slide of the analysis window is L. In this example, M = 300 and L = 150.

First, as shown in FIG. 5, q + 2L−
Obtain the square sum P _S for the data of 2L sets of samples up to a point. Further, in steps S16 and S17, the linear prediction coefficients α ₁ to α ₁ obtained from the analysis window shown above in the analysis unit 2 in the analysis unit 2
α _p and α ₁ ′ to α ₁ ′
Using α _p ′, the spectrum envelope H (k) of the original speech and the spectrum envelope (k) of which the formant is changed are obtained by the equations (6) and (7).

Here, N is a power of 2 larger than M and is set to 512.
H (k) is a physical quantity including a lot of phonological and personality of the original voice, while (k) emphasizes and suppresses phonological and personality,
Or it has been changed.

Note that (k) can be directly calculated from p z _i 'obtained by the formant frequency control unit using the following equation (8).

However, since the FFT (fast Fourier transform) algorithm can be applied to the forms of equations (6) and (7), the use of equation (8) is disadvantageous in time.

Next, in steps S9 and S10, q + L−N / 2
N samples up to + L + N / 2−1 points are newly added to x
(1) to x (N), and multiplied by a time window coefficient as shown in the following equation (9) to obtain y (1) to y (N).

y (m) = w (m) · x (m) m = 1 to N (9) where T = N / 2−L + 1 and T ′ = N / 2 + L, w (m) = 0.5 · ｛1−cos (Πm / T)｝ 1 ≦ m ≦ Tw (m) = 1 T <m <T'w (m) = 0.5 · [1-cos {π (m-T ') / T}]
T ′ ≦ m ≦ N These y (m) are subjected to a fast Fourier transform at N points and converted into a frequency domain to be Y (k). In step S11, the absolute value of this Y (k) is shown below. Change using equation (10). The phase component is left as it is.

(K) = A (k) · (k) / H (k) · Y (k) k = 1 to N (10) where A (k) is a factor other than the change of the formant based on the linear prediction coefficient. Which gives a change to the spectrum envelope, and is a real array between 0 and 1, and A (k) = A
(N−k + 2). For example, the following (1
According to the expression (1), the characteristic emphasizes a high frequency.

A (k) = 1.4 · (k−1) /N+0.3 k = 1 to N / 2 + 1
(11) In step S12, waveforms (1) to (N) of time conversion are performed by inverse fast Fourier transform on (k) in equation (10).
And N / 2−L + 1 to N / 2 + of the obtained N points of data
To L of 2L samples of the data, we obtain the square sum P _S ', which is to be equal to P _S previously obtained, i.e., the adjustment of the gain as shown as (12) in which the magnitude of voice equal And (N / 2−L + 1) − (N / 2 + L). That is, (m) = (P _S / P ′ _S ) ^1/2 · (m) m = N / 2−L + 1 to N / 2 + L (12) On the other hand, a Hanning window or a triangular window is set so that 0 is at both ends and 1 is at the center, and this waveform is temporarily stored in a RAM or a memory disk. This windowing can reduce the end effect at the time of waveform connection.

Next, the q point is shifted to the point after the L point, and after performing the same series of processing, the L point in the first half of the data of the 2L sample and the L point in the second half of the immediately preceding processing frame are determined as shown in FIG. Overlap and add sequentially.

Thereafter, if the same operation is repeated until the end of the sexual sound section, a continuous sound waveform with a changed spectral envelope can be obtained.

When the processing for one voiced sound section is completed, in step S13, the process is connected to the preceding or next unvoiced sound section or the silent section, and the process proceeds to the next voiced section. Finally, the synthesized voice is D / A converted to output voice.

In this embodiment, linear prediction analysis is used as a method for detecting voiced sound sections and extracting formants, but the effect of the present invention does not change even if it is obtained by other methods.

[Effects of the Invention] As described above, according to the present invention, the quality of voice can be converted by changing the frequency spectrum envelope at each time point based on the speech generation model.

Fast Fourier transform is used to change the spectral coverage, and the sound quality is better than that of a conventional method using a digital filter. Also, since the spectrum can be freely controlled in the frequency domain, spectrum control other than the concept of formants is also possible, and while maintaining the pitch frequency of the original sound, not only control of personality and improvement of clarity, but also various voices Can be controlled.

[Brief description of the drawings]

FIG. 1 is a block diagram of a system according to one embodiment of the present invention, FIG. 2 is a flowchart showing one embodiment of the present invention, and FIG. 3 is a diagram for explaining a change over time axis of a formant frequency in the embodiment. FIG. 4 is a diagram for explaining a uniform change of the formant frequency over the time axis in the embodiment. FIG. 5 is a waveform diagram for explaining a processing section in the embodiment. FIG. FIG. 7 is a waveform chart for explaining superposition of waveforms in an example. 2 ... analyzing unit 4 ... formant frequency control unit 6 ... spectrum control unit

 ──────────────────────────────────────────────────続 き Continuation of the front page (56) References Kuwahara and Tsuki, "Voice quality conversion by analysis and synthesis and application to hoarseness improvement," IEICE Technical Report SP86-57, PP. 45− 52

Claims (1)

(57) [Claims] 1. A voiced sound section is extracted from an input voice, and a formant frequency and a bandwidth in each short time section set by an analysis window width and a sliding period of the window width are calculated in the voiced sound section. In addition, by calculating a linear prediction coefficient in each of the short time intervals, a spectrum envelope is calculated, a Fourier transform is performed on the voiced sound interval to convert it into a frequency domain, and a time trajectory of the formant frequency is obtained. Change the formant frequency or the bandwidth at each time point, calculate a spectrum envelope based on the changed formant frequency and bandwidth, and change the spectrum envelope calculated after the formant frequency change to the formant frequency change The quotient divided by the spectrum envelope calculated previously is used as a change component, and transformed by the Fourier transform. After multiplying the changed frequency component by the changed component and adding a spectrum change other than the formant frequency change, the waveform is returned to the time domain by the inverse Fourier transform, and connected to the unvoiced sound section, the unvoiced section or the preceding and following voiced sound sections. And a new voice waveform.