JP2612867B2 - Voice pitch conversion method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to audio processing in broadcast, movie, music, etc., such as changing the pitch, accent, intonation, etc. of sound, adding vibrato, etc. The present invention relates to a voice pitch conversion method for controlling a pitch frequency of voice.

[Summary of the Invention] The present invention relates to a technology for temporarily recording a human voice, changing the pitch cycle thereof, and outputting the voice again as a voice. Extract the pitch frequency, divide the waveform at each pitch interval,
Expanding and contracting the cycle of each pitch, connecting these so that the speech time length does not change, and further performing a Fourier transform on the waveform,
After reducing the distortion component caused by the pitch change in the frequency domain, returning it to the time domain by inverse Fourier transform, and then performing D / A conversion, the phonology and naturalness of the original voice are maintained well This is a method that allows the voice pitch and intonation to be freely converted.

[Prior Art] As this kind of technique, there is a classic example of a method of recording sound on an analog tape recorder and changing the reproduction speed. In this method, not only the pitch frequency but also the entire frequency band including the formant frequency changes uniformly, and the speech time length also changes simultaneously.

That is, if the reproduction speed is R times that at the time of recording, the pitch and the formant frequencies are all R times, and the speech time length is 1 / R times.

Here, the pitch gives the level of the voice, and characterizes the accent and intonation by its temporal change, and the formant characterizes the phonology of the voice, and has a great individual difference.

In contrast to the above-mentioned conventional example, a method has been developed in which the speech time length is not changed using digital technology.

That is, if the voice waveform written at the sampling frequency F is read at a sampling frequency of F × R, the pitch and the formant frequency become R times. At this time, if the waveform is thinned or repeated using an appropriate time window and cycle, the utterance time length can be kept the same as the original voice. Such a device is called a "harmonizer" or the like, and is generally used as a sound effect device.

[Problems to be Solved by the Invention] However, in any of the conventional examples described above,
When the pitch frequency is changed, it is inevitable that the formant frequency also changes at the same time.

When the formant frequency changes, the individual difference in the voice becomes unclear, and when the change amount is large, the phonological properties deteriorate, resulting in a non-human voice. Therefore, there is a problem in that a change in the formant frequency accompanying a change in the pitch frequency is harmful unless the above-described effects are actively used.

Further, in the conventional voice processing device, since the pitch of the voice is mainly controlled, it is easy to control the average change of the pitch frequency over a long period of time. There was a problem that the change in frequency could not be controlled.

Accordingly, an object of the present invention is to solve the above-mentioned conventional problems and to make the formant frequency invariable even when the pitch frequency of the original voice is largely changed, and to reduce the frequency distortion due to the change, thereby improving personality and phonology. It is an object of the present invention to provide a voice pitch conversion method capable of controlling the pitch, accent, and the like of a voice without deteriorating the naturalness of human voice.

Another object of the present invention is to provide a voice pitch conversion method capable of freely controlling the pitch frequency even within a short period of time so that the intonation and vibrato can be freely emphasized and replaced.

[Means for Solving the Problems] For this purpose, in the present invention, a voiced sound section is extracted from an input voice, a pitch cycle is extracted from the voiced sound section, and a linear prediction coefficient is calculated in each pitch section corresponding to the extracted pitch cycle. , Calculate the spectral envelope using the linear prediction coefficient, expand and contract the waveform of each pitch section by using the linear prediction coefficient, and make the expanded and contracted waveform equal to the utterance time length of the input voice. Connect by thinning out or repeating the waveforms in the pitch section, find the linear prediction coefficient in the connected waveform, calculate the spectral envelope using the linear prediction coefficient, and calculate the spectral envelope calculated before and after the waveform expansion and contraction The difference from the obtained spectral envelope is used as a distortion component, and the connected waveform is transformed into the frequency domain by Fourier transform. After correcting the distortion component from the minute, the waveform is returned to the time domain by the inverse Fourier transform, and a comb-shaped filter corresponding to the average pitch period of the returned waveform is applied to the waveform. , A new voice waveform.

[Operation] According to the above configuration, it is possible to change the pitch frequency without changing the formant frequency of the original voice while keeping the frequency spectrum envelope of that of the original voice.

Further, the pitch period can be changed for each pitch section.

EXAMPLES Hereinafter, the present invention will be described in detail based on examples shown in the drawings.

FIG. 1 shows a block diagram of a pitch frequency conversion system according to one embodiment of the present invention. In the figure, reference numeral 2 denotes an analysis unit, 4 denotes a pitch frequency control unit, 6 denotes a waveform connection unit, and 8 denotes a distortion correction unit. Each unit is configured in an electronic computer and uses a memory such as a ROM, a RAM, or a disk memory. The pitch frequency conversion process is executed while performing the process. The A / D converted and sampled speech waveform is input to the analysis unit 2 to determine whether the speech is voiced or unvoiced or voiced and unvoiced, and a pitch interval is determined for voiced speech.

Next, the pitch frequency control unit 4 makes a desired change for each pitch section obtained by the analysis unit 2, calculates a new pitch cycle sequence, and expands and contracts the waveform for each pitch according to the new pitch cycle. I do. Thereby, the level of the sound, intonation, and the like are controlled.

The waveform connection unit 6 connects the waveforms of the respective pitches changed by the pitch frequency control unit 4 by appropriately thinning out or repeating them so that the speech time length does not change.

The distortion correcting unit 8 sequentially obtains the short-time spectral envelope of the synthesized waveform in the voiced sound section obtained by the waveform connecting unit 6 and corrects the short-time spectral envelope so that it becomes the same as the spectral envelope of the original voice.

When the above-described pitch frequency conversion processing for a series of voiced sounds is completed, the unvoiced sound section and the silent section are connected,
The process proceeds to the next voiced sound section. The finally synthesized voice waveform is D / A converted to output voice.

The details of the processing in each section will be described with reference to the flowchart shown in FIG.

First, at step S1 in the analyzing unit 2, a voiced section and a silent section are determined based on the presence or absence of voice power in the voice A / D-converted at a conversion bit number of 12 bits and a sampling frequency of 15 kHz. Next, in step S2, PA
An RCOR analysis and a zero-crossing analysis are performed to determine an unvoiced consonant section and a voiced sound section. This is performed by checking the ratio of the high frequency component of the input frequency with reference to the first-order PARCOR coefficient or checking the number of zero crossings. That is, the energy of unvoiced consonants is distributed up to the high-frequency region, and unvoiced consonants and voiced sounds are determined by examining the proportion of high-frequency components and the number of zero crossings that increase at higher frequencies. In addition, PARCOR
The reason why the discrimination is performed using both the analysis and the zero-crossing analysis is to ensure the discrimination.

The time of the silent section and the waveform of the unvoiced consonant section determined in steps S1 and S2 are stored in RAM or a memory disk as they are in steps S21 and S22, respectively.

Next, in step S3, a linear predictive analysis is performed by passing a sample value of the voice waveform in the voiced sound section through a so-called vocal tract inverse filter based on a voice generation model. By this linear prediction analysis, a linear prediction coefficient and a residual waveform are obtained. The obtained residual waveform is stored in a RAM or a disk memory or the like in step S23.

In step S4, a temporary pitch period is determined from the period between the phases of the residual waveform obtained in step S3 and the interval between the peaks of the original audio waveform.

Next, in step S5, as shown in FIG. 3, the point immediately before the point where the waveform level suddenly increases is set as the pitch start point, and one sample of the start point of the next pitch is obtained based on the pitch period obtained above. One pitch section is determined with the near side as the end point.

In step S6, windowing is performed for about 20 msec with the midpoint of the one pitch section determined above as the center of the analysis window. This windowing makes it possible to perform a short-time spectrum analysis using a finite number of sample values, and performs linear prediction analysis again based on the windowing data. That is, the linear prediction coefficients α _{1 to} α _p are calculated by obtaining a correlation function based on the data obtained by windowing the sample values. Here, p is the order of the linear prediction analysis, and generally, p = 14 is used for a male voice and p = about 10 is used for a female voice.

In steps S7 and S8, a value obtained by dividing the sum of the squares of the sample values in one pitch section by the pitch section length is defined as a normalized power,
It is stored in a RAM or a memory disk together with the length of the pitch section and the linear prediction coefficient.

When the processing for one pitch section in steps S6 to S8 is completed, the processing section is shifted backward by one pitch, the processing for the next pitch section is performed, and these operations are repeated until the voiced section ends.

First, in step S9, the pitch frequency control unit 4 adds a desired change to each of the series of pitch periods obtained by the analysis unit 2, and newly calculates a pitch period sequence. Namely, within certain voiced section, the period P _n of the n-th pitch from the beginning of the pitch, the pitch frequency is F _n = 1 / P _n, also the total number of pitches and L. Furthermore, the average pitch frequency F
_AVE is defined as the geometric mean of all pitch frequencies taking into account the perceptual mechanism of height in human speech.

That is, F _AVE = (F ₁ × F ₂ ×... F _L ) ^{1 / L} = (P ₁ × P ₂ ×... P _L ) ^{−1 / L} (1) At this time, for example, the level of voice is controlled. If it is desired to make the average pitch frequency R times for this, all pitch periods should be made 1 / R times from the equation (1). When the intonation is changed like an accent, the pitch must be expanded or contracted at a different rate for each pitch cycle. Therefore, as shown in FIG. 4, the n-th pitch frequency F _n for multiplying R _n for each pitch cycle.

In the case of emphasizing or suppressing intonation around the average pitch frequency of the original speech as shown in Fig. 5, it may be used as shown in the R _n (2) expression. In other words, R _n = (F _n / F _AVE ) ^C−1 At this time, if C> 1, the intonation is emphasized, and if 0 ≦ c <1, the intonation is suppressed.

Next, in step S10, the waveform for each pitch is expanded or contracted in accordance with the new pitch cycle obtained in step S9. That is, assuming that the number of samples in a certain pitch section in the original voice is k and the number of samples corresponding to the changed pitch section length is k ′, when the pitch period is shortened, the waveform is changed from the start point of the pitch section by k ′. When the sampling period is censored up to the 'th sample value and the pitch period is extended, the linear prediction coefficients α _{1 to} α _p obtained by the analysis unit 2 are used to obtain m = k + 1 to m = The sample values up to the k'th are obtained to obtain the subsequent waveform.

x (m) = α ₁ x (m−1) α ₂ x (m−2) +... + α _px (m−p) (3) However, considering the characteristics of human voice, the following part is an exponent Multiplied by a window coefficient that attenuates in time.

In step S11, the normalized power obtained in step S7 is adjusted. That is, when the pitch period is changed, the above-described normalized power generally also changes. Therefore, each sample value is multiplied by a constant so as to be the same as the value obtained in step S7.

The waveform connection unit 6 first compares the utterance time length in step S12. That is, the speech time length of the original voice is T, n
Th the pitch period of the pitch interval between P _n, those after the change pitch frequency respectively T ', P _{n' When, T = P 1 + P 2} + ...... + P L (4) T '= P 1' + P ₂ ′ +... + P _L ′ (5) Generally, the utterance time length changes by changing the pitch frequency, so that T ≠ T ′.

Therefore, γ is set to T ′ / T, and pitch sections are thinned out or repeated in step S13 according to the value of γ. That is, if γ> 1, thinning is performed at a pitch of 1 pitch per γ / (γ−1) pitch, and if γ <1, γ / (1
-Γ) The same waveform is repeated at a rate of one pitch per pitch.

FIGS. 6 (A) and 6 (B) show the states of processing when γ = 1.5 and γ = 0.667, respectively. As shown in the figure, in the case of γ = 1.5, the voice pitch sections 3 and 6 after the pitch change once in 3 pitches are thinned out, and in the case of γ = 0.667,
The waveforms of the pitch sections 2, 4 and 6 of the voice after the pitch change once for two pitches are repeated.

As a result, the speech time length of the original sound can be generally maintained, and there is no sense of incongruity in the sense of hearing. In general, in a waveform in which the pitch cycle is changed, the final sampling point of the pitch section of the waveform and the next sample point Since there is a large discontinuity in the sample value between the start sample point of the pitch section and the sample point, a step S14 uses the least squares method by using the data of the connection point, ie, the last sample point and several samples before and after the start sample point. An approximation using a quadratic curve is performed to connect continuously.

First, in step S15, the distortion correcting unit 8 sets the seventh
As shown in the figure, the sum of squares P is obtained for data of M samples from point q to q + M−1 of the waveform whose pitch period has been changed, and linear prediction analysis is performed on the M sample values. To obtain the linear prediction coefficients α ₁ ′ to α _p ′.

In steps S24 and S25, the linear prediction coefficient α ₁ ′
To? _{P 'and} using the linear prediction coefficients alpha ₁ to? _P obtained in the analysis unit 2 was described above from the portion corresponding to the time the same section of the original speech, it it shown below (6) Then, the spectral envelopes (K) and H (K) are obtained by the equations (7).

Here, M is the number of samples in a time length of 20 to 30 msec,
Since the sampling frequency is 15 kHz, the value is about 300 to 450, and N is 512 which is a power of 2 larger than M.

The spectral envelope H (K) contains a lot of phonological and individual characteristics of the original speech, that is, a physical quantity characterizing the formant frequency. However, (K) is not necessarily H (K) due to distortion caused by a change in pitch period. Does not match. The following processing is performed to correct this distortion.

First, in step S16, q- (N
−M) / 2 points to q + (N + M) / 2−1 are newly set as x (1) to x (N), and a time window coefficient w ( m) and y (1) -y
(N). That is, y (m) = w (m) · x (m) m = 1 to N (8) where L = (N−M) / 2 + 1, L ′ = (N + M) / 2, and w (m) = 0.5 · {1-cos (πm / L)} 1 ≦ m ≦ Lw (m) = 1 L ≦ m ≦ L ′ w (m) = 0.5 · [1 + cos {π (m−L ′) / L} L ′ ≦ m ≦ N The obtained y (m) is subjected to N-point Fast Fourier Transform, converted to the frequency domain, and set as Y (K). Next, in step S17, as shown by the following equation (9), Y
The absolute value of (K) is changed using the ratio of the spectral envelopes H (K) and (K). That is, (K) = H (K) / (K) .Y (K) K = 1 to N
(9) In step S18, the obtained (K) is converted into time-domain waveforms (1) to (N) by inverse fast Fourier transform. (1) to (N). This attenuates distortion components generated at a frequency that is a non-integer multiple of the pitch frequency.

(M) = 0.25 ｛(1-a) (m−kp) +2 (1+
a) (m) + (1−a) (m + Kp)｝ (m) = (1) m ≦ 0, (m) = (N) m>
N (10) where Kp is the number of samples in the pitch section corresponding to the average pitch period in the processing section after the pitch frequency is changed,
A is a constant between 0 and 1, and about 0.01 is used.

For the data of the central M samples among the data at the N points obtained from equation (10), the sum of squares Ps' is
The gain is adjusted by multiplying each sample value by a constant so that S15 becomes equal to the obtained Ps. This keeps the volume of the voice equal. Furthermore, when connecting the waveforms, a Hanning window or a triangular window is set to 0 at both ends and 1 at the center in order to reduce the effect of the end, and this waveform is stored in a RAM or a memory disk.

Next, the point g shown in FIG. 7 is shifted backward by M / 2 points to shift the processing section, and after performing a series of processes from step S16, the first half of the M sample values as shown in FIG. And the M / 2 point of the latter half of the immediately preceding processing frame are superimposed and sequentially added.

Thereafter, if the same operation is repeated until the end of the voiced sound section, a voice waveform having the same spectral envelope as the original voice can be obtained. As a result, the formant frequency becomes invariable, and the phonological and individual characteristics of the original voice can be maintained.

When Kp is larger than a certain value, M is set to 450 to 6
00, and at the same time, if N is also increased to 1024, better sound quality can be obtained.

When the processing of one voiced sound section is completed, in step S20, the preceding and following unvoiced sound sections or silence sections are connected,
After step S2, the process moves to the next voiced section. Finally, the synthesized voice is D / A converted to output voice.

[Effects of the Invention] As described above, according to the present invention, it is possible to change the pitch frequency without changing the formant frequency of the original voice while keeping the frequency spectrum envelope of the original voice. it can. Therefore, it is possible to change the pitch frequency in a state of higher naturalness than the conventional technique without affecting the phonological and personality depending on the formant structure.

Also, in the conventional apparatus, the change amount of the pitch frequency is constant for a long time, but in the present invention, the intonation is changed by changing the change amount for each pitch, and the intonation of conversation and the vibrato of singing voice are changed. Can be controlled.

[Brief description of the drawings]

FIG. 1 is a block diagram of a system according to one embodiment of the present invention, FIG. 2 is a flowchart showing one embodiment of the present invention, and FIG. 3 is a waveform diagram for explaining how to determine a pitch section in the embodiment. 4 and 5 are diagrams showing a pitch cycle sequence for explaining a change in pitch frequency in the embodiment. FIG. 6 is a waveform diagram for explaining thinning or repetition of a pitch section in the embodiment. FIG. 7 is a waveform chart for explaining distortion correction in the embodiment, and FIG. 8 is a waveform chart for explaining waveform superposition in the embodiment. 2 ... analysis part, 4 ... pitch frequency control part, 6 ... waveform connection part, 8 ... distortion correction part.

Continuation of the front page (56) References JP-A-59-82608 (JP, A) JP-A-1-93795 (JP, A) JP-A-1-93796 (JP, A) Kuwahara, Tsuki, "Analytical Synthesis Conversion and its application to hoarseness improvement, IEICE Technical Report SP86-57, pp. 45-52, Tosuki and Umeda, "Voice Quality Conversion to Correct Distortion at Pitch Change in the Spectral Domain," EA87-82, PP.49-56

Claims (1)

(57) [Claims] 1. A voiced sound section is extracted from an input voice, a pitch period is extracted from the voiced sound section, a linear prediction coefficient is obtained in each pitch section corresponding to the extracted pitch period, and the linear prediction coefficient is calculated. Calculating the spectral envelope by using the linear prediction coefficient to expand and contract the waveform of each of the pitch sections, and to make the expanded and contracted waveform equal to the utterance time length of the input voice. The waveform is thinned or repeated to obtain a connection, a linear prediction coefficient is obtained in the connected waveform, a spectrum envelope is calculated using the linear prediction coefficient, and the spectrum envelope calculated before the expansion and contraction of the waveform and the A difference from the spectrum envelope calculated after expansion and contraction is defined as a distortion component, and the connection waveform is transformed into a frequency domain by Fourier transform. After correcting the distortion component from each frequency component in several domains, the waveform is returned to the time domain by inverse Fourier transform, and after applying a comb filter corresponding to the average pitch period of the returned waveform to the waveform, A voice pitch conversion method characterized in that the voice pitch is connected to a preceding or following unvoiced section or a non-voice section to form a new voice waveform.