JP2904279B2 - Voice synthesis method and apparatus - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice synthesizing method and apparatus for synthesizing a pre-stored voice uttered by a human based on output voice information and outputting the synthesized voice.

[Summary of the Invention] The present invention connects a pre-stored voice uttered by a human and outputs a meaningful voice, and affects the synthesized voice due to the connection and the addition of the prosodic information. And a voice synthesizing method and apparatus in which the clarity and naturalness of synthesized voice are not impaired.

[Prior Art] Conventionally, in this kind of technology, a recording and editing method for connecting and reproducing voices of words and phrases recorded in advance, and parameters obtained by analyzing a waveform once are recorded. A parameter editing method for controlling the synthesizer by using these parameters at the time of reproduction, and synthesis from parameters stored as unit sounds such as phonemes and syllables and prosody information such as accents and intonations generated by predetermined rules during synthesis. Speech synthesis methods such as a rule synthesis method have been known.

[Problems to be Solved by the Invention] However, in the method described above, since the output voice is selected and output from those registered in advance by a person, the sound quality can be maintained in the registered voice. ,
Since the voices are connected and output as they are, the vocal cord vibration (pitch) frequency and the formant discontinuity occur, resulting in unnatural voices.

In addition, the method of the above is a method of outputting an impulse or white noise corresponding to a pitch frequency or an estimated vocal cord waveform as a sound source through a vocal tract characteristic filter at the time of synthesis, and also changes the accent and intonation in these cases. That is, when the pitch cycle of the voice is changed, the formant also changes at the same time, resulting in a phonologically unclear voice, and furthermore, unnaturalness occurs at the connection point between the unit voices to be synthesized and the unit voice. .

Therefore, an object of the present invention is to solve the above-mentioned conventional problems and to control pitch and formant in order to improve the sound quality of synthesized speech, for example, by reducing the synthesis unit of speech and adding prosody information such as intonation. It is an object of the present invention to provide a speech synthesis method that retains the phonological properties of synthesized speech even when it is finely divided and has naturalness as human speech.

It is another object of the present invention to provide a method for measuring the perceptual characteristics of human linguistic speech by enabling control of prosody such as intonation and phonology based on formant frequency.

[Means for Solving the Problems] For this purpose, in the present invention, unit voice information and prosody information are determined based on output voice information, and the predetermined unit voice is selected from pre-stored unit voice data uttered by a human. Based on the information, unit voice data corresponding to the unit voice information is selected, and a pitch cycle, a spectral envelope, a formant locus, and a unit voice waveform are calculated or extracted from the selected unit voice data, and the calculation or extraction is performed. The calculated or extracted pitch period is changed so as to smoothly connect the unit sound waveforms thus obtained and to add the prosody information, and the spectrum envelope by the pitch change is calculated in the changed pitch period. ,
A first spectrum change is calculated based on the spectrum envelope due to the pitch change and the calculated or extracted spectrum envelope, and the calculated or extracted unit voice waveform is calculated or extracted so as to smoothly connect the calculated or extracted unit sound waveform. A formant trajectory is changed, a spectrum envelope by a formant change is calculated based on the changed formant trajectory, and a second spectrum change is calculated based on the spectrum envelope by the formant change and the calculated or extracted spectrum envelope. Then, based on the first and second spectrum changes, the spectrum envelope of the unit speech waveform related to the change of the pitch period is changed, and after connecting the unit speech waveform having the changed spectrum envelope, the connected speech is connected. Is output.

A unit for determining unit voice information and prosody information based on the output voice information; and a unit corresponding to the unit voice information based on the determined unit voice information from unit voice data uttered by a human stored in advance. Means for selecting unit voice data; means for calculating or extracting each of a pitch period, a spectral envelope, a formant trajectory and a unit voice waveform from the selected unit voice data; and smoothing the calculated or extracted unit voice waveform. Means for changing the calculated or extracted pitch cycle so as to be connected to and adding the prosody information; means for calculating a spectrum envelope by pitch change in the changed pitch cycle; A first spectrum based on the changed spectrum envelope and the calculated or extracted spectrum envelope; Means for calculating a change in the formant trajectory; means for changing the calculated or extracted formant trajectory so as to smoothly connect the calculated or extracted unit sound waveform; and a formant change based on the changed formant trajectory. Means for calculating a spectrum envelope by the formant change, means for calculating a second spectrum change based on the spectrum envelope by the formant change and the calculated or extracted spectrum envelope, and means for calculating the first and second spectrum changes. Means for changing the spectrum envelope of the unit sound waveform according to the change of the pitch period, and means for connecting the unit sound waveform whose spectrum envelope has been changed and outputting the connected sound. And

[Operation] According to the above configuration, it is possible to perform high-quality speech synthesis in which necessary intonation is added to meaningful speech while maintaining the phonological and natural characteristics of each unit speech.

[Example] Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of a speech synthesis system showing one embodiment of the present invention. In the figure, 2 is a unit sound selection unit,
Reference numeral 4 denotes a vocal cord frequency correction unit, 6 denotes a formant correction unit, and 8 denotes a frequency characteristic change unit. Each unit is configured in an electronic computer. With this configuration, voice synthesis is performed while using a memory such as a ROM, a RAM, or a disk memory. The processing is executed.

When output speech information such as text is input to the unit speech selection unit 2, the output speech information is first converted into unit speech information, which is a phoneme symbol string as discrete linguistic information, and continuation of speech is performed. Prosodic information such as time length, accent, intonation, and pause is determined. Further, a sequence of unit speeches constituting a phoneme symbol sequence determined from a set of unit speeches stored in advance is selected.

Instead of determining the phonetic symbol sequence based on the output voice information, a word or character may be determined as the unit voice information, and the unit voice sequence may be selected based on this. Further, the unit voice to be stored is only the waveform in this example. However, the linear prediction coefficient or the pitch or formant trajectory of the unit voice is also stored as the unit voice data, so that the pitch shown below is obtained. The calculation of the period, the spectral envelope, and the formant trajectory may not be performed each time.

Next, the tone frequency correction unit 4 determines a voiced sound section from the unit sound obtained by the unit sound selection unit 2, and obtains a pitch frequency of the voiced sound section and its time locus. Further, the pitch frequency trajectory of the preceding and following unit voices is referred to, the pitch frequency is converted so that the trajectory is smoothly connected, and components such as accent and intonation are further added as changing components, and each component is changed according to the new pitch frequency. The duration of the waveform is expanded or contracted for each pitch cycle. Thereby, the pitch frequency is smoothly connected, and the accent, intonation, and the like as the prosody information are controlled. In addition, a change in the spectrum envelope changed by this processing is obtained.

The formant correction unit 6 determines a formant from a spectrum envelope form (resonance frequency) for a voiced sound section and obtains a locus thereof. Next, the formant frequency is changed with reference to the formant frequency trajectories of the unit voices before and after so as to connect smoothly. Further, a change caused by the formant change from the spectrum envelope of the original voice is obtained.

The frequency characteristic changing unit 8 changes the spectral envelope of the waveform expanded and contracted in accordance with the change of the pitch frequency according to the spectrum change obtained by the formant correction unit and the spectrum change obtained by the vocal cord frequency correction unit. .

Details of the processing in each of the above units will be described with reference to block diagrams and flowcharts shown in FIGS. 2 (A) and 2 (B). FIG. 2A shows the details of the configuration shown in FIG. 1. The unit voice selection unit 2 is composed of the unit voice storage unit 22 and the voice selection unit 24, the vocal cord frequency correction unit 4 is composed of the pitch extraction control unit 42, Expansion / contraction section 44, spectrum envelope extraction section 46
And the correction component extraction unit 48, the formant correction unit 6 is further controlled by the spectrum envelope extraction unit 62, the spectrum envelope control unit 64, and the correction component extraction unit 66.
Is composed of an FFT unit 82, a spectrum envelope changing unit 84 and an IFFT unit 86, respectively. The numbers attached to the left side of each step in the flowchart of FIG. 2 (B) indicate the numbers of the respective parts in FIG. 2 (A). Indicates that it will be executed.

In the above configuration, the vowel-
A set of phonemes such as consonants and vowels.
D-converted data is stored as unit sound. Here, the speech selection unit 24 determines a phoneme symbol string according to a predetermined phoneme rule based on output speech information such as an input sentence and the like, and further according to a prosody rule, a duration time, an accent,
Intonation, pose, etc. are determined. Further, a corresponding unit voice is selected from the unit voice storage unit 22 based on the determined phoneme symbol string.

Each of the extracted unit sounds is subjected to discrimination between a sound section and a silent section in the pitch extraction control section 42 based on the presence or absence of the sound power. The number is obtained, and the unvoiced consonant section and the voiced sound section are discriminated. This is because a reliable discrimination is performed by examining both the high-frequency component in the voice, the first-order correlation coefficient, and the number of zero crossings.

Here, the determined time length of the silent section and the waveform of the unvoiced consonant section are recorded in the memory as they are.

Further, the pitch extraction control unit 42 performs a linear prediction analysis on the voice waveform in the voiced sound section using a so-called voice conduction inverse filter to obtain a residual waveform, and performs a correlation analysis on the residual waveform to obtain a correlation peak. Is obtained from the interval of. This is performed for the entire voiced sound section on the unit voice.

Next, for each of the obtained pitch periods, the connection with the unit sound (at the past time) immediately before the connection is changed so that the connection is smoothly connected in terms of audibility, and the phrases and sentences as a whole are adjusted to accept and intonation. To calculate a new pitch period sequence.

That is, first, the average pitch period of the entire pitch period in the obtained unit voice is determined as the geometric mean pitch in consideration of human hearing.

Here, assuming that Pn is the n-th pitch cycle and the total pitch number in the unit voice is L, the average pitch cycle Pave is expressed as Pave = (P ₁ × P ₂ ×... × P _L ) ^{1 / L.} Pavel the average pitch period of the unit voice of
Where R = Pavel / Pave
And Further, the period change coefficient calculated from the accent rule and the intonation rule is defined as Qn. Further, as shown in FIG. Adjustment is performed by the indicated coefficient Sn so that the pitch period trains are smoothly connected.

Here, P ′ _L is the last pitch cycle of the pitch cycle sequence after the correction of the immediately preceding (past) unit sound.

If the above adjustments are combined and Rn is added to obtain an adjusted component, then Rn = R · Qn · Sn, and new pitch cycle information can be obtained by multiplying Pn by Rn for each of the L pitch cycles. Can be

At this time, the spectrum envelope extraction unit 62 obtains the spectrum envelope of the original voice. That is, as shown in FIG. 4, immediately before the point at which the level of the waveform suddenly increases from the basic unit audio waveform, the start point of the pitch is used, and based on the pitch period first obtained by the pitch extraction control unit 42, the next pitch One pitch section is determined with the end point one sample before the start point of
Windowing of about 20 mSec is performed with the center of one pitch section as the center of the analysis window. This windowing makes it possible to perform a short-time spectrum analysis using a finite number of sample values. Based on the windowing data, a linear prediction analysis is performed again to obtain linear prediction coefficients α _{1 to} α _p.
Is calculated. Where p is the order of the linear prediction analysis,
Generally, P = about 10 is used for a female voice, and P = about 14 is used for a male voice.

Further, the spectrum envelope H (k) of the original speech is obtained using the above-described linear prediction coefficients α _{1 to} α _p by the following equation.

Here, N is a power of 2 which is larger than the number of samples and is 512.

This process is repeated while shifting one pitch section until the voiced sound section ends.

The waveform expansion / contraction unit 44 expands / contracts the waveform for each pitch in accordance with the new pitch cycle information obtained by the pitch extraction control unit 42. That is, when the number of samples per pitch of the basic unit audio waveform is k, and the number of samples corresponding to the changed pitch is k ', if it is desired to reduce the pitch period, the k'th sample point from the start of the pitch section is used. When the waveform is to be cut off and the pitch period is to be extended, on the other hand, the linear prediction coefficients α _{1 to} α _p obtained by the spectrum envelope extraction unit 62 are used to obtain
= K + 1th to m = k'th sample values are obtained to obtain the subsequent waveform.

x (m) = α ₁ x (m−1) + α ₂ x (m−2) +... + α _p x
(MP) However, this process is repeated until the voiced sound section ends while shifting the pitch section by one pitch section. At this time, since the utterance speed changes by the expansion and contraction of the pitch cycle, it is thinned out in units of one pitch cycle, or the same waveform is used. And keep the utterance duration of the basic unit voice. Also, the duration is corrected here based on the prosody information obtained from the voice selecting unit 24 by the same means.

Since the pitch is changed, there is a large discontinuity between the last sample point of the waveform of the pitch section and the start sample point of the next pitch section. Approximation using a cubic curve is performed by the least squares method using the data, and continuous connection is performed.

When the processing by the pitch extraction control unit 42, the waveform expansion / contraction unit 44, and the spectrum envelope extraction unit 62 ends, first, the spectrum envelope extraction unit 46 performs one pitch section of the waveform obtained by changing the pitch period obtained from the waveform expansion / contraction unit 44. In the same manner as described above, windowing of about 20 mSec is performed, and linear prediction analysis is performed on this sample value to calculate linear prediction coefficients α ₁ ′ to α _p ′ .The spectrum envelope after pitch change is calculated by the following equation Ask for.

Here, N is set to 512 as described above.

Next, the spectrum extractor H of the original voice is output by the correction extracting unit 48.
In contrast to (k), the spectral envelope distorted by changing the pitch period Is calculated.

That is, Is calculated for each pitch cycle and stored in the memory.

Further, the spectrum envelope control unit 64 uses the α _{1 to} α _p obtained by the spectrum envelope extraction unit 62 as coefficients to obtain complex numbers Z _{1 to} Z _p that are P roots satisfying the following expression.

1 + α ₁ Z ⁻¹ + α ₂ Z ⁻² +... + Α _p Z ^−p = 0 Among these P roots, there exists a pair of conjugate complex roots.
A pair of conjugate complex roots may correspond to one formant. That is, the resonance frequency F _i and its bandwidth B _i are obtained from these roots Z _{i according} to the following equation, and the voiced sound section in the unit voice is recorded in the memory while the above-described processing is shifted for each pitch section. Repeat until done.

F _i = Fs / (2π) · arg (Z _i ) B _i = Fs / π · | log (| Z _i |) | where Fs is a sampling frequency.

Further, from the series of resonance frequencies, the resonance frequencies having a narrow bandwidth are selected as a first formant, a second formant, a third formant... In order from a lower frequency in consideration of the bandwidth and continuity, and the locus of the formant frequency is selected. Ask for.

In order to improve the connectivity with the immediately preceding unit sound, the spectrum envelope control unit 64 further processes the formant and its trajectory as shown in FIG. Using several samples before and after the starting sample point in the unit voice, interpolation is performed by cubic curve approximation by the least squares method in the same manner as described above, and continuous connection is made.

Next, when the locus of the new formant frequency and the bandwidth are determined as described above, a new linear prediction coefficient is obtained as follows.

That is, the new resonance frequency including the changed formants, the resonance frequencies of the unchanged formants, and the resonance frequencies not recognized as formants
F _i ′, its bandwidth as B _i ′, and a new root Z _i ′ using the following equation:
Ask for. Note that Z _i ′ includes a conjugate complex root pair corresponding to a formant.

Z _i ′ = exp (−πB _i ′ / F _s + j2 πF _i ′ / F _s ) These P-order equations rooted at Z _i ′ are expressed as (1-Z ₁ ′ Z ⁻¹ ) (1-Z ^{_{2 'Z -1) ... (1}} -Z p' Z -1) =
Assuming that the coefficient of Z ^−k when this equation is expanded is β _k , the above equation is expressed as 1 + β ₁ Z ⁻¹ + β ₂ Z ⁻² +... + Β _p Z ^−p = 0. β _{1 to} β _p give new linear prediction coefficients. Using this new linear prediction coefficient, the spectral envelope is Ask for.

Here, N is 512.

The correction component extraction unit 66 compares the spectrum envelope H (k) of the original speech obtained by the spectrum envelope extraction unit 62 with the spectrum changed by changing the formant. Is calculated. That is, Is calculated for each pitch period and recorded in memol.

In the frequency characteristic changing unit 8, first, in the FFT unit 82, a time window w represented by the following equation is used for N samples x (1) to x (N) in the pitch cycle whose pitch has been changed by the pitch extraction control unit 42. (I) is multiplied to obtain y (1) to y (N). That is, y (i) = w (i) · x (i) 1 ≦ i ≦ N where w (i) = 0.5 · {1−cos (πi / L)} 1 ≦ i <L w (i) = 1 L ≦ i <N−L w (i) = 0.5 · [1 + cos ｛π (i−N + L) /｝ L]
NL ≦ i ≦ N The above-mentioned y (i) is subjected to fast Fourier transform at N points, converted into the frequency domain, and set as Y (k).

Next, in the spectrum conversion unit 84, the correction component extraction unit 66
The change component V (k) due to the change of the formant calculated in step (1) and the change U due to the change in the cycle calculated by the correction extractor 48
(K) is used to change Y (k). That is, the frequency domain expression (k) corrected as (k) = U (k) · V (k) · Y (k) 1 ≦ k ≦ N is obtained.

Next, the IFFT unit 86 converts this (k) into a time-domain audio waveform (k) by inverse fast Fourier transform, and reduces the effect of the end distortion at the time of waveform connection among the obtained N data. To do this, a 20mSec Hamming window is applied to the center of the sample data and cut out.

Next, as shown in FIG. 6, y (k) is shifted by 10 mSec, and the above-described series of operations of correcting the spectrum envelope and cutting out is repeated, and a continuous sound is superimposed on the waveform cut out immediately before.

When the processing of one voiced sound section is completed, it is connected to the unvoiced consonant section or the non-voice section stored in the memory, and the process proceeds to the next voiced sound section. In this way, processing of all unit voices is performed, and finally synthesized voices are processed.
D / A conversion and output sound.

By the way, there are still many unknown parts regarding the perceptual characteristics of human speech. How the physical changes in the audio signal waveform affect the sound of the language's phonology, i.e., how it affects human perceptual properties, such as how it sounds when the pitch or prosody of the voice is subtly changed, When trying to test, using a real human as a speaker,
It is generally difficult to produce the exact same sound due to a change in physical condition or to make subtle articulation.

For these reasons, a method of artificially adjusting the generated sound was required. However, when the conventional method is used, unnaturalness occurs due to discontinuity of the tone band vibration frequency and the formant frequency, which causes difficulty in a subtle psychoacoustic experiment.

On the other hand, according to the above-described embodiment of the present invention, fine control can be performed while maintaining the naturalness of voice, so that this difficulty can be solved.

[Effects of the Invention] As is apparent from the above description, according to the present invention, sound quality deterioration due to discontinuity when connecting unit sounds stored in advance is prevented, and the naturalness and individuality of the original sound are affected. Can be synthesized by adding intonation, accent, and the like without giving the character.

Further, according to the present invention, a voice can be synthesized by arbitrarily changing the rhythm of the intonation and the accent and the phonology based on the formant frequency and the like, so that it is possible to provide a method for measuring the perceptual characteristics of human speech to speech.

[Brief description of the drawings]

FIG. 1 is a block diagram of a speech synthesis system showing one embodiment of the present invention, FIG. 2 (A) is a block diagram showing details of the system shown in FIG. 1, and FIG. 2 (B) is FIG. FIG. 3 is a flowchart showing the processing of the system shown in FIG. 3A. FIG. 3 is a diagram for explaining a conversion method that maintains the continuity of the locus of the sound vibration frequency. FIG. 4 shows a waveform segmentation method for the pitch section. FIG. 5 is a diagram showing a conversion method for maintaining the continuity of the formant frequency bandwidth, and FIG. 6 is a diagram showing a method for connecting the processed waveforms. 22 unit voice storage unit, 24 voice selection unit, 42 pitch extraction control unit, 44 waveform expansion unit, 46 spectral envelope extraction unit,
48: Correction extraction unit, 62: Spectrum envelope extraction unit, 64: Spectrum envelope control unit, 66: Correction extraction unit, 82: FFT unit, 8
4 ... Spectrum envelope changing unit, 86 ... IFFT unit.

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields surveyed (Int. Cl. ⁶ , DB name) G10L 3/00-9/18 JOIS file (JICST)

Claims (2)

(57) [Claims] A unit voice information and a prosody information are determined based on output voice information, and corresponding to the unit voice information based on the predetermined unit voice information among unit voice data which is stored in advance and uttered by a human. Select unit voice data to be calculated, and calculate or extract each of the pitch period, spectrum envelope, formant trajectory and unit voice waveform from the selected unit voice data, and connect the calculated or extracted unit voice waveform smoothly. The calculated or extracted pitch period is changed so as to add the prosody information, and the spectrum envelope by the pitch change is calculated in the changed pitch period, and the spectrum envelope by the pitch change and the calculation are calculated. Alternatively, a first spectrum change is calculated based on the extracted spectrum envelope, and Changes the calculated or extracted formant trajectory so as to smoothly connect the extracted unit sound waveforms, calculates a spectrum envelope by a formant change based on the changed formant trajectory, and calculates a spectrum envelope by the formant change. Calculating a second spectrum change based on the calculated or extracted spectrum envelope; and changing a spectrum envelope of a unit sound waveform related to the change of the pitch cycle based on the first and second spectrum changes. A speech synthesis method comprising: connecting the unit speech waveform whose spectrum envelope has been changed, and outputting the connected speech. Means for determining unit voice information and prosody information based on output voice information; and unit voice information based on the determined unit voice information from unit voice data uttered by a human stored in advance. Means for selecting unit voice data corresponding to the above, means for calculating or extracting each of the pitch period, spectral envelope, formant trajectory and unit voice waveform from the selected unit voice data, and the calculated or extracted unit voice Means for changing the calculated or extracted pitch period so as to connect the waveforms smoothly and to add the prosody information, and means for calculating a spectrum envelope by a pitch change in the changed pitch period. Based on the spectral envelope due to the pitch change and the calculated or extracted spectral envelope Means for calculating the spectrum change of the formant; means for changing the calculated or extracted formant trajectory so as to smoothly connect the calculated or extracted unit sound waveform; and formant based on the changed formant trajectory. Means for calculating a spectrum envelope due to the change; means for calculating a second spectrum change based on the spectrum envelope due to the formant change and the calculated or extracted spectrum envelope; and a first and a second spectrum change. Means for changing the spectrum envelope of the unit sound waveform related to the change of the pitch period, and means for outputting the connected sound after connecting the unit sound waveform whose spectrum envelope has been changed. Characteristic speech synthesizer.