JP3444396B2 - Speech synthesis method, its apparatus and program recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a text-to-speech conversion technique using speech units, which produces a synthesized speech that is generated when the fundamental frequency pattern of the generated speech is significantly different from the pattern of the speech units. The present invention relates to a speech synthesizing method intended to prevent quality degradation and prevent quality degradation of synthetic speech that occurs when synthetic speech that is significantly different from a fundamental frequency pattern of original speech is generated in analysis and synthesis.

[0002]

2. Description of the Related Art Conventionally, for example, when converting text into speech, one period waveform is cut out from a prerecorded speech segment for each fundamental period and matched with a fundamental frequency pattern generated from a text analysis result. hand,
It was to rearrange the waveform. This is PS
It is called the OLA method, and is referred to as “Pitch-sync” by M. Moulines et al.
hronous waveform, processing techniques for text-t
o-speech synthesis using diphones "Speech Communic
ation, vol. 9, pp.453-467 (1990-12).

Further, in the analysis and synthesis, the original voice is analyzed to retain the spectral feature amount, and the original voice is synthesized using this spectral feature amount. With conventional technology,
When the fundamental frequency pattern of the voice unit recorded in advance and the fundamental frequency pattern of the voice to be synthesized are greatly different, the quality of the synthesized voice is significantly deteriorated. For these, for example, T. Hirokawa et al. “Segment
Selection and Pitch Modification for High Quality
Speech Synthesis using Waveform Segments "ICSL
P90 pp. 337-340, DH Klatt et al., "Analysi
s, synthesis, and perception of voice quality vari
ations among female and male talkers ”J. Acoust.
Soc. Am. 87 (2), February 1990, 820-857.
Page. For this reason, in the conventional PSOLA method, when the waveform array is directly aligned with the basic frequency pattern generated from the text analysis result, the quality may be significantly deteriorated. Therefore, a flat one with a small change in the basic frequency pattern is used. There was an occasion.

It is considered that the cause of the quality deterioration of the synthesized speech caused when the fundamental frequency of the speech unit is largely changed is that the fundamental frequency and the spectrum do not acoustically match. Therefore, if a large number of speech units having a spectral structure matched with the fundamental frequency are prepared, a synthesized speech with good quality can be obtained. However, it is difficult to utter at the desired fundamental frequency for all speech units,
Even if it is possible, the storage capacity will be huge and the feasibility will be poor.

From this point of view, Japanese Unexamined Patent Publication No. 57-
In 171398 gazette (published on October 21, 1982), spectrum envelope parameter values for a plurality of voices having different fundamental frequencies are stored for each phoneme, and the spectrum envelope parameter of the closest fundamental frequency is used. This is because there are few types of fundamental frequencies, so there is a slight improvement in quality, and the storage capacity is significantly large.

Further, in Japanese Patent Laid-Open No. 7-104795 (published on April 21, 1995), a human voice is modeled, a conversion rule is created, and a spectrum is transformed according to a change in fundamental frequency. . In this method, the modeling of the voice is not always performed accurately, and therefore the conversion rule does not match the human voice correctly, and high quality cannot be expected.

[0007] Further, it is proposed in the Acoustical Society of Japan, March 1996, Proceedings, pages 337 to 338, to change the fundamental frequency and the spectrum to perform speech synthesis. This method is a rough modification in which the spectrum interval is widened when the fundamental frequency F ₀ is increased, and a good quality synthesized speech cannot be obtained. Further, also in the analysis and synthesis, there is a problem that the quality of the synthesized speech is deteriorated when the synthesized speech having the pitch period greatly different from the pitch period of the original speech is generated.

[0008] Note that, after the priority claim date of this application, September 11, 1996, the present inventor has announced some or all of the inventions of this application in the following academic conferences and their collections of papers. A. Kimihito Tanaka, and Masanobu Abe, "A New Fundam
ental Frecuency Mod-ification Algorithm With Trans
formation of Spectrum Envelope According to F0 ”,
1997 International Conference on Acoustics, Speech,
and Signal Processing (ICASSP 97) Vol.II, pp.951-954,
The Institute of Electronics Engineers (IEEE) Signa
l Processing Society, April 21-24, 1997 B. Kimito Tanaka, Masanobu Abe "Text-to-speech synthesis system that transforms spectral envelope according to fundamental frequency" IEICE Technical Report Vol. 96
No. 566, pages 23-30, SP96-13019.
March 7, 1997 (published 6th) The Institute of Electronics, Information and Communication Engineers C.I. Kimito Tanaka, Masanobu Abe "Voice Synthesis Method that Transforms Spectral Envelope According to F0", Acoustical Society of Japan, 1997 Spring Research Presentation, Proceedings I 217-218
March 17, 1997, Acoustical Society of Japan D.A. Presentations in Japan + Collection of papers K. Tanaka, T. Abe "Speech synthesis method that transforms spectral envelope according to fundamental frequency" Acoustical Society of Japan 1996
Autumn Research Conference, Proceedings I pp. 217-218 September 25, 1996, The Acoustical Society of Japan

[0009]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention relates to a natural speech in accordance with the difference between the fundamental frequency of the input speech, that is, the fundamental frequency of the speech unit or the original speech, of the synthesized speech. The transformation processing is performed on the spectrum envelope by utilizing the relationship between the spectrum envelope of s and the fundamental frequency.

For that purpose, for example, a codebook is prepared in advance for each basic frequency range from learning voice data in which the same text is uttered in several basic frequency ranges. In these codebooks, code vectors are associated with each other in one-to-one correspondence between the fundamental frequency ranges. When synthesizing speech, the speech feature amount of the spectral envelope extracted from the input speech is codebook of the fundamental frequency range of the input speech (reference codebook).
Then, the vector envelope is transformed by using and the spectrum envelope is transformed by decoding on the mapping codebook of the fundamental frequency range to be synthesized. In the modified spectrum envelope, the fundamental frequency and the spectrum are acoustically matched, and by using this, high-quality speech synthesis is possible.

A difference vector codebook is prepared by obtaining a difference vector between corresponding code vectors of the reference codebook and a codebook of another fundamental frequency range, and further, a difference vector codebook is prepared. A difference frequency codebook is prepared by obtaining the difference between the average values of the fundamental frequencies of the element vectors belonging to each corresponding class with the codebook, and the spectrum envelope of the input speech is vector-quantized by the reference codebook, and the quantization is performed. A difference vector corresponding to the code is obtained from the difference vector codebook, and a difference frequency corresponding to the quantized code is obtained from the difference frequency codebook, and the difference frequency, the fundamental frequency of the input voice, and the desired fundamental frequency. The expansion ratio is calculated from the difference between the two fundamental frequencies and To extend and retract the difference vector Te,
The expanded / contracted difference vector is added to the spectrum envelope of the input voice, and the added spectrum envelope is transformed into the time domain to obtain a speech unit with a modified spectrum envelope. In this case, it is possible to modify the spectrum envelope that matches any fundamental frequency different from the fundamental frequency range for which the codebook was created. According to the invention of claim 1
For example, input speech segment waveform (hereinafter referred to as input speech)
To a voice with a desired fundamental frequency different from the fundamental frequency of
Learning with different fundamental frequency ranges
Input audio fundamental frequency range created from audio data
A codebook created for the spectral envelope of
Below, this codebook is referred to as the reference codebook),
With each code vector and input voice of this reference codebook
Corresponding code bases of code books with different basic frequency ranges
Difference vector code block consisting of the difference vector
And the spectral envelope of the input speech above
Vector quantization using the standard codebook and
The difference vector corresponding to the quantized code is
The difference vector obtained from the difference vector codebook
The desired fundamental for the fundamental frequency of the input voice.
Expands and contracts according to the amount of frequency shift, and expands and contracts the differential vector.
And the vector of the vector quantized code above
Is added to the spectral envelope of the input speech above.
The result is a deformed one.

[0012]

1 shows the basic procedure of the present invention. In step S1, the spectrum feature amount of the input voice is extracted, and in step S2, the process of transforming into the spectrum envelope of the input voice is performed using the relationship between the fundamental frequency and the spectrum envelope according to the fundamental frequency difference between the input voice and the synthesized voice. And then
Get synthetic speech.

An embodiment in which the present invention is applied to text-to-speech synthesis will be described below. A text-to-speech synthesis system using speech units analyzes an input text and obtains a sequence of speech units and a fundamental frequency pattern used for synthesis. When the fundamental frequency pattern of the speech unit to be synthesized and the fundamental frequency pattern originally possessed by the speech unit are largely different, in the present invention, the fundamental frequency pattern of the speech unit, in accordance with the amount of deformation with respect to the given fundamental frequency pattern, Transform the spectral envelope of a speech unit. For this modification, the speech unit, that is, the spectral feature quantity of the input speech waveform is extracted, which is performed as shown in FIG. In addition, it is assumed that all the voice data used here are provided with pitch marks representing boundaries of phonemes and fundamental periods.

FIG. 2 is a procedure for extracting a voice feature amount representing spectral envelope information for efficiently expressing a voice signal. This method is a method of estimating the spectrum envelope by the least square approximation of the cosine model by sampling the maximum value of the logarithmic spectrum in the vicinity of an integer multiple of the fundamental frequency (H. Matsumoto et al. “A Minimum Distortion SpectralMap
ping Applied to Voice Quality Conversion "ICS
LP90, 5, 9, pp. 161-164 (1990))
Is an improvement of.

When the voice waveform is input, step S10
In 1, a window function having a length of, for example, 5 times the basic period is applied to the center of the pitch mark to cut out the waveform. Step S1
In 02, the cut-out waveform is subjected to FFT (Fast Fourier Transform) to obtain a logarithmic power spectrum. Step S1
In 03, the logarithmic power spectrum obtained in step S102 is in the vicinity of an integral multiple of the fundamental frequency F ₀ (nF ₀
In _{-F 0/2 <f n <} nF 0 + F 0/2), sampling the maximum value of the log power spectrum. Here, n represents an integer. That is, as shown in FIG. 3, the frequencies F ₀ , 2F ₀ , 3F _0, ...
The maximum value of each logarithmic power spectrum in the interval of ₀ is extracted. The example of the adjacent frequency f ₃ of the maximum value extracted by the interval centered on 3F ₀ is at 3F ₀ less 4F
_The maximum frequency f ₄ extracted in the section centered around ₀
Is higher than 4F ₀ and the difference ΔF between f ₃ and f ₄ , that is, the section where the adjacent sampling interval is larger than 1.5 F ₀ , the maximum value of the logarithmic power spectrum in the section f _{3 to} f ₄ is also sampled.

In step S104, step S103
The sampling points obtained in step 1 are interpolated with a straight line. Step
In step S105, the linear interpolation pattern obtained in step S104 is
Turn, F ₀ / M <50Hz maximum F₀ / M
Sampling at intervals. Here, m represents an integer.

In step S106, step S105
The sampling points sampled by
Least squares approximation is performed using the cosine model shown.           Y (λ) = Σ^M _{i = 1} A_icosiλ, (0 ≦ λ ≦ π) (1) From the above equation (1), the voice feature amount (cepstrum) A_iBut
I want it. This speech feature extraction method uses the power spectrum
Faithfully represents the ark. This audio feature A _iExtraction of
The method is called the IPSE method.

Next, an algorithm for creating a codebook having different fundamental frequency ranges used for transforming the spectrum envelope will be described with reference to FIG. Here, as an example, consider a case where the range of the fundamental frequency has three stages of “high”, “medium”, and “low”. The voice data (learning voice data) used as input has three basic frequency ranges,
One speaker speaks the same text.

In FIG. 5, steps S201 and S20 are performed.
2, in S203, the basic frequency range "high",
From each of the "medium" and "low" voice data, the voice feature amount, in this example, the IPSE cepstrum is extracted for each pitch mark by the algorithm shown in FIG. Step S2
In 04, S205, and S206, step S20 is performed.
The IPSE cepstrum extracted in S1, S202, and S203 is changed to a mel scale in the frequency scale in order to improve the auditory characteristics, and is referred to as a mel IPSE cepstrum. For the Mel scale, for example, “Computation of Spectra with Uneq
ual Resolution Using theFast Fourier Transform ”Pr
oceeding of The IEEE February 1971, 299-3
It is shown on page 01.

In step S207, as shown in FIG. 4, between the pitch mark string in the voice data of the basic frequency range "high" and the pitch mark string of the voice data in the basic frequency range "medium" for the same text. Then, linear expansion / contraction matching is performed for each voiced phoneme to find the correspondence between the pitch marks of both voice data. That is, the pitch mark sequence in the voice data of the fundamental frequency range "high" of the voiced phoneme A is H1, H2, H3, H4, H5, and the pitch mark sequence in the voice data of the basic frequency range "medium" is M.
If it is 1, M2, M3, M4, H1 is M1 and H
2 is associated with M2, H3 and H4 are associated with M3, and H5 is associated with M4. In this way, each pitch mark in the corresponding phoneme section of the fundamental frequency range “high” and the fundamental frequency range “medium” is timed. The axes are linearly expanded and contracted so that those that are close to each other in the section are associated with each other. Similarly in step S208, the basic frequency range is "low".
Then, the correspondence between the pitch marks is obtained between the voice data of No. 1 and the voice data of the basic frequency range “medium”.

In step S209, the voice feature amount (mel IPSE cepstrum) extracted for each pitch mark from the voice data in the basic frequency range "medium" is clustered by the LBG algorithm, and the codebook CB _M in the basic frequency range "medium" is obtained. create. For details of the LBG algorithm, see "An Algorithm" by Linde et al.
mfor Vector Quantization Design, "(IEEE CO
M-28 (1980-01) pp. 84-95).

In step S210, step S209
Vector quantizing the mel IPSE cepstrum in the fundamental frequency range "medium" using the codebook in the fundamental frequency range "medium" created in. That is, the cluster to which the mel IPSE cepstrum in the basic frequency range “medium” belongs is obtained. In step S211, the result of the correspondence between the pitch marks of the audio data of the basic frequency range "high" and the audio data of the basic frequency range "medium" obtained in step S207 is used to generate the codebook created in step S209. For each code vector, each voice feature amount (mel IPSE cepstrum) extracted from the voice data of the corresponding basic frequency range “high” is assigned to the code vector class. That is, for example, the feature amount (mel IPSE) in the pitch mark H1 (FIG. 4) of the voiced phoneme A is measured.
Cepstrum) belongs to the class of code vector numbers in which the feature quantity (mel IPSE cepstrum) in the pitch mark M1 is quantized, and the feature quantity in H2 belongs to the class of quantized code vector numbers of the feature quantity in M2. Each of the feature quantities in H3 and H4 belongs to the class of the quantized code vector number of the feature quantity in M3, and the feature quantity in H5 is the class of the quantized code vector number of the feature quantity in M4. Each "high" feature amount (mel IPSE cepstrum) is classified into a quantized code vector number of a corresponding feature amount (mel IPSE cepstrum) in the basic frequency range "medium". Clustering is performed on the feature amount (mel IPSE cepstrum) of the audio data in the basic frequency range “high”.

In step S212, the clustered mel IPSE cepstrum of the basic frequency range "high" is obtained for each class for the centroid vector (average) of the feature quantities belonging to it, and this is calculated as the basic frequency range "high". The codebook CB _H is obtained as the code vector of In this way, take time corresponding to each cycle waveform, the spectrum for the audio data of the basic frequency range "high" with reference to the results of clustering in the codebook (the reference codebook) CB _M of the fundamental frequency range, "middle" A mapping codebook, which is a mapping destination of parameters, is created. Also in step S213, step S
Using the same method as 211, the fundamental frequency range "low"
Of the voice data of the above (mel IPSE cepstrum) are clustered, and the centroid vector of the feature amount of each class is obtained in step S214 to obtain the basic frequency range "low".
To create a code book CB _L.

From the above, the basic frequency range "low",
Three codebooks CB _L , CB _M , and CB _H were created in which code vectors having the same code number were respectively associated with each other for three of “medium” and “high”. Next, in step S215, the codebook CB _{H of the} basic frequency range “high” and the basic frequency range “medium”.
Then, the difference of each corresponding code vector between the code books CB _M of the above is obtained, and a difference vector code book CB _MH is created. Similarly the codebook CB _L and the fundamental frequency range of the step S216 the fundamental frequency range, "low", "medium"
The difference of each corresponding code vector between the codebooks CB _M of the above is calculated, and a difference vector codebook CB _ML is created.

In this embodiment, further, step S217,
In S218 and S219, in each codebook CB _H and CB
Determining _M, the average value F _H of the fundamental frequency that is included with the element vector belonging to respective classes of CB _L, F _M, the F _L, respectively. In step S220, codebooks CB _H and CB _M
Frequency mean value F _H between corresponding code vectors between
And the difference ΔF _HM between F _M and F _M are obtained to create an average frequency difference codebook CB _FMH . Similarly, in step S221, the difference ΔF _LM between the frequency average values F _M and F _L between the corresponding code vectors between the codebooks CB _M and CB _L is obtained to create the average frequency difference codebook CB _FML .

In this embodiment, five codebooks CB _M of the basic frequency range “medium”, two difference vector codebooks CB _MH and CB _ML , and two average frequency difference codebooks CB _FMH and CB _FML are prepared. To be done. Next, the processing procedure of the speech synthesis method for performing the spectral envelope transformation according to the fundamental frequency using the five mapping codebooks created by the method shown in FIG. 5 will be described with reference to FIG. The input of this algorithm is the speech unit waveform selected in the text-to-speech synthesis unit, the fundamental frequency F _{0t of the} speech to be synthesized, and the fundamental frequency F _{0u of the} selected speech unit waveform. The output is a synthetic voice. Hereinafter, each processing will be described in detail.

In step S401, a voice feature amount, in this example, IPSE cepstrum, is extracted from the input voice segment by a method similar to the algorithm described in steps S201 to S203 in FIG. Further step S
In 402, the frequency scale of the extracted IPSE cepstrum is converted into a mel scale to be a mel IPSE cepstrum.

In step S403, the voice feature quantity extracted in step S402 is fuzzy vector quantized using the codebook CB _M of the fundamental frequency range "medium" created by the algorithm shown in FIG. The k-neighborhood fuzzy class function μ _k as shown in FIG. μ _k = (1 / (Σ (d _k / d _j ) ^{1 / (f-1)} (2) d _j is the distance between the input vector and the code vector, f is the fuzziness, and Σ is j = 1 to j = K For details of fuzzy vector quantization, see Nakamura,
Kano's "Normalization of spectrograms using fuzzy vector quantization" (Academic Society of Japan, Vol. 45, No. 2 (1989))
Or (A. Ho-Ping Tseng, Michael J. Sabin and Edward
A Lee, "Fuzzy Vector Quantazation Applied to Hidde
n Markov Modeling ", Proceedings of IEEE Internationa
l Conference on Acoustics, Speech, and Signal Proces
sing (ICASSP) Vol.2, pp.641-644, April 1987.).

In step S404, as shown in equation (3),
And the difference vector codebook CB _HMOr CB_HLUsing
Difference vector V in k-neighborhood_iAgainst fuzzy
Class function μ_kThe weighted synthesis by
The difference vector V with respect to it is calculated.   V = Σμ_jV_j/ Σμ_j                                        (3) Σ is from j = 1 to k The fundamental frequency F of the voice you want to synthesize_0tIs the input speech unit F
_0uCodebook CB if higher_HMAnd if lower
Is the codebook CB_MLTo use. Such a difference vector
The method to obtain the rule V is the so-called moving vector field smoothing method.
This is the same as the method by Hashimoto and Higuchi.
“Voice quality conversion using speaker selection and motion vector field smoothing
Spectrum map for "The Institute of Electronics, Information and Communication Engineers, IEICE
Report SP95-1 (1995-051)
Makoto Hasimoto and Norio Higuchi, "Spectral Mappin
g for Voice Conversion Using Speaker Selection and
 Vector Field Smoothing ", Proceedings of 4th Europ
ean Conference on Speech Communication and Technon
ogy (EUROSPEECH) Vol.1, pp.431-434, Sept.95.
Le field smoothing method).

In step S405, the fundamental frequency F _{0t of the} speech to be synthesized, the fundamental frequency F _0u of the input speech segment, and the average frequency difference codebook CB _FMH or C _found in FIG.
The expansion / contraction ratio r for the difference vector V is obtained by the equation (4) using B _{FM L.} r = (F _0t −F _0u ) / ΔF (4) ΔF = Σμ _j ΔF _j / Σμ _j (5) Σ is from j = 1 to k, ΔF _j is the codebook CB _FMH
Alternatively, it is the difference between the code average fundamental frequencies of CB _FML .

In step S406, the difference vector V calculated in step S405 is linearly expanded / contracted according to the expansion / contraction ratio r calculated in step S406. In step S407, the difference vector linearly expanded and contracted in step S406 is added to the mel IPSE cepstrum (input vector) obtained in step S402, and the fundamental frequency F _{0t of the} speech to be synthesized is added.
The mel IPSE cepstrum deformed according to is obtained.

In step S408, the modified IP
The SE cepstrum is transformed from Mel scale to linear scale by Oppenheim's recurrence formula. Step S
In 409, the IPSE cepstrum that has been used as the linear scale is subjected to inverse fast Fourier transform (zero phase) to obtain a speech waveform whose spectrum envelope is modified according to F _0t .

In step S410, the voice waveform obtained in step S409 is low-pass filtered to obtain a waveform of only low-frequency components. In step S411, step S4
From the speech waveform obtained in 09, only the high frequency component is extracted by the high pass filter. The cutoff frequency of the high pass filter is made equal to the cutoff frequency of the low pass filter used in step S410.

In step S412, from the input speech unit,
A waveform is cut out by applying a Hamming window having a length twice the basic period centering on the pitch mark position. Step S413
Then, the input waveform cut out in step S412 is passed through the same high-pass filter as that used in step S411 to extract high-pass components. In step S414, step S
The level of the high frequency component of the input waveform obtained in 413 is adjusted so as to be the same level as the high frequency component of the speech waveform whose spectrum envelope is deformed, obtained in step S411.

In step S415, in step S414
Extract the high-frequency component whose level has been adjusted in step S410
The low frequency components thus generated are added together. Step S416
Then, the waveform obtained in step S415 is changed to the desired fundamental frequency.
Number F _0tTo produce a synthetic voice. Above
Figure 7 shows a conceptual representation of the process of transforming the spectral envelope.
Will come to you. Input vector (obtained in step S402
Mel IPSE Cepstrum) Codebook CB_MAt
K vector for quantized vector 11
And its code code 12
Book CB_HDifference vector V from the corresponding code vector of
_iIs the codebook CB_MHThen, in equation (3)
From the fuzzy vector quantized vector 11
Then, the difference vector V is obtained, and this V is also applied to the equation (4).
The linearly expanded and contracted vector with the expansion / contraction ratio r
The target vector is transformed by adding the input vector to
A vector (mel IPSE cepstrum) 14 was obtained.
It Difference vector codebook CB_MH, CB_MLUse
Without codebook CB_H, CB_LCan also be used
it can. FIG. 8 shows the same processing as that of FIG.
One step number is attached and shown.

In this case, in order to simplify the processing, the mel scale conversion is not performed, but the mel scale conversion may be performed. In step S801, a codebook having a frequency closest to the basic frequency of the voice to be synthesized is selected from the basic frequency ranges “high” and “low”. In step S802, the fundamental frequency range selected in step S801, for example, "high"
Using the codebook CB _H , the speech feature quantity fuzzy vector quantized in step S403 is decoded.

In step S409, step S8
The vector (voice feature quantity) decoded in 02 is IFF
A voice waveform is obtained by performing T (Inverse Fast Fourier Transform). In step S410, the voice waveform obtained in step S409 is filtered by a low-pass filter to obtain a waveform of only low-pass components.

In this example, steps S411 and S in FIG.
If 414 is omitted or simplified, step S415
Then, the waveform of only the low frequency component obtained in step S410 and the waveform of only the high frequency component obtained in step S413 are added together. The subsequent processing is the same as in FIG. 1
For example, a technique for extracting the code vector corresponding to the code vector in one code book CB _M from another code book CB _H and changing the nature of the voice is disclosed in Reference H. Matsumot.
o "A Minimum Distortion Spectral Mapping Applied t
o Voice Quality Conversion "ICSLP90 161
~ 164.

In the speech synthesis algorithm shown in FIG. 8, a moving vector field smoothing method is used instead of the fuzzy vector quantization of the speech feature quantity in S403, and the basic frequency range is set in the basic frequency range "medium" codebook. It is also possible to perform vector quantization of the "medium" voice data, obtain a movement vector to the codebook in the basic frequency range to be synthesized, and decode at the movement destination.

Further, not only in the case where the movement vector to the codebook is obtained by the fuzzy vector quantization or the movement vector field smoothing method in step S403, one input feature quantity is converted into one vector as in the ordinary vector quantization. It may be quantized as a code. However, if the fuzzy vector quantization or the moving vector field smoothing method is used, the continuity of the time domain signal obtained in step S416 is superior to the above case.

The low-pass component is extracted by the low-pass filter in step S410. The component in which the difference between the fundamental frequency pattern of the input speech unit and the fundamental frequency pattern to be synthesized affects the spectrum envelope is extracted, and step S413 is performed. On the contrary, the high-pass filter of (1) takes out the high-frequency component that has almost no effect on the spectrum envelope due to the difference (change) in the fundamental frequency pattern. The boundary frequency between the low frequency component and the high frequency component is selected to be about 500 to 2000 Hz.

The input speech waveform is first separated into a low frequency component and a high frequency component, and the step S4 of FIG. 6 or FIG. 8 is performed, respectively.
01, S412 may be passed. In the above description, the present invention is applied so that the fundamental frequency and the spectrum of the synthesized speech match when the difference between the input speech unit and the input fundamental frequency pattern in text synthesis is large. The present invention can be applied not only to this case but also to general waveform synthesis, and also in the case of analysis and synthesis, the present invention can be applied to a case where the fundamental frequency of the synthesized speech is relatively different from the fundamental frequency of the analyzed original speech. By applying, good quality synthetic speech can be obtained. In this case, the original voice is used as the input voice waveform of FIG. 6, and the codebook of the basic frequency range “medium”, that is,
The reference codebook may be created by the same method as described above for the fundamental frequency range of the original voice.

In the analysis and synthesis, the original speech corresponds to the input speech unit (input speech waveform) in the above-mentioned embodiment, and this original speech is normally quantized as a vector code of the feature quantity, which is decoded. When the present invention is applied to analysis and synthesis, the vector code may be decoded in step S802 using a codebook corresponding to the fundamental frequency of the synthesized speech in FIG. 8, for example. . To apply the method shown in FIG. 6 to the analysis and synthesis, the vector code of the speech to be synthesized and the corresponding code vector and difference vector are extracted from the codebook CB _M and the difference vector codebook CB _MH or CB _ML , respectively. Depending on the difference between the fundamental frequency of the original voice and the fundamental frequency of the voice to be synthesized, the expansion / contraction rate is obtained, and by this expansion / contraction rate, the extracted difference vector is expanded / contracted, and this and the extracted code vector are added. Good.

The above-mentioned voice synthesis processing is usually a DSP.
The program is decoded and executed by (Digital Signal Processor) and processed. Therefore, the program for that is recorded on the recording medium. A listening experiment when the present invention is applied to text synthesis will be described. ATR phoneme balance 520 words uttered by one female speaker at three different pitches: high pitch, medium pitch, and low pitch. For each pitch, 327 were made into a codebook and 74 were made into evaluation data. The experimental conditions used for the sampling frequency are 12 KHz, the band separation frequency is 500 Hz (step S4
10, the cutoff frequency of the filter in S411, S413), the codebook size 512, the cepstrum order (feature amount obtained by the method of FIG. 2) 30th, the number of k neighborhoods 12, and the fuzzyness 1.5.

Next, in order to evaluate whether the modification of the spectrum envelope by codebook mapping is effective in improving the quality of synthesized speech, a listening experiment of the fundamental frequency modified speech was conducted. In the experiment, the basic frequency pattern of natural speech B having the same text as natural speech A but different fundamental frequency range is transformed into that of natural speech A by the conventional PSOLA method (conventional technique: synthetic speech (1 )), Correct speech (natural speech A) input (synthetic sound (2)), Fig. 6
By the method shown in (3), the fundamental frequency pattern of natural speech B is transformed into that of natural speech A (synthesized speech (3)).
Two synthetic voices were evaluated by the ABX method. A, B
Are synthetic sounds (1) and (3) respectively, and X is a synthetic sound.
Using (1) to (3), the subject was made to judge whether X was closer to A or B. The basic frequency pattern is modified from a medium pitch (average basic frequency 216 Hz) to a low pitch (average basic frequency 172 Hz) and a medium pitch to a high pitch (average basic frequency 310 Hz). It was realized by replacing. The expansion / contraction rate r of the difference vector is 1.
It was fixed at 0, and the power and the phoneme duration were matched with the word to which the fundamental frequency was transformed. There are 12 test subjects. The judgment rate CR (CR = Pj / Pa * 100 (%)) was determined from the results of the listening experiment. Here, Pj is the number of times X is determined to be close to the synthetic sound (3), and Pa is the number of presentations. The results are shown in FIGS. 9A and 9B.

FIG. 9A is for the conversion from medium pitch to low pitch. The judgment rate of natural speech (2) is 85%, and the judgment rate of natural speech when raising from medium pitch to high pitch (FIG. 9B) is Since it is 59%, it is understood that according to the present invention, it is possible to synthesize a fundamental frequency modified voice closer to a natural voice as compared with the conventional PSOLA method. It can be seen that the present invention is very effective especially when the fundamental frequency is lowered.

The case where the method shown in FIG. 6 is applied to text-to-speech synthesis is compared with the case where the conventional PSOLA method is applied. 5 extracted from 503 sentences of ATR phoneme balance
Two sentences were synthesized in a pitch range of "low", "medium", and "high", and evaluated by a preference test. In order to avoid the influence of the unnaturalness of the pitch pattern obtained from the rule on the test, the pitch pattern extracted from the natural speech was used as the fundamental frequency pattern of the "medium" pitch. Raise the pitch range to raise the pitch to "high" and lower it to lower the pitch to "low".
The pitch pattern was prepared and used for synthesis. The codebook used for the spectral envelope transformation is the same as that used in the previous experiment, and the experimental conditions are also the same as in the previous case. The results are shown in FIGS. 10A, 10B, 10C. A is a low pitch range, B is a medium pitch range, and C is a high pitch range. From this result, it is understood that the method of the present invention is preferred by the subject for the synthesized sounds having the pitch ranges of "low" and "medium" as compared with the PSOLA method.

The method of the present invention shown in FIG. 8, the conventional method (PSOLA method) and the listening experiment are shown. The experimental conditions are the same as the previous case except that the band separation frequency is 1500 Hz. In an experiment in which the fundamental frequency modified speech synthesized by the conventional waveform synthesis method and the method according to the present invention are compared by a listening experiment, in order to see the maximum potential of the method according to the present invention, a modification of the low frequency spectrum envelope (IPSE) is performed. Assuming that is completely completed, the spectral envelope (correct spectral envelope) extracted from the word of the fundamental frequency pattern transformation destination was input. The modification of the basic frequency pattern was realized by changing the basic frequency pattern of the same word speech with different pitch range from high pitch to low pitch and from low pitch to high pitch. Moreover, the power and the phoneme continuation part were matched with the FO transformation destination word. As for the evaluation, the superiority and inferiority of five words were compared in five stages. There are eight test subjects. The results of this experiment are shown in FIG. 11A. From this figure,
It can be seen that the synthesized speech produced by the method of the present invention has considerably higher quality than the synthetic speech produced by the conventional waveform synthesis.

Evaluation 1 in FIG. 11A is very good in the conventional waveform synthesis, evaluation 2 is slightly better in the conventional waveform synthesis, evaluation 3 is the same, evaluation 4 is a little better in the method of the present invention, Evaluation 5 shows that the method of the present invention is much better. An experiment similar to the experiment result shown in FIG. 9 was performed. The experimental conditions are the same as the previous case except that the band separation frequency is 1500 Hz. The result is shown in FIG. 11B,
Shown in C. B is the transformation from medium pitch to low pitch,
C is the transformation from medium pitch to high pitch.

The judgment rates of the synthetic sounds (1) and (2) are 21 when the fundamental frequency is changed from the medium pitch to the low pitch.
% And 91%, and 10% and 94% for medium to high pitches. Further, the determination rate of the synthetic sound (3) is 90% from the middle pitch to the low pitch and 85% from the middle pitch to the high pitch, and it can be seen that the low-frequency spectrum envelope can be appropriately transformed by the codebook mapping. Considering together with the result of FIG. 10A, it can be seen that the speech synthesis method of the present invention can synthesize higher-quality fundamental frequency modified speech as compared with the conventional waveform synthesis method.

[0051]

As described above, according to the present invention, for example, in a text-to-speech synthesis system, it is possible to prevent the quality of synthesized speech from being deteriorated by largely synthesizing the fundamental frequency pattern of the speech unit. Is possible. As a result, it is possible to synthesize higher quality speech as compared with the conventional text speech synthesis system. Further, in the analysis and synthesis, it is possible to obtain high quality synthetic speech even if the fundamental frequency is relatively different from the original speech. That is, in order to synthesize a more human voice and emotional voice, it is necessary to modify the fundamental frequency pattern in various ways, but according to the present invention, such a voice can be synthesized with high quality.

[Brief description of drawings]

FIG. 1 is a diagram showing a basic procedure of the principle of the present invention.

FIG. 2 is a flowchart showing an algorithm for extracting a spectrum envelope from a voice waveform in the present invention.

FIG. 3 is a diagram for explaining a maximum sampling point in the algorithm of FIG.

FIG. 4 is a diagram for explaining correspondence between pitch marks between voice data having different fundamental frequency ranges in the present invention.

FIG. 5 is a flowchart showing a method of creating three mapping codebooks to be incorporated in advance in the text-to-speech synthesis system according to the embodiment of the present invention.

FIG. 6 is a flowchart showing an algorithm for transforming the spectrum envelope of a speech unit according to a fundamental frequency pattern to be synthesized in the embodiment of the present invention.

FIG. 7 is a diagram showing a concept of a spectrum envelope transformation process using the difference vector shown in FIG.

FIG. 8 is a flowchart showing an algorithm for modifying the spectrum envelope of a speech unit according to a fundamental frequency pattern to be synthesized in another embodiment of the present invention.

9A and 9B are diagrams showing experimental results for explaining the effect of the embodiment shown in FIG.

10A and 10B are diagrams showing other experimental results for explaining the effect of the embodiment shown in FIG.

11A to 11C are diagrams showing experimental results for explaining the effect of the embodiment shown in FIG.

Continuation of front page Application for application of Article 30 (1) of the Patent Law ・ Kito Tanaka, Masanobu Abe, application of speech synthesis method to transform spectrum envelope according to F0 to rule synthesis system, Acoustics Society of Japan, Spring 1997 Research Presentation, Proceedings, 2-7-1, p. 217-218, March 17, 1997, application for application of Article 30, Paragraph 1 of the Patent Act ・ Kimihito Tanaka, Masanobu Abe, A New Funding Frequency Requirement Alignment Requirement Alignment Requirement Alignment Alignment Requirement Alignment Requirement Alignment Requirement Requirement Alignment of Alignment Requirement Requirement Alignment Alignment Requirement Alignment Alignment of the Scope ICASSP97, Vol. II, p. 951-954, 1997.4.21 (56) Reference JP 56-55999 (JP, A) JP 1-237600 (JP, A) JP 1-97997 (JP, A) JP 7-104792 (JP, A) JP-A-8-248994 (JP, A) JP-A-9-152892 (JP, A) K. Tanaka, Masanobu Abe, Speech synthesis that transforms the spectral envelope according to the fundamental frequency. Method, Proceedings of the 1996 Autumn Research Conference of ASJ, September 25, 1996, 1-4-14, p. 217-218 (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 13/00 G10L 13/06 G10L 21/04 JISST file (JOIS)

Claims (10)

(57) [Claims] 1. An input speech segment waveform (hereinafter referred to as an input speech).
Vinegar), made from training speech data of different basic frequency range in a speech synthesis method for synthesizing the sound <br/> voice desired fundamental frequency different from its fundamental frequency, spectrum of the fundamental frequency range of the input speech The codebook created for the envelope (hereinafter , this codebook is referred to as the reference codebook ) , each code vector of this reference codebook , and the input
By using the differential vector codebook consisting difference vector between corresponding code vectors of different codebook click of force speech and the fundamental frequency range, the spectral envelope of the input speech, and vector quantized using the reference codebook, the vector The difference vector corresponding to the quantized code is obtained from the difference vector codebook, and the difference vector is set to the fundamental frequency of the input speech.
That the desired base according to the amount Re without frequency stretch, the difference vector that expands and contracts, the vector quantized code vector and the input speech from those obtained by adding the
Speech synthesis method characterized by relative spectral envelope obtain those modified process. 2. The speech synthesis method according to claim 1, wherein the vector quantization is fuzzy vector quantization. 3. A one speech synthesis method according to claim 1 or 2, wherein creating the reference codebook clustered by statistical methods the spectral envelope of learning speech data in the same basic frequency range and the input speech , Between the input voice and the learning voice data having a different fundamental frequency range, and the learning voice data having the same fundamental frequency range as the input voice, the time axis of the pitch mark in each voiced phoneme in the same text. The linear expansion / contraction matching is performed above, the time correspondence is obtained for each period waveform, and the codebook having a different fundamental frequency range from the input speech is created with reference to the clustering result in the reference codebook. Characteristic speech synthesis method. 4. A either speech synthesis method according to claim 1 to 3, wherein, on a logarithmic power spectrum, samples the maximum value in the vicinity of integral multiples of the fundamental frequency, interpolating between the sampling points by a straight line, A speech synthesis method characterized by sampling the interpolated linear pattern at equal intervals, approximating the sampling sequence with a cosine model, and using the coefficient of the model as the spectral envelope. 5. Any of the speech synthesis method according to claim 1 to 4, wherein the speech synthesis method and performing deformation processing of the spectral envelope only for low-frequency component than a predetermined frequency in the spectral region. 6. Any of the speech synthesis method according to claim 1 to 4, wherein the spectral envelope of the input speech, after conversion into mel scale, perform the above spectral envelope transformation processing, which is the spectrum envelope deformation processing A speech synthesis method characterized by converting an object into a linear scale. 7. Any of the speech synthesis method of claims 1 to 6, wherein, different codebooks is higher than the fundamental frequency range of the input speech fundamental frequency Ren of the fundamental frequency range of the input speech
And a low fundamental frequency range . 8. An input speech segment waveform (hereinafter referred to as an input speech).
In a voice with a desired fundamental frequency different from the fundamental frequency , a spectral envelope of learning voice data having the same fundamental frequency range as the input speech is clustered by a statistical method. Was created from the reference codebook created as above and the learning voice data having the same basic frequency range as the above input voice and the same text as the above learning voice data, in association with the code vector of the above reference codebook. Another range codebook, a difference vector codebook consisting of difference vectors of corresponding code vectors with the reference codebook, and a basic frequency average value of element vectors between corresponding classes of the reference codebook and other range codebooks. The difference frequency codebook consisting of the difference and the spectrum envelope of the input speech are Quantization means for performing vector quantization using the reference codebook, difference vector evaluation means for obtaining the difference vector corresponding to the code quantized by the quantization means using the difference vector codebook, and the input An expansion / contraction ratio calculation means for calculating an expansion / contraction ratio from the fundamental frequency of the voice, the desired fundamental frequency, and the difference frequency obtained from the difference frequency codebook corresponding to the quantized code, and An expansion / contraction means for expanding / contracting the difference vector, a means for adding the expanded / compressed difference vector, and a spectrum envelope of the input speech signal, and a time domain conversion means for converting the added spectrum envelope into a time domain, A voice synthesizing device comprising. 9. The speech synthesizer according to claim 8, wherein a low-pass filter for extracting a low-pass component of the signal converted into the time domain and a cut-off frequency same as a cut-off frequency of the low-pass filter are provided. A voice synthesizing apparatus comprising: a high-pass filter for extracting a high-pass component of the input voice signal; and means for adding an output of the low-pass filter and an output of the high-pass filter. 10. A voice synthesizer according to claim 8 or 9.
A computer-readable recording medium recording a program for causing a computer to function as a storage device.