JP5717097B2 - Hidden Markov model learning device and speech synthesizer for speech synthesis - Google Patents

Description

  The present invention relates to speech synthesis, and more particularly to a technique for generating parameters of a speech synthesis filter using an HMM (Hidden Markov Model).

  The essential technologies for man-machine interface include speech recognition technology and speech synthesis technology. By combining speech recognition and speech synthesis, it is possible to use the latest devices that use speech and that are natural for humans and require complex operation instructions.

  Among these technologies, with regard to speech synthesis technology, it is not necessary to simply utter the desired text, but it is necessary to obtain a more natural utterance. Various schemes have been proposed for this purpose.

  One such method uses HMM. In speech synthesis using an HMM, an HMM for estimating speech rule synthesis parameters from a large number of speeches is learned in advance. At the time of speech synthesis, the input text is analyzed to obtain phoneme label strings, and filter parameters for synthesizing each phoneme included in these phoneme label strings are generated from the above HMM.

  Such a technique is disclosed in Patent Document 1, for example. A basic configuration of the speech synthesizer disclosed in Patent Document 1 is shown in FIG.

  Referring to FIG. 1, a conventional speech synthesis system 40 is roughly divided into a learning device 50 for learning a speech synthesis HMM, an HMM storage unit 52 for storing the learning device 50, and an input text. 54, for each phoneme constituting the input text 54, a synthesis filter parameter for rule synthesis and an F0 parameter for speech generation are generated using the HMM stored in the HMM storage unit 52. And a speech synthesizer 56 for synthesizing speech.

  The learning device 50 includes a speech database 60 that stores a large number of speech data labeled by phoneme. Audio is framed with a predetermined frame length and a predetermined shift length. The learning device 50 further includes an F0 extraction processing unit 62 for extracting a fundamental frequency (F0) for each frame of speech stored in the speech database 60, and an acoustic parameter for each frame of speech stored in the speech database 60. MFCC (Mel Frequency Cepstrum Coefficient) 64, phoneme labels, F0 extracted by the F0 extraction processing unit 62, and MFCC calculation unit for each frame of audio data stored in the audio database 60 HMM learning data storage unit 66 that stores a set of MFCCs calculated by H.64 as HMM learning data, and HMM learning data stored in HMM learning data storage unit 66 HTS two to do Kit contains an HMM learning section 68 using (Reference 1), HMM learning by HMM learning section 68 is performed is stored in the HMM storage 52. Typically, the HMM stored in the HMM storage unit 52 is a context-dependent triphone HMM.

  On the other hand, the speech synthesizer 56 performs text analysis on the input text 54 and receives a phoneme label sequence 82 and a text analysis unit 80 that outputs a phoneme label sequence 82 to which prosodic information that the synthesized speech should have is attached. , F0 parameter for speech synthesis by selecting and connecting the most suitable HMM from the HMM storage unit 52 based on the context and prosodic information of each phoneme for each phoneme in the phoneme label sequence 82 from the HMM storage unit 52 In accordance with the F0 parameter sequence generated by the parameter generation unit 84, the parameter generation unit 84 that generates the sequence and the MFCC parameter sequence, and the MFCC generated by the parameter generation unit 84 The sound source signal generated by the sound source generator 86 is filtered according to the parameter series of By grayed (modulation), and a synthesis filter 88 to generate a synthesized speech signal.

  It is known that speech synthesis using such an HMM is fast, flexible for speakers, and flexible for various utterance styles. However, in speech synthesis using HMM, the synthesized speech often becomes unnatural due to generalization processing. In order to solve such a problem, a method using a dynamic feature amount of speech and global variation has been proposed. For example, the MFCC difference (delta) and the difference between the differences (delta-delta) are used as dynamic features.

JP 2011-02811 A

  Problems in speech synthesis using HMM can be divided into the following three aspects.

  (1) Since the voice parameters are statistically processed and smoothed when the HMM is generated, the sound quality is deteriorated.

  (2) Since the voices of various speakers are used, the voice change acts as noise and the sound quality deteriorates.

  (3) In a non-standardized voice recording environment, synthesized voices are distorted because voices of different utterance styles of different speakers are used for HMM learning.

  As for the first aspect, it is known that it is necessary to include not only the amplitude but also the phase in the MFCC parameter. However, information about such phases is usually not available. Strictly speaking, the MFCC parameter having no phase information should be considered as a non-linear parameter from the viewpoint of generating the feature amount of the utterance. Accordingly, the MFCC parameters of various phases are statistically processed and averaged during HMM learning, so that the synthesized speech is distorted. Such distortion causes buzz noise.

  Regarding the second aspect, the susceptibility to utterance change can be considered as one of the sources of noise.

  The third aspect is a serious problem for users who are not experts to communicate using speech synthesis.

  Regarding buzz noise, it has been found that the use of dynamic acoustic features (MFCC delta and delta-delta) as described above significantly improves speech. When such a method is used, it is necessary to use the feature values of a plurality of frames before and after the frame for calculating the feature value of the frame. That is, the response of the MFCC parameter extends over a plurality of frames as well as one frame.

  When a window is used for signal processing by such a method, it is necessary to maintain attributes that do not cause interference between spectra. Otherwise, there is a problem that the synthesized speech is distorted.

  Therefore, an object of the present invention is to provide a speech synthesizer using an HMM, which can suppress the occurrence of distortion in a synthesized speech waveform, and an HMM learning device therefor.

  The hidden Markov model learning device for speech synthesis according to the first aspect of the present invention includes a speech database storage means for storing a speech database including a plurality of speech units each having a phoneme label, and a plurality of speech A fundamental frequency extracting means for extracting a fundamental frequency from each unit and outputting fundamental frequency information; and an acoustic feature quantity calculating means for calculating a predetermined acoustic feature quantity for each of a plurality of speech units. . The hidden Markov model learning device further performs a frequency domain sampling that is dual with a time domain sampling for calculating a predetermined acoustic feature, thereby obtaining a predetermined acoustic feature for each of a plurality of speech units. For a plurality of speech units included in the speech database and the conversion means for converting into angle quantities, the fundamental frequency information output by the fundamental frequency extraction means, and the angle quantities output by the conversion means are labeled with the speech units. Learning means for learning hidden Markov models for different phoneme contexts and learning decision trees for selecting either hidden Markov models from phoneme label sequences, using the attached learning data, and learning Storage means for storing the hidden Markov model learned by the means and the decision tree.

  Preferably, the predetermined acoustic feature amount includes MFCC. The acoustic feature quantity calculating means may include means for calculating an MFCC up to a predetermined dimension for each of a plurality of sound units.

  A speech synthesizer according to the second aspect of the present invention uses a hidden Markov model learned by any of the above-described hidden Markov model learning devices for speech synthesis to synthesize speech for input text. This is a speech synthesizer. This speech synthesizer uses a text analysis means for outputting a phoneme label string by performing text analysis on the text and a phoneme label string output by the text analysis means, and determines a decision tree for each phoneme label. To select a hidden Markov model, and based on the hidden Markov model, parameter generation means for generating fundamental frequency information and angle amount, and a sound source signal based on the fundamental frequency information generated by the parameter generation means Sound source generating means for generating. The speech synthesizer further includes an inverse conversion unit for performing a conversion corresponding to the inverse conversion of the conversion by the conversion unit on the angular amount generated by the parameter generation unit, and calculating a predetermined acoustic feature amount, and an inverse conversion unit And a synthesis filter for modulating the sound source signal generated by the sound source generation means, based on the filter characteristics based on the acoustic feature value converted by.

It is a block diagram which shows schematic structure of the conventional speech synthesis system. 1 is a block diagram showing a schematic configuration of a speech synthesis system 100 according to an embodiment of the present invention. FIG. 3 is a schematic diagram showing a configuration of a decision tree for selecting an HMM in the system shown in FIG. 2. It is a graph which shows the experimental result for showing the effect of the speech synthesis by the system shown in FIG.

  In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated.

[Constitution]
In this embodiment, in order to reduce the distortion of the synthesized speech, in-band shaping is used in which the MFCC parameters are shaped without increasing the bandwidth of the speech signal. Therefore, in this embodiment, dual sampling is used. Dual sampling in this specification means sampling in both time domain and frequency domain. Based on this dual sampling, voice parameters are dual quantized. Further, in-band waveform shaping (not increasing the band) by anti-aliasing filtering and smoothing is performed on the MFCC parameters.

  Referring to FIG. 2, speech synthesis system 100 according to one embodiment of the present invention stores learning device 110 corresponding to learning device 50 shown in FIG. 1 and an HMM that has been learned by learning device 110. And a speech synthesizer 116 corresponding to the speech synthesizer 56 shown in FIG.

  The learning device 110 is different from the learning device 50 (see FIG. 1) in that the MFCC parameter Λ calculated by the MFCC calculation unit 64 for each frame after the MFCC calculation unit 64 in FIG. One FMF extracted by the F0 extraction processing unit 62 instead of the HMF learning data storage unit 52 shown in FIG. And an HMM learning data storage unit 122 that stores the parameter Θ calculated for each frame by the MFCC conversion unit 120 together with a label of the frame as HMM learning data, and the HMM in FIG. Instead of the learning unit 68, the HMM learning data stored in the HMM learning data storage unit 122 is used for speech synthesis. Perform learning of HMM, it is a point including the HMM learning section 124 made of the same HTS toolkit and HMM learning section 68 (Reference 1). The HMM after the learning is completed is stored in the HMM storage unit 112 instead of the HMM storage unit 52 in FIG. 1. The HMM storage unit 112 and the HMM storage unit 52 have the HMM parameters stored therein. The only difference is the hardware.

  The speech synthesizer 116 shown in FIG. 2 differs from the speech synthesizer 56 shown in FIG. 1 in that the phoneme label string 82 is received instead of the parameter generator 84 and the phoneme label and prosody information are received from the HMM storage unit 112. The MFCC converter 120 shown in FIG. 2 receives a point including a parameter generation unit 134 that selects a suitable HMM and outputs an F0 sequence and a parameter Θ sequence, and a parameter Θ sequence output from the parameter generation unit 134. The process includes a MFCC reverse conversion unit 136 that outputs a MFCC sequence by performing a process having a reverse relationship to the process performed in, and sets the synthesized filter 88.

  Hereinafter, a method of calculating the MFCC parameter Λ from the parameter Θ performed by the MFCC conversion unit 120 and the parameter Θ performed by the MFCC inverse conversion unit 136 will be described. The processing in the MFCC conversion unit 120 corresponds to dual sampling and dual quantization.

  Basically, dual sampling can give an accurate reconstruction of functions that change over time. In dual quantization, speech parameters are encoded in both time and frequency based on the result of dual sampling. Dual quantization provides some room for frequency bandwidth limitations. In-band shaping reduces distortion of synthesized speech due to noise and utterance fluidity, and improves the quality of synthesized speech by HMM.

  Dual sampling means sampling a band-limited signal in both time and frequency domain. The sample pair at each sampling point is coherent with each other.

  Dual sampling can be expressed as:

ただしＡは対称な共鳴曲線を表し，λは周波数比の二乗を表し，ζは強制振動の減衰係数を表し，ζ^２＜０．５である。ｎは整数でｎ＝０，…，Ｎ，本実施の形態ではＮ＝１０^６，ε_ｎはｎにより変化する，ほぼ１０^−１０程度の小さな値である。

Where A represents a symmetrical resonance curve, λ represents the square of the frequency ratio, ζ represents the damping coefficient of forced vibration, and ζ ² <0.5. n is an integer and n = 0,..., N, and in this embodiment, N = 10 ⁶ , and ε _n is a small value of about 10 ⁻¹⁰ that varies with n.

ζ _n is further converted into a rotation angle α _n (radian) around the unit circle by the following equation.

Therefore, the nth sampling point λ _n (0 <λ _n <1) is in reverse order with respect to the angle α _n (0 <αn <w _c , but fixed to w _c = 0.33325 radians in this embodiment). Make a dual. Further, by calculating the θn which folded alpha _n around the zero point alpha _z according to the following equation, theta _n is the variable with the same order as the lambda _n.

この折り返しの関係から，周波数領域におけるデュアルサンプリングは，平行移動に関して不変であり，かつ線形であるということができる。したがって，離散周波数系は線形かつ平行移動に関して不変であり，離散時間系も同様である。

From this aliasing relationship, it can be said that dual sampling in the frequency domain is invariant and linear with respect to translation. Therefore, discrete frequency systems are linear and invariant with respect to translation, as are discrete time systems.

Dual quantization for MFCC can be expressed as: The MFCC coefficients k-th dimension and lambda _k, the maximum value Λ _kmax MFCCΛ _k is the minimum value _{Λ kmin (k = 0, ...} , K: K is the maximum dimension of the dimension number) shall be in the range of between.

Here, Λ _k is resampled and quantized in the time domain by the following equation.

ただしＱ［ｘ］はｘを最も近いλ_ｎ，ｎ∈｛０，…，Ｎ｝に丸めることを示す。

However, Q [x] indicates that x is rounded to the nearest λ _n , nε {0,.

_Assume that θ _nk is dual with λ _nk and that the relationship between θ _m and λ _n is looked up as a look-up table. The frequency domain dual function for Λ _k is expressed by the following equation.

Information represented by Λ _k with phase (if possible) in the time domain exists in 3 / 2-dimensional (circular) space, not in 1-dimensional (linear) space. Roughly speaking, the mapping from Λ _k to Θ _k is geometrically a mapping from a 3/2 dimensional external plane represented by λ _nk to a 2 dimensional sphere represented by θ _nk. It can be said that there is. By re-sampling in the frequency domain, the information is randomly distributed on the sphere Θ _k if no phase is considered. When phase information is not included in Λk, it can be assumed that phase information need not be considered.

  In-band waveform shaping is closely related to HMM learning and speech parameter generation in this embodiment. Basically, the procedures for incorporating these techniques into the generation of speech by HMM include:

<Parameterization>
Convert MFCC into angular quantities.

For all utterances in the utterance corpus, for example, MFCC is calculated with K = 39 and frame shift = 5 milliseconds. MFCC is denoted by Λ _ki (k = 0,..., K, i = 0,..., I. I indicates the number of frames of speech). Find Λ _kmax and Λ _kmin from the set of _MFCCs and map all of Λ _ki to Θ _ki .

<Learning HMM>
The MFCC is extended to the remaining bandwidth, and decoding is performed according to the maximum likelihood criterion. The HTS toolkit (reference document 1) is used for this work, but the band is expanded to 1.4 times for in-band shaping by using γ _e × Θ _ki instead of Λ _ki .

<Speech synthesis>
Perform anti-aliasing and smoothing. GV (indicated by ^ Θ _kj, where k = 0,..., K, J = 0,..., J. J is the number of frames in speech.) First, Θ _kj is converted to α _kj . If α _kj > w _c , α _kj = w _c is set to reduce aliasing. Thereafter, α _kj is quantized to any α _nkj in {α _n , n = 0,..., N}. A minimum error criterion is used for this quantization. Further, α _nkj is multiplied by γ _c to smooth the band by 1.2, and the result is quantized again. Finally, recalculate the MFCC by mapping α _nkj to Λ _nkj . When this mapping is one-to-many, in this embodiment, an arbitrary one of the maps is selected at random. As a result, Λ _kj , k = 0,..., K and j = 0 _,.

<HMM after learning>
The learned HMM stored in the HMM learning data storage unit 122 will be described with reference to FIG. In this embodiment, the HMM is a context-dependent three-state HMM. For example, consider HMMs 140, 142, and 144 that include / a / as an intermediate phoneme. These have / a / as the second phoneme 160 but have c ₁₁ , c ₂₁ and c ₃₁ as the first phoneme, respectively, and c ₁₂ , c ₂₂ and c ₃₂ as the third phoneme, respectively. To do. There can be many other three-state HMMs having / a / in the second phoneme, but only these three HMMs 140, 142, and 144 are shown here for easy understanding of the drawing.

  In order to select one of the HMMs having / a / as the second phoneme 160, learning of the decision tree 162 related to the HMM is performed. The decision tree 162 has a plurality of nodes 180 to 200, for example. Among these, the nodes 184, 188, 190, 196, 198 and 200 are leaf nodes and correspond to any one of the HMMs 140 to 144 and the like. A binary question is associated with each node of the decision tree 162, and the decision tree 162 is answered while answering the question of each node according to the speech synthesis condition (determined by a label string having prosodic information). Is selected from the root node 180, and the HMM corresponding to the reached leaf node is selected.

[Operation]
The speech synthesis system 100 shown in FIG. 2 operates as follows. In the voice database 60, a large number of utterance data are prepared as a voice database. These speech data are all framed and phoneme-labeled. The F0 extraction processing unit 62 extracts F0 from each frame in the audio database 60 and outputs it. The MFCC calculation unit 64 calculates an MFCC parameter Λ _ki from each frame and supplies it to the MFCC conversion unit 120. MFCC conversion unit 120 finds the lambda _kmax and lambda _kmin from a set of MFCC as described above, to map all lambda _ki to theta _ki.

The F0 and Θ _ki calculated for each frame are assigned the phoneme label of that frame and stored in the HMM learning data storage unit 122.

The entity of the HMM learning unit 124 is an HTS toolkit similar to the HMM learning unit 68 as described above, and learns the HMM in the HMM storage unit 112 using Θ _ki . When the HMM learning is completed for all utterance data, it is possible to synthesize speech using the HMM storage unit 112.

In speech synthesis, when input text 54 is given, the text analysis unit 80 of the speech synthesizer 116 performs text analysis on the input text 54 and gives a phoneme label sequence 82 with prosodic information to the parameter generation unit 134. The parameter generation unit 134 selects the HMM corresponding to each phoneme by following the decision tree 162 (see FIG. 3) stored in the HMM storage unit 112 using the given phoneme label string with prosodic information. The sequence of is output. Corresponding to this sequence, a sequence of F0 is also obtained and given to the sound source generator 86. Each Θ _kj obtained from the HMM sequence is converted to α _kj . If α _kj > w _c , α _kj = w _c is set to reduce aliasing. Further, α _kj is quantized to any α _nkj in {α _n , n = 0,..., N}. A minimum error criterion is used for this quantization. Further, α _nkj is multiplied by γ _c for smoothing, and the result is quantized again. Finally, recalculate the MFCC by mapping α _nkj to Λ _nkj . If this mapping is one-to-many, an arbitrary one of the maps is selected at random. As a result, a sequence of Λ _kj (k = 0,..., K and j = 0,..., J) is obtained as the MFCC parameter. A synthesis filter 88 is set for each frame by each of the MFCC parameters Λ _ki constituting this sequence, and the synthesis filter 88 filters the sound source signal generated by the sound source generation unit 86 based on F0 for the frame, thereby synthesizing the synthesis filter 88. Voice is obtained.

[Effect of the embodiment]
As described above, according to the present embodiment, the samples at the dual sampling points in the time and frequency domains are coherent. If there is some change in either one, the corresponding change will occur in the other. This depends on the resonance curve and the equilibrium conditions. That is, the value of ζ is selected so that the values of the input λ and the output λ are equal to each other. As a result, the dual sampling provides a basic framework for quantizing speech parameters in both the time and frequency domains, and allows speech parameters to be processed in both domains.

  Second, since the object to be processed is a circle in the frequency domain, the “amplitude” is constant, and thus the statistical average value is represented by an angular amount that is linear.

Third, the MFCC quantization basically extracts 0.3535 × 10 ⁶ positions out of 10 ⁶ positions defined by dual sampling, and performs further interpolation if necessary. There is room for it. Such room for phase information that is not available is suitable for dealing with noise caused by statistically averaging Θ _k when learning HMMs. However, it is assumed that this noise exhibits the same statistical characteristics as Gaussian noise. It is well known that human hearing is insensitive to a certain amount of phase. Thus, a means for efficiently statistically classifying and averaging speech parameters is obtained.

  Fourth, vocoders typically make use of grouping certain frequency groups, particularly high frequency groups, to a significant degree. Dual sampling in the frequency domain meets this requirement. The degree of compression of the high frequency is about 2.5 times that of the low frequency.

Finally, by multiplying the parameter Θ _k by the linear coefficient γ, a simple means for enabling group delay in the time domain by using the dual sampling is obtained.

[Usage example]
Using the ATR503 data set by a small number of female speakers, an experiment was conducted comparing the method according to the above embodiment with the conventional method. The results are shown in FIG. FIG. 4 shows the result of in-band shaping of the MFCC when the MFCC response is expanded to a frame larger than 1. This result shows that the number of leaf nodes in the present invention is generally smaller than that of the conventional method, and the diversity of acoustic features is reduced. This is because, as a result of the method according to the above embodiment, the speaker-specific features and the universal features are well separated, and the averaging that the speakers-specific features are subjected to during HMM learning has been improved. Means.

  As a result of the inventors listening and evaluating the speech synthesized by the above method, it has been confirmed that the buzz noise is considerably reduced by the present embodiment and the sound quality of the synthesized speech by the HMM is improved as compared with the conventional method. It was.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

[References]
[1] K. Tokuda, H .; Zen, J. et al. Yamagishi, T .; Masuko, S .; Sako, A .; B. Black, T.M. Nose, “The HMM-Based Speech Synthesis System (HTS) Version 2.1.” [Online]. URL: http://hts.sp.nitech.ac.jp/.

40, 100 Speech synthesis system 50, 110 Learning device 52, 112 HMM storage unit 54 Input text 56, 116 Speech synthesis device 60 Speech database 62 F0 extraction processing unit 64 MFCC calculation unit 66, 122 Data storage unit 68, 124 for HMM learning HMM learning unit 80 Text analysis unit 82 Phoneme label sequence 84, 134 Parameter generation unit 86 Sound source generation unit 88 Synthesis filter 136 MFCC inverse conversion unit

Claims (3)

Speech database storage means for storing a speech database including a plurality of speech units each having a phoneme label;
A fundamental frequency extracting means for extracting a fundamental frequency from each of the plurality of voice units and outputting fundamental frequency information;
Acoustic feature amount calculating means for calculating a predetermined acoustic feature amount for each of the plurality of speech units;
The predetermined acoustic feature quantity is converted into an angular quantity for each of the plurality of speech units by performing frequency domain sampling that is dual with the time domain sampling for calculating the predetermined acoustic feature quantity. Conversion means for,
Learning data in which the fundamental frequency information output from the fundamental frequency extraction unit and the angular amount output from the conversion unit are labeled with the unit of the speech unit for the plurality of speech units included in the speech database. Learning means for learning a hidden Markov model for different phoneme contexts and learning a decision tree for selecting one of the hidden Markov models from a phoneme label sequence;
An apparatus for learning a hidden Markov model for speech synthesis, comprising: a storage means for storing the hidden Markov model learned by the learning means and the decision tree. The predetermined acoustic feature amount includes a mel frequency cepstrum coefficient,
2. The hidden Markov model learning device for speech synthesis according to claim 1, wherein the acoustic feature amount calculating means includes means for calculating a mel frequency cepstrum coefficient up to a predetermined dimension for each of the plurality of speech units. . A speech synthesizer for synthesizing speech for input text using a hidden Markov model learned by a speech synthesis hidden Markov model learning device according to claim 1 or 2,
Text analysis means for outputting a phoneme label string by performing text analysis on the text;
Using the phoneme label sequence output by the text analysis means, for each phoneme label, select a hidden Markov model using the decision tree, and generate fundamental frequency information and the angular amount based on the hidden Markov model. Parameter generation means for
Sound source generating means for generating a sound source signal based on the fundamental frequency information generated by the parameter generating means;
An inverse conversion means for calculating the predetermined acoustic feature quantity by performing a conversion corresponding to an inverse conversion of the conversion by the conversion means on the angle amount generated by the parameter generation means;
A speech synthesizer comprising: a synthesis filter for modulating the sound source signal generated by the sound source generation means based on a filter characteristic based on the acoustic feature value converted by the inverse conversion means.

Images