JP6142401B2 - Speech synthesis model learning apparatus, method, and program - Google Patents

Description

  The present invention relates to a speech synthesis model learning apparatus, method, and program, and more particularly, to a speech synthesis model learning apparatus, method, and program for learning a speech synthesis model for synthesizing a speech waveform from text data.

  The basic strategy of a text-to-speech synthesis method based on a statistical model is to create a probabilistic speech generation model, learn its model parameters from learning data, and generate speech for any text data using the learned model. It is to do. Therefore, the quality of the synthesized speech depends on how various properties and behaviors in speech can be described appropriately in the form of a generation model. In particular, focusing on the phoneme of speech, it is important how to properly model the time series of spectral envelope features, but speech synthesis using a conventional Hidden Markov Model (HMM) or its variants (hereinafter referred to as “Hidden Markov Model”) The “HMM speech synthesis” method (see, for example, Non-Patent Document 1) was devised under the idea of capturing time expansion and contraction of a speech spectrum sequence as a stochastic phenomenon.

  In the conventional HMM speech synthesis method, cepstrum and line spectrum pairs (Line Spectral Pairs, LSP) are used as speech feature quantities expressing the spectral envelope. When the cepstrum is used as a feature value, it corresponds to a model that expresses a phenomenon in which the spectral envelope is stochastically fluctuated only in the power direction. Is equivalent to a model that expresses In the HMM speech synthesis method using cepstrum as a feature, the spectrum envelope of the synthesized speech tends to be smoothed in the frequency direction, but this is a model in which the generated model cannot capture fluctuations in the frequency direction of the spectrum well. Is the cause. When the spectral envelope is smoothed, it generally becomes a buzzy sound, which is a well-known trend in conventional HMM speech synthesis.

  Therefore, for example, for the purpose of emphasizing between the peak and the dip of the spectral envelope, improvement has been achieved by introducing Global Variance (GV) into the probabilistic model. It is difficult to restore the peaks and dips that should be, and the fundamental solution has not been reached.

  Since the frequency and power of the spectral envelope peak correspond to the resonant frequency and power of resonance in the vocal tract, the speech spectral envelope actually has fluctuations in both the power direction and the frequency direction. It is considered that the resonance frequency and power continuously change in the time direction according to the physical change of the vocal tract shape. Therefore, for example, when comparing the spectral envelope near the central part of a phoneme and the spectral envelope near the connected part of the subsequent phoneme, the latter is a process in which the vocal tract shape continuously changes to the vocal tract shape of the subsequent phoneme. Therefore, there is a difference in resonance frequency and power between the spectral envelopes of both, and it is important to model this as fluctuations.

  As a speech spectrum model for speech analysis and synthesis systems, a composite wavelet model (Gaussian Mixture Model, GMM) that represents the entire spectrum envelope based on the assumption that each peak of the spectrum envelope can be approximated by a Gaussian distribution. A model called Composite Wavelet Model (CWM) has been proposed (see, for example, Non-Patent Document 2).

  Since CWM has both the frequency and power of the spectrum envelope peak as parameters, it is suitable for probabilistic modeling of fluctuations in both the power direction and frequency direction of the spectrum envelope. When a speech waveform is synthesized from CWM parameters, a Gaussian distribution function in the frequency domain corresponds to a Gabor function in the time domain. Therefore, the speech waveform is synthesized by arranging the Gabor function at time intervals corresponding to the fundamental frequency. Is done. Speech analysis and synthesis based on CWM is a synthesis method using an FIR filter. Compared to conventional synthesis methods using cyclic filters such as LSP and cepstrum, even a filter having a high Q value has a time characteristic that is independent of the fundamental frequency. Good voice can be synthesized.

  Due to the above-mentioned advantages of CWM, an HMM speech synthesis method using CWM parameters as speech feature amounts has been proposed (see, for example, Non-Patent Document 3). In this method, in parameter learning, first, CWM parameter extraction is first performed for the speech spectrum envelope at each time (short-time frame), and a vector sequence in which the extracted CWM parameter sets are arranged is used for speech in HMM speech synthesis. It is a feature quantity series.

T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis", in Proc. Of Eurospeech 1999, 1999, pp. 2347 -2350. Takeya Tsuji et al., “Examination of speech synthesis using composite wavelet model”, Proceedings of the Acoustical Society of Japan 2006 Spring Conference, 2-11-7, 2006. Hojo et al., “HMM speech synthesis based on composite wavelet model analysis and synthesis system”, no.2-2-7, 2012.

  The technique of Non-Patent Document 3 has a problem that sufficient performance cannot be obtained due to the difficulty inherent in the problem of formant frequency estimation. The formant trajectory appears clearly in the speech spectrogram, but it is not easy to extract automatically. This is because a formant that should actually exist cannot be detected, or a formant that should not actually exist is erroneously detected. Since estimation of CWM parameters in each short-time frame can be regarded as equivalent to the formant extraction problem, the same method as in Non-Patent Document 3 often causes similar errors in estimation of CWM parameters in the previous stage. End up.

  FIG. 9 shows an example of the result of estimating CWM parameters for each time (short-time frame) for a certain audio signal sample. In FIG. 9, the center of each Gaussian function in the CWM estimated at each time is plotted with a different marker for each Gaussian function index. As shown in FIG. 9, the indexing method of each Gaussian function in CWM often does not match every time (for example, the elliptical part in FIG. 9). For example, in two different times at which the same phoneme is emitted, the first and second formants are fitted with the first Gaussian function and the second Gaussian function, while the other time In such a case, the second Gaussian function and the third Gaussian function are frequently fitted. Such inconsistencies in the CWM parameter index cause performance degradation in the subsequent HMM speech synthesis parameter learning. This is because the average of the centers of Gaussian functions corresponding to different spectral peaks is calculated when obtaining the average of the feature quantity distributions in the respective states.

  From the above, the spectral representation by the CWM parameter has the advantage that it is suitable for probabilistic modeling of fluctuations in both the power direction and the frequency direction of the peak of the spectral envelope, while estimating the CWM parameter and the HMM parameter. There is a problem that it does not work well with a method that simply connects learning with multiple stages.

  The present invention has been made in view of the above circumstances, and a speech synthesis model learning apparatus capable of learning an HMM using CWM parameters that are guaranteed to have matching indexes in the same state as speech features, It is an object to provide a method and a program.

  In order to achieve the above object, the speech synthesis model learning apparatus of the present invention is expressed by parameters of a composite wavelet model CWM in which a spectral envelope of each time of a speech signal is expressed by a mixed Gaussian model and information obtained from text data. An estimation unit that estimates the CWM parameters by alternately updating the parameters of the Hidden Markov Model HMM that outputs a sequence of the CWM parameters corresponding to the state at each time to maximize the same criterion And a learning unit that learns the HMM using a CWM parameter estimated by the estimation unit and a label indicating the state of each time of the audio signal.

  According to the speech synthesis model learning apparatus of the present invention, the estimation unit represents each parameter represented by information obtained from the parameters of the composite wavelet model CWM in which the spectral envelope of each time of the speech signal is expressed by a mixed Gaussian model and text data. The parameters of the hidden Markov model HMM that outputs a series of CWM parameters corresponding to the time state are alternately updated so as to maximize the same criterion, and the CWM parameters are estimated. Then, the learning unit learns the HMM using the CWM parameter estimated by the estimation unit and the label indicating the state of each time of the audio signal.

  In this way, the same criterion is maximized between the CWM parameters and the parameters of the hidden Markov model HMM that outputs a series of CWM parameters corresponding to the state at each time represented by the information obtained from the text data. Since the HMM is learned using the CWM parameters that are alternately updated and estimated as described above, the HMM can be learned using the CWM parameters that are guaranteed to match the indices of the Gaussian functions in the same state as speech features.

  Further, the estimation unit determines that the same criterion is obtained when the CWM parameter is determined, the probability that the spectrum envelope is output, the probability of the state sequence of the HMM, and the state sequence. , And the probability that the CWM parameter is output.

  In addition, the estimation unit represents the same criterion by the HMM parameter, the CWM parameter, and an auxiliary variable, and the logarithm of the probability that the spectrum envelope is output when the CWM parameter is determined. The HMM parameter, the CWM parameter, and the auxiliary variable can be updated alternately with a function that does not exceed the logarithm and is in contact with the logarithm.

  In addition, the estimation unit may use the same criterion as a lower limit function obtained by Jensen's inequality using the convexity of a negative logarithmic function.

  Further, in the speech synthesis model learning method of the present invention, each of the estimation units is represented by parameters of the composite wavelet model CWM in which the spectrum envelope of each time of the speech signal is expressed by a mixed Gaussian model and information obtained from the text data. Estimating the CWM parameters by alternately updating the parameters of the Hidden Markov Model HMM that outputs a sequence of the CWM parameters corresponding to the state of time so as to maximize the same criterion; And learning the HMM using a CWM parameter estimated by the estimation unit and a label indicating a state of each time of the audio signal.

  The speech synthesis model learning program of the present invention is a program for causing a computer to function as each part constituting the speech synthesis model learning apparatus.

  As described above, according to the speech synthesis model learning apparatus, method, and program of the present invention, CWM parameters and CWM parameter sequences corresponding to the state at each time represented by information obtained from text data. Since the HMM is trained using the CWM parameters estimated by alternately updating the parameters of the hidden Markov model HMM that outputs the same criterion so as to maximize the same criterion, the indices of the Gaussian functions are matched in the same state. As a result, it is possible to learn the HMM using the guaranteed CWM parameter as a voice feature amount.

It is an image figure which shows the outline of HMM which outputs a CWM parameter. It is a functional block diagram which shows schematic structure of the speech synthesizer which concerns on this Embodiment. It is a functional block diagram which shows schematic structure of a CWM parameter estimation part. It is a flowchart which shows a learning process. It is a flowchart which shows a CWM parameter estimation process. It is a flowchart which shows a synthetic | combination process. It is a spectrogram which shows an example of a verification result. It is a graph which shows an example of a verification result. It is a figure for demonstrating the problem of a prior art.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of the present embodiment>
CWM parameters and hidden Markov models in which the indices of each Gaussian function in the composite wavelet model (CWM) in which the entire spectral envelope is expressed by a Gaussian Mixture Model (GMM) are matched in the same state. The fact that an integrated model with (Hidden Markov Model, HMM) has been constructed, and that an iterative algorithm with guaranteed convergence for learning the parameters of the model given learning data has been realized. This is the point of the embodiment. Specifically, it is realized as follows.

1. 1. Update HMM parameters and CWM parameters alternately to increase the same criterion. In the above 1, when the CWM parameter is determined based on the same criterion, the probability that the spectrum envelope is output, the probability of the HMM state sequence, and the CMM parameter is output when the HMM state sequence is determined. 2. Product with probability (or its logarithm) In the above 2, the function that is expressed by the HMM parameter, the CWM parameter, and the auxiliary variable λ, does not exceed the logarithm of the probability that the spectrum envelope is output when the CWM parameter is determined, and is in contact with this is defined as the same criterion. 3. Update HMM parameters, CWM parameters, and auxiliary variables alternately so as to increase this criterion. In 3 above, the same criterion is a lower limit function created using Jensen's inequality using the convexity of the negative logarithmic function.

<Spectrum envelope sequence generation model by CWM>
First, a generation model of a spectrum envelope sequence will be described.

  In the conventional HMM speech synthesis method, an HMM that outputs a cepstrum feature quantity sequence is set up, an output distribution parameter is learned from learning data, and an average cepstrum feature quantity in each state is estimated. However, in this method, a smoothing phenomenon of the spectral envelope occurs. Because the cepstrum is obtained by linear transformation of the spectral envelope, obtaining the average of the cepstrum is equivalent to obtaining the average of the power direction of the spectral envelope. However, if there is a fluctuation in the frequency of the spectrum envelope peak, the peaks and valleys of the spectrum envelope are averaged and smoothed into a gentle shape. Thus, it is considered that the cause of the spectrum smoothing is that the stochastic fluctuation of the cepstrum feature is assumed and the fluctuation only in the power direction of the spectrum envelope is modeled.

Since fluctuations in the speech spectral envelope are thought to include fluctuations in resonance frequency and power based on physical changes in the vocal tract shape, the probability that both fluctuations in the spectral envelope peak frequency and power can be expressed. Should create a static generation model. Therefore, if a CWM having the frequency and power of the spectrum envelope peak as parameters is used, such probability modeling can be performed. The CWM is a model that approximates a spectrum envelope by a GMM and uses the parameters of the GMM as speech feature amounts. In CWM, the spectral envelope f _{ω, l} is expressed by the following equation (1). Note that f _{ω, l} is hereinafter referred to as a “model spectrum envelope”.

However, K is the number of mixed GMMs. μ _k , w _k , and σ _k are the mean, weight, and dispersion parameter of the GMM, respectively, and can be regarded as corresponding to the frequency, power, and sharpness of the model spectrum envelope peak, respectively.

Next, the process of generating an observed spectrum envelope sequence will be described. Consider an HMM that outputs CWM parameters of average μ _{k, l} , reciprocal variance ρ _{k, l} , and weight w _{k, l} at each discrete time l as shown in FIG. Each state of the HMM represents one state of a language label, and for example, as shown in FIG. 1, each state can correspond to one phoneme. In addition to the phoneme state, one state of the context label using information such as the accent positions of the preceding and following phonemes may be associated with the phoneme state, as in the conventional HMM speech synthesis method. In the present embodiment, the probability distribution of the CWM parameter output from each state is assumed to be the following expressions (2) to (4) for the state s _{1 at} each time l.

Here, N (x; m, η ² ) is a normal distribution, and Gamma (x; a, b) is a gamma distribution represented by the following equation (5).

_Given a sequence of CWM parameters ^ μ = {μk} _{k, l} , ^ ρ = {ρ _k } _{k, l} and ^ w = {w _k } _{k, l} , at time l, the observed spectral envelope The probability distribution for generating {y _{w, l} } is the following equation (6). In addition, the symbol of bold notation in a numerical formula, and the symbol preceded by "^" in a sentence represent a matrix or a vector.

Here, fw _{, l} is a spectrum envelope represented by the equation (1) using the CWM parameter at time l when CWM parameter sequences ^ μ, ^ ρ, and ^ w are given, and Poisson ( x; λ) is a Poisson distribution represented by the following equation (7).

  By defining the above generation model, the following parameter estimation algorithm can be applied.

<Parameter estimation algorithm>
Parameter learning (estimation) maximizes the posterior probability P (Θ | Y) of the parameter Θ of the spectrum envelope sequence generation model when the observed spectrum envelope sequence Y = {y _{w, l} } _{w, l} is given. It is formulated as a problem. The parameter Θ to be estimated is the HMM state sequence ^ s = {s ₁ } ₁ , the state output distribution of each state i of the HMM ^ θ = {m _{k, i} , η _{k, i} , a _{k, i} ^(σ) , B _{k, i} ^(σ) , a _{k, i} ^(w) , b _{k, i} ^(w) }, and CWM parameter sequences ^ μ, ^ ρ, and ^ w.

  Although it is difficult to obtain Θ that maximizes the posterior probability P (Θ | Y) of the parameter Θ, it is possible to repeat local optimization for each variable. At this time, P (Θ | Y) can be written as in the following equation (8).

  Here, α is a regularization parameter and represents the weight of the log prior distribution for the log likelihood. A symbol with “C” on “=” means matching except for a constant part.

In the parameter estimation algorithm in the present embodiment, parameter estimation is performed by repeating the minimization of -logP (Θ | Y) for each variable. Here, -logP (Y | Θ) is equivalent to that observed spectrum envelope y _omega at each _{time, l} and the model spectral envelope f _omega, the I-divergence is the pseudo distance between _l sum for all time . I-divergence is indicated by the following (10).

  Thus, maximizing P (Θ | Y) is equivalent to minimizing I (Θ) -αlogP (Θ) for Θ.

  Minimization for the I-divergence term can be performed sequentially for each parameter using the auxiliary function method. That is, applying Jensen's inequality based on the convexity of the logarithmic function yields (12) below.

Here, g _{k, ω, l} is the following equation (13). Further, the condition for establishing the equal sign in equation (12) is the following equation (14).

  Let J (Θ, λ) be the upper limit function of I (Θ), that is, the right side of equation (12). Here, for an arbitrary Θ, when λ is given by equation (14), the auxiliary function J (Θ, λ) −αlogP (Θ) is equal to the objective function I (Θ) −αlogP (Θ). Then, Θ that reduces J (Θ, λ) −αlogP (Θ) for any fixed λ necessarily reduces I (Θ) −αlogP (Θ) according to equation (12). From the above, by repeating the update of λ according to the equation (14) and the update of Θ so as to decrease J (Θ, λ) −αlogP (Θ), the objective function is monotonous until the local optimal solution is reached. To decrease.

<Configuration of speech synthesizer>
The speech synthesizer according to the present embodiment includes a computer including a CPU, a RAM, and a ROM that stores a program for executing a speech synthesis processing routine including a learning process and a synthesis process described later. .

  As shown in FIG. 2, the computer constituting the speech synthesizer 10 can be functionally represented by a configuration including a learning unit 20 and a synthesis unit 40. The learning unit 20 is an example of a speech synthesis model learning device of the present invention.

Furthermore, the learning unit 20 can be represented by a configuration including a fundamental frequency sequence extraction unit 22, an observed spectrum envelope sequence extraction unit 24, a CWM parameter estimation unit 26, and an HMM learning unit 28. The learning section 20, from a database, the label containing information of the time series data and state s _l of each time of the audio signal. The CWM parameter estimation unit 26 is an example of the estimation unit of the present invention, and the HMM learning unit 28 is an example of the learning unit of the present invention.

  The basic frequency series extraction unit 22 extracts time series data of the basic frequency from the time series data of the input audio signal, converts the time series data to be expressed in the discrete time l, and outputs the time of the basic frequency of the audio signal. A basic frequency sequence that is sequence data is extracted. This extraction process of the fundamental frequency can be realized by a well-known technique. For example, Non-Patent Document 4 (H. Kameoka, “Statistical speech spectrum model incorporating all-pole vocal tract model and F0 contour generating process model,” in Tech. Rep. IEICE, 2010, in Japanese.), For example, the fundamental frequency can be extracted every 8 ms. The fundamental frequency sequence extraction unit 22 outputs the extracted fundamental frequency sequence to the HMM learning unit 28.

  The observed spectrum envelope series extraction unit 24 performs Fourier transform on the input time series data of the audio signal for each time (short-time frame) to extract the observed spectrum envelope series Y. The observed spectrum envelope sequence extraction unit 24 outputs the extracted observed spectrum envelope sequence Y to the CWM parameter estimation unit 26.

  The CWM parameter estimating unit 26 receives the observed spectrum envelope sequence Y output from the observed spectrum envelope sequence extracting unit 24 and the label input from the database, and maximizes the observed spectrum envelope sequence posterior probability P (Θ | Y). Estimate the parameter Θ. Then, the CWM parameter estimation unit 26 outputs the CWM parameters ^ μ, ^ ρ, and ^ w included in the estimated parameter Θ to the HMM learning unit 28. Further, as shown in FIG. 3, the CWM parameter estimation unit 26 further includes an initial update unit 260, an auxiliary variable update unit 262, a CWM parameter update unit 264, a first convergence determination unit 266, a state output distribution update unit 268, and a state series update. Unit 270, observed spectrum envelope sequence posterior probability update unit 272, and second convergence determination unit 274.

  The initial update unit 260 performs initial update of the observed spectrum envelope sequence posterior probability P (Θ | Y) using the initial value of the parameter Θ. As initial values of the parameter Θ, values set appropriately in advance are used for the initial values of the state output distribution ^ θ and the CWM parameters ^ μ, ^ ρ, and ^ w. Information included in the input label is used as the initial value of the state sequence ^ s of the HMM.

  The auxiliary variable update unit 262 uses the CWM parameters ^ μ, ^ ρ, and ^ w updated last time, or the CWM parameters ^ μ, ^ ρ, and ^ w set as initial values, according to the equation (14). The auxiliary variable λ is updated.

  The CWM parameter updating unit 264 fixes the state series ^ s and the state output distribution ^ θ with values updated last time or values set as initial values, and sets the auxiliary variable λ updated by the auxiliary variable updating unit 262. And update the CWM parameters ^ μ, ^ ρ, and ^ w with the update formulas of the following formulas (15) to (17) so as to decrease the auxiliary function J (Θ, λ) −αlogP (Θ). To do.

However, C _{k, l} , D _{k, l} , and E _{k, l} are the following formulas (18) to (20).

  The first convergence determination unit 266 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, each process of the auxiliary variable update unit 262 and the CWM parameter update unit 264 is performed. repeat. If the first convergence determination unit 266 determines that the convergence condition is satisfied, the first convergence determination unit 266 outputs the CWM parameters ^ μ, ^ ρ, and ^ w when the convergence condition is satisfied to the state output distribution update unit 268.

As the convergence condition, it may be used that the number of repetitions n ₁ has reached a predetermined number N ₁ (for example, 20 times). It should be noted that the difference between the value of the auxiliary function when the n ₁ −1 parameter is used and the value of the auxiliary function when the n ₁ time parameter is used is smaller than a predetermined threshold. It may be used as a convergence condition.

The state output distribution updating unit 268 fixes the CWM parameters {circumflex over (μ)}, {circumflex over (ρ)}, and {circumflex over (w)} to {circumflex over (μ)}, {circumflex over (ρ)}, and {circumflex over (w)} output from the first convergence determination unit 266 and Is fixed at a value updated last time or a value set as an initial value, and the auxiliary function J (Θ, λ) −αlogP (Θ) is calculated using the auxiliary variable λ updated by the auxiliary variable update unit 262. {M _{k, i} , η _{k, i} ² } _{k, i} included in the state output distribution ^ θ is updated by the update formulas of the following formulas (21) and (22) so as to decrease.

However, Ti = {l | s _l = i}. Also, the update formula for {a _{k, i} ^(ρ) , b _{k, i} ^(ρ) , a _{k, i} ^(w) , b _{k, i} ^(w) } _{k, l} included in the state output distribution ^ θ. Is obtained as the root of the following equations (23) to (26).

  However, (psi) (a) represents the digamma function shown to following (27) Formula.

  The state series update unit 270 fixes the CWM parameters ^ μ, ^ ρ, and ^ w with ^ μ, ^ ρ, and ^ w output from the first convergence determination unit 266, and sets the state output distribution ^ θ. The state sequence ^ s is updated so that the auxiliary function J (Θ, λ) −αlogP (Θ) is decreased by the Viterbi algorithm, fixed at a previously updated value or a value set as an initial value.

  The observed spectrum envelope sequence posterior probability update unit 272 includes CWM parameters ^ μ, ^ ρ, and ^ w updated by the CWM parameter update unit 264, the state output distribution ^ θ updated by the state output distribution update unit 268, and the state The observed spectrum envelope sequence posterior probability P (Θ | Y) is updated using the state sequence ^ s updated by the sequence update unit 270.

  The second convergence determination unit 274 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the auxiliary variable update unit 262, the CWM parameter update unit 264, and the first convergence are determined. Each process of the determination part 266, the state output distribution update part 268, the state series update part 270, and the observed spectrum envelope series posterior probability update part 272 is repeated. If it is determined that the convergence condition is satisfied, the second convergence determination unit 274 outputs the CWM parameters ^ μ, ^ ρ, and ^ w when the convergence condition is satisfied to the HMM learning unit 28.

As the convergence condition, it may be used that the number of repetitions n ₂ has reached a predetermined number N ₂ (for example, 20 times). Note that the difference between the value of the auxiliary function when using the n ₂ -first parameter and the value of the auxiliary function when using the n ₂ -th parameter is smaller than a predetermined threshold value. It may be used as a convergence condition.

  The HMM learning unit 28 uses the CWM parameters ^ μ, ^ ρ, and ^ w output from the CWM parameter estimation unit 26 and the labels input from the database, for example, using conventional techniques such as Non-Patent Document 1. , HMM30 is learned. When a model spectrum envelope sequence is obtained from text data using the learned HMM, the model spectrum envelope sequence obtained simply based on the maximum likelihood criterion becomes discontinuous in the vicinity of the phoneme boundary, resulting in deterioration of the synthesized speech quality. Cause. Therefore, for example, as in the method of Non-Patent Document 1, the phoneme state is finely divided, and further, the HMM 30 is learned using dynamic feature amounts (first and second time difference amounts of feature amounts). As a result, it is possible to learn the HMM 30 that can output a continuous model spectrum envelope sequence. The HMM learning unit 28 stores the learned HMM 30 in a predetermined storage area.

  As shown in FIG. 2, the synthesis unit 40 can be represented by a configuration including a text analysis unit 42, a parameter synthesis unit 44, and a speech waveform synthesis unit 46. Text data is input to the combining unit 40.

  The text analysis unit 42 analyzes the input text data, analyzes a state represented by a label corresponding to each phoneme, for example, and outputs a label series to the parameter synthesis unit 44.

  The parameter synthesis unit 44 uses the HMM 30 learned by the learning unit 20 for the label sequence output from the text analysis unit 42 to obtain a CWM parameter sequence based on the maximum likelihood criterion. A model spectrum envelope sequence can be obtained based on this CWM parameter sequence. Further, the parameter synthesis unit 44 obtains a fundamental frequency sequence based on the label sequence output from the text analysis unit 42. When outputting the CWM parameter series, a model related to duration of phoneme state is separately required. Further, in order to obtain the fundamental frequency sequence from the label sequence, a model related to the fundamental frequency is required separately. As these models, for example, the model described in Non-Patent Document 1 can be used. The parameter synthesizer 44 outputs the obtained fundamental frequency sequence and CWM parameter sequence to the speech waveform synthesizer 46.

  The speech waveform synthesizer 46 synthesizes a speech waveform using the CWM parameter sequence and the fundamental frequency sequence output from the parameter synthesizer 44 by a technique such as Non-Patent Document 2, Non-Patent Document 3, or the like. That is, as shown in the following equation (28), since the GMM in the frequency domain corresponds to a Gabor function in the time domain, a Gabor Wavelet that is a superposition of the Gabor function is generated from the CWM parameter, and the time interval corresponding to the fundamental frequency is generated. The voice waveforms are synthesized by arranging them on the time axis.

  This is a synthesis method using an FIR filter, and speech synthesis with good time characteristics is possible regardless of the fundamental frequency. The speech waveform synthesis unit 46 outputs the synthesized speech waveform.

<Operation of speech synthesizer>
Next, the operation of the speech synthesizer 10 according to the present embodiment will be described. First, the learning section 20, from the database, the label inputs including information of the time series data and state s _l of each time of the audio signal, the learning section 20, by executing the learning processing shown in FIG. 4, HMM30 Is learned. Then, text data is input to the synthesizing unit 40, and the synthesizing unit 40 executes a synthesizing process shown in FIG. Hereinafter, each process is explained in full detail.

  In step S10 of the learning process shown in FIG. 4, the fundamental frequency sequence extraction unit 22 extracts the time series data of the fundamental frequency from the time series data of the input speech signal, and expresses them in the discrete time l. The fundamental frequency sequence, which is the time series data of the fundamental frequency of the audio signal, is extracted and output to the HMM learning unit 28.

  Next, in step S12, the observed spectrum envelope sequence extraction unit 24 performs Fourier transform on the input time series data of the audio signal for each time (short-time frame) to extract the observed spectrum envelope sequence Y, and the CWM parameter It outputs to the estimation part 26.

  Next, in step S14, the CWM parameter estimation unit 26 executes a CWM parameter estimation process shown in FIG.

  In step S140 of the CWM parameter estimation process shown in FIG. 5, the initial update unit 260 uses values appropriately set in advance as initial values of the state output distribution ^ θ and the CWM parameters ^ μ, ^ ρ, and ^ w. , Using the information included in the input label as the initial value of the state sequence ^ s of the HMM, the observed spectrum envelope sequence posterior probability P (Θ | Y) is initially updated.

  Next, in step S142, the auxiliary variable updating unit 262 uses the previously updated CWM parameters ^ μ, ^ ρ, and ^ w, or the CWM parameters ^ μ, ^ ρ, and ^ w set as initial values. Thus, the auxiliary variable λ is updated by the equation (14).

  Next, in step S144, the CWM parameter updating unit 264 fixes the state series ^ s and the state output distribution ^ θ with values updated last time or values set as initial values, and updated in step S142. Using the auxiliary variable λ, the CWM parameters ^ μ, ^ ρ, and ^ w are updated so that the auxiliary function J (Θ, λ) −αlogP (Θ) is decreased. Update with formula.

  Next, in step S146, the first convergence determination unit 266 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the process returns to step S142, and the processes of steps S142 and S144 are repeated. On the other hand, if the convergence condition is satisfied, the CWM parameters ^ μ, ^ ρ, and ^ w when the convergence condition is satisfied are output to the state output distribution update unit 268, and the process proceeds to step S148.

  In step S148, the state output distribution updating unit 268 fixes the CWM parameters ^ μ, ^ ρ, and ^ w with ^ μ, ^ ρ, and ^ w output from the first convergence determination unit 266, and the state The output distribution {circumflex over (θ)} is fixed at the previously updated value or the value set as the initial value, and the auxiliary function J (Θ, λ) −αlogP (Θ) is used by using the auxiliary variable λ updated at step S142. ) Is updated by the equations (21) to (26) so as to reduce the.

  Next, in step S150, the state series update unit 270 fixes the CWM parameters ^ μ, ^ ρ, and ^ w with ^ μ, ^ ρ, and ^ w output from the first convergence determination unit 266. The state output distribution ^ θ is fixed at a value updated last time or a value set as an initial value, and the auxiliary function J (Θ, λ) −αlogP (Θ) is decreased by the Viterbi algorithm. Update the sequence ^ s.

  Note that either step S148 or step S150 may be executed first.

  Next, in step S152, the observed spectrum envelope sequence posterior probability updating unit 272 performs the CWM parameters ^ μ, ^ ρ, and ^ w updated in step S144, and the state output distribution ^ θ updated in step S148. In addition, the observed spectrum envelope sequence posterior probability P (Θ | Y) is updated using the state sequence ^ s updated in step S150.

  Next, in step S154, the second convergence determination unit 274 determines whether or not a predetermined convergence condition is satisfied. If the convergence condition is not satisfied, the process returns to step S142, and the processes of steps S142 to S152 are repeated. On the other hand, when the convergence condition is satisfied, the CWM parameters ^ μ, ^ ρ, and ^ w when the convergence condition is satisfied are output to the HMM learning unit 28, and the process returns to the learning process.

  Next, in step S16 of the learning process shown in FIG. 4, the HMM learning unit 28 uses the CWM parameters ^ μ, ^ ρ, and ^ w output in step S14 and the labels input from the database, For example, the conventional technique such as Non-Patent Document 1 is used to learn the HMM 30, store the learned HMM 30 in a predetermined storage area, and terminate the learning process.

  Next, in step S20 of the synthesizing process shown in FIG. 6, the text analysis unit 42 analyzes the input text data, for example, analyzes a state represented by a label corresponding to each phoneme, and sets the label series as a parameter. The data is output to the combining unit 44.

  Next, in step S22, the parameter synthesizer 44 uses the HMM 30 learned in the learning process shown in FIG. 4 for the label sequence output in step S20 to generate a CWM parameter sequence based on the maximum likelihood criterion. Obtained and output to the speech waveform synthesis unit 46. Further, the parameter synthesizer 44 obtains a fundamental frequency sequence based on the label sequence output in step S20 and outputs it to the speech waveform synthesizer 46.

  Next, in step S24, the speech waveform synthesizer 46 uses the CWM parameter sequence and the fundamental frequency sequence output in step S22, for example, by the technique of Non-Patent Document 2, Non-Patent Document 3, etc. Are synthesized and output, and the synthesis process is terminated.

<Experiment>
Regarding the speech synthesis method using the speech synthesizer 10 according to the present embodiment, a verification result that CWM parameter estimation and speech synthesis can be appropriately performed will be described.

  According to the ATR503 J04 sentence “A ticket is purchased from a vending machine” (A) spectrogram of sample voice (real voice) and (B) the method of the present embodiment (hereinafter referred to as “the present method”). A spectrogram of the synthesized speech is shown in FIG. In addition, the spectrum envelope of the central part of the phoneme / i / of the opening “ticket” is shown in FIG. 8 for the present method (solid line), the conventional method (dashed line), and the real voice (dotted line). Here, the conventional method is a method using a 24th order mel cepstrum (see Non-Patent Document 1).

  As shown in FIG. 7, the spectrogram of the synthesized speech according to the present method is similar to the spectrogram of the real voice, and it is shown that the speech synthesis of text data is possible by the present method. The spectrum envelope reproduced by this method has a tendency to successfully reproduce the spectrum envelope dip mainly at frequencies of 4 kHz to 7 kHz. This can be considered to be a result of the spectral envelope becoming difficult to smooth compared to the conventional method because the CWM parameter captures both the frequency and power fluctuations of the spectral envelope peak.

  On the other hand, at a low frequency of 1 kHz or less, a plurality of spectral envelope peaks are reproduced with gentle curves, and the resonance frequency is unclear, which is considered to cause quality degradation. This is presumably because a plurality of spectral envelope peaks are approximated by a sum of a small number of Gaussian functions when extracting CWM parameters. For example, it may be possible to synthesize speech with a clearer resonance frequency by precisely associating a Gaussian function with each peak of the spectral envelope, such as increasing the number of GMM mixtures.

  As described above, according to the speech synthesizer according to the embodiment of the present invention, the CWM parameter obtained by alternately updating the CWM parameter and the HMM parameter so as to maximize the same criterion is used as the speech feature. By using it as a quantity, it is possible to learn the HMM using the CWM parameter guaranteed to match the index of each Gaussian function in the same state as the speech feature quantity.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, in the above embodiment, the case where the learning unit and the synthesis unit are configured by the same computer has been described. However, the learning unit and the synthesis unit may be configured by different computers.

  The above speech synthesizer has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Speech synthesizer 20 Learning part 22 Fundamental frequency sequence extraction part 24 Observation spectrum envelope series extraction part 26 CWM parameter estimation part 28 HMM learning part 30 HMM
40 synthesis unit 42 text analysis unit 44 parameter synthesis unit 46 speech waveform synthesis unit 260 initial update unit 262 auxiliary variable update unit 264 CWM parameter update unit 266 first convergence determination unit 268 state output distribution update unit 270 state series update unit 272 observation spectrum Envelope sequence posterior probability update unit 274 second convergence determination unit

Claims (5)

A hidden that outputs a series of parameters of a composite wavelet model CWM expressing a spectral envelope of each time of an audio signal by a mixed Gaussian model and a parameter of the CWM corresponding to each time state represented by information obtained from text data An estimation unit for alternately updating the parameters of the Markov model HMM so as to maximize the same criterion, and estimating the parameters of the CWM;
Parameters of CWM estimated by the estimation unit, and with a label indicating the state of each time of the audio signal, seen including a learning section, a learning of the HMM,
Based on the same criteria, when the CWM parameter is determined, the probability that the spectrum envelope is output, the probability of the state sequence of the HMM, and the parameter of the CWM are output when the state sequence is determined. Speech synthesis model learning device as a product of the probability of being played . The estimation unit represents the same criterion by the HMM parameter, the CWM parameter, and an auxiliary variable, and exceeds the logarithm of the probability that the spectrum envelope is output when the CWM parameter is determined. not, and a function that is in contact with the log, the parameter of the HMM, the parameters of the CWM, and voice synthesis model learning device according to claim 1, wherein updating the auxiliary variable alternately. The speech synthesis model learning device according to claim 2 , wherein the estimation unit uses the same criterion as a lower limit function obtained by Jensen's inequality using the convexity of a negative logarithmic function. A series of parameters of the composite wavelet model CWM in which the estimation unit expresses a spectral envelope of each time of the speech signal by a mixed Gaussian model, and a parameter of the CWM corresponding to each time state represented by information obtained from text data Alternately updating parameters of a hidden Markov model HMM that outputs the same criterion to maximize the same criterion, and estimating the parameters of the CWM;
Learning section, parameters of the CWM estimated by the estimation unit, and with a label indicating the state of each time of the speech signal, see contains the steps of: learning the HMM,
Based on the same criteria, when the CWM parameter is determined, the probability that the spectrum envelope is output, the probability of the state sequence of the HMM, and the parameter of the CWM are output when the state sequence is determined. Speech synthesis model learning method as product of the probability of being played . The speech synthesis model learning program for functioning a computer as each part which comprises the speech synthesis model learning apparatus of any one of Claims 1-3 .