JP2004279454A - Method for speech generation model speaker adaptation, and its device, its program, and its recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech generation model adapting method which can adapt a speaking style based upon articulation motion of a speaker and obtain sufficient precision and to adapt not only a sound space but also a speech generation model itself according to articulation motion estimated from a speech while taking dynamic features into consideration. <P>SOLUTION: As articulation and sound mapping of a speech generation model, adaptation into a speech given a statistical speech generation model composed of a linear function of determining a speech spectrum from an articulation parameter is performed by a hidden Markov model (hereinafter HMM) constituted as dynamic models for articulation motion and states of the HMMs. A speech generated by a model is adapted so that the output probability of a given speech becomes maximum, articulation motion maximizing the fact probability is estimated from the given speech by using the adapted speech, and adaptation that maximizes the output probability of the articulation motion and also maximizes the output probability of the given speech for the speech generated from the estimated articulation motion is performed. Consequently, the speech generation model is so adapted to maximize the output probability of the given speech. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a speech generation model including a probabilistic dynamic model expressing articulation movement (hereinafter referred to as an articulation model) and an articulation / acoustic mapping coefficient for associating an articulation parameter vector with an audio parameter vector. The present invention relates to a speech generation model adaptation method, a device, a program, and a recording medium for adapting using a speech signal of a user.
[0002]
[Prior art]
A hidden Markov model (hereinafter referred to as HMM) describing the dynamic behavior of articulatory motion, an articulatory parameter vector and a speech spectrum (speech parameter vector) of the articulatory motion, as an inverse estimation method of the articulatory motion of the voice from the voice signal. Proposed a method for inversely estimating the articulation motion (articulation parameter vector sequence) of a speech from a speech signal based on a speech generation model composed of articulation and acoustic mapping coefficients for functionally approximating the relationship of 1).
[0003]
However, many studies on inverse estimation of articulatory movements focus on specific speakers. So far, research on speech input from unspecified speakers has considered a speaker adaptation method based on an inverse estimation method using a neural network. (S. Dusan and L. Deng, "Vocal-Tact Length Normalization for Acoustic-to-Articulatory Mapping Utilizing Neural Networks," in The 138.^th  Meeting of the Acoustic Society of America, 1999. ).
In addition, there is a method of adapting the parameters of a model having a certain voice parameter so that the output probability (likelihood) of the input voice parameter is maximized (Non-Patent Document 2).
[0004]
[Non-patent document 1]
Sadao Hiroya and Masaaki Honda, “Acoustic-to-articulatory inverse mapping using an HMM-based speech production model,” in ICSLP, 2002, pp. 2305-2308.
[Non-patent document 2]
C. J. Leggetter and P.M. C. Woodland, "Maximum Likelihood linear regression for speakers adaptation of continuity density hidden markovov models," in Computer magazine. 9, pp. 171-185, 1995.
[0005]
[Problems to be solved by the invention]
Conventional research on inversion estimation of articulatory motion for speech input of unspecified speakers based on vocal tract length normalization adapts the acoustic space of the input speech to the acoustic space of the model of the specific speaker. . However, due to the redundancy between the speech spectrum and articulatory movements, the adaptation of the acoustic space does not directly lead to the adaptation of the articulatory movements, and thus the adaptation of the acoustic space alone will adapt the speaking style based on the articulatory movements of the speaker. Can not. In addition, in the study of conventional adaptation methods, parameters are adapted to increase the likelihood of an input parameter sequence for a certain model parameter. Since it is not used, sufficient accuracy cannot be obtained. An object of the present invention is to provide a method, an apparatus, a program, and a recording medium for adapting a speech generation model capable of adapting an utterance style based on articulatory movements of a speaker and obtaining sufficient accuracy. In other words, the problem to be solved by the present invention is not only the adaptation of the acoustic space, but also the adaptation of the speech generation model itself based on the articulation motion estimated from the speech in consideration of the dynamic features. I can.
[0006]
[Means for Solving the Problems]
According to the present invention, (1) adapting the speech spectrum generated by the existing speech generation model to the input speech spectrum from the speech signal of the input speaker, thereby providing the speech generation model to the input speaker. Adapting or (2) determining the articulation of the input speech from the input speaker's speech signal based on the existing speech production model, and using the determined articulation motion, the articulation in the existing speech production model. A probabilistic dynamic model of the motion trajectory (referred to as articulation model) and, if necessary, a mapping coefficient relating the speech spectrum vector to the articulation parameter vector are adapted to the input speaker.
[0007]
In the method (1), the speech spectrum (parameter) vector generated by the existing speech generation model is adapted to maximize the output probability of the speech spectrum (parameter) vector sequence of the input speech of the input speaker. . Adapting the articulatory / acoustic mapping coefficients in the speech production model using the relation coefficients relating the adaptation.
According to the method (2), an articulatory motion (articulatory parameter vector sequence) that maximizes the posterior probability of the input speaker with respect to the input speech spectrum (parameter) vector sequence is determined using an existing speech generation model. The articulatory model is adapted so that the output probability of the performed articulatory movement (articulatory parameter vector sequence) is maximized. Further, if necessary, articulation / acoustic mapping is performed such that the output probability of the input speech spectrum (parameter) vector sequence of the input speaker with respect to the speech spectrum (parameter) vector generated from the determined articulation motion is maximized. Adapt coefficients.
[0008]
The method (2) is a combination of the method (1), and firstly converts the speech spectrum (parameter) vector generated by the existing speech generation model into the speech spectrum (parameter) of the input speech of the input speaker. After adaptation to the vector sequence, articulation motion (articulation parameter vector) is determined from the input speech spectrum (parameter) vector sequence using the adapted speech spectrum (parameter) vector, and the determined articulation motion is determined. Is used to at least adapt the articulatory model and, if necessary, the articulatory / acoustic mapping coefficients.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
First, a method for creating a speech generation model to be adapted in the present invention will be described.
Model creation
An articulatory / sound pair codebook is created using an audio signal obtained by continuously uttering sentences and articulatory data simultaneously observed by a magnetic sensor system. The audio signal is cut out by a Blackman window having a window length of 32 ms at a rate of, for example, 250 times per second for each frame, and subjected to spectrum analysis. For example, a 16th-order mel-cepstral coefficient excluding the 0th-order term is obtained as an audio parameter. If necessary, velocity and acceleration parameters are detected as temporal changes by differentiation (difference) from the voice parameters, and a vector having these voice parameters and the speed and acceleration parameters as elements is generated as a voice parameter vector y.
[0010]
At each of a plurality of simultaneously observed articulatory positions, for example, the lower jaw, upper and lower lips, four points on the tongue, and eight positions of the soft palate, each position information signal in the horizontal and vertical directions is 250 per second. The velocity parameter is taken in at each rate, and the velocity parameter as a temporal change is obtained from the position parameter by differentiation (difference) as needed, and the temporal change is obtained by differentiation (difference) of each velocity parameter as needed. Is obtained. An articulation parameter vector x having these 16 position parameters, velocity parameters, and acceleration parameters as elements is generated.
[0011]
That is, in this example, the voice parameter vector y and the articulation parameter vector x are vectors composed of 48 elements, respectively, as described below.
y = [k₁, ……, k₁₆, K₁', ……, k₁₆', K₁″, ……, k₁₆″]
x = [p_a  , ……, p_n  , P_a', ..., p_n', P_a″, ……, p_n″]
A plurality of, for example, 200,000 sets of pairs of the speech parameter vector y and the articulation parameter vector x obtained at the same point in time as described above are stored to form the articulation / sound pair codebook.
[0012]
A probabilistic dynamic model (hereinafter referred to as an articulatory model) that expresses articulation using the articulatory parameter vector x and the speech parameter vector y obtained in this manner. In this example, a hidden Markov model (hereinafter referred to as an HMM) λ Create The generation of the HMM model λ is based on the entire speech parameter vector sequence obtained by continuous utterance of the sentence.yOutput probability P (y, Q | λ) is maximized. Where q is the entire speech parameter vector sequenceyRepresents a state sequence for. In this example, the structure of the HMM model λ is a left-to-right model with no skip and a three-state one-gaussian mixture of two phonemes. For example, as shown in FIG.₁, Q₂  , Q₃  The output probabilities of the articulatory parameter vector and the speech parameter vector in each state are each set to one Gaussian distribution, and the state process is a transition (loop) from the same state to the same state, and q₁  To q₂  Or q₂  To q₃  There are only a total of five transitions to. For each phoneme, a model is created for each of the following different phonemes.
[0013]
Articulation parameter vector sequencexIn the model including, each state constituting the state sequence q is represented by q_j  Then state q_j  The output probability of the voice parameter vector y of_j  Transition probability P to_t  = P (q_j| Λ) and its state q_jOutput probability P of articulation parameter vector x for_x  = P (x | q_j  , Λ) and its state q_jOutput probability P of speech parameter vector y for articulation parameter vector x for_y= P (y | x, q_j, Λ). Therefore, P (y, q_j| Λ) = ∫P (y | x, q_j, Λ) P (x | q_j, Λ) P (q_j| Λ) Each model may be created such that dx is maximized. The output probability P (y | x, q) of the speech parameter for the given articulation parameter vector_j, Λ) and the output probability P (x | q) of the articulatory parameter vector._j, Λ) both assume a Gaussian distribution.
[0014]
FIG. 2 shows an example of a model creation processing procedure. This learning method is called “Viterbi learning method”. First, the input speech parameter vector sequenceyAnd input articulation parameter vector sequencexAnd the utterance sentence, the two parameter vector pairs of the same phoneme are collected, and for each phoneme, for each of the plurality of parameter vector pairs, the three state q₁, Q₂, Q₃As the same time length, and model parameters A, b, x for each state_m, Σ_x, W_m, Σ_wIs calculated, that is, an initial model λ is created and stored (S1).
[0015]
That is, as a function y = f (x) for determining the speech parameter vector y from the articulation parameter vector x, y = Ax + b, the speech parameter vector y ′ = Ax + b calculated using the articulation parameter vector x, and the articulation parameter vector A and b that minimize the square error between the speech parameter vector y forming a pair with x are obtained, and the error w of y ′ with respect to y is obtained. The average w of the errors w is obtained._mAnd the covariance σ of the error w_wAnd calculate the average x of the articulation parameter vector x_mAnd calculate the covariance σ of the articulation parameter vector x_xAnd the state transition probability γ is calculated. The initial state transition probability γ is set to an appropriate value such as 0.8 for the self transition probability and 0.2 for the probability of transition from one state to another state._jFor all frames corresponding to that state, the value obtained by dividing the number of frames transitioning to the same state by the total number of frames corresponding to that state is defined as the self-transition probability. The transition probability is calculated as (1-self transition probability).
[0016]
These model parameters A, b, w_m, Σ_w, X_m, Σ_m, P_tIs calculated for each state of each phoneme and stored for each phoneme. In this example, the vector y on the left side has 48 elements, the vector x on the right side has 48 elements, the coefficient A is a 48 × 48 matrix, and the constant b has 48 elements in this example. Is the vector of
Therefore, at least 48 pairs of y and x are required to determine A and b.
Next, for this initial model λ, the output probability of the input speech parameter vector y
P (y, q | λ) = ∫P (y | x, q, λ) P (x | q, λ) P (q | λ) dx (1)
To the speech parameter vector y and the articulation parameter vector x such that_jIs determined using the Viterbi algorithm (S2). That is, the output probability P (y | x, q, λ) of the speech parameter vector y with respect to the articulation parameter vector x in each state and the output probability P (x | q , Λ) are obtained by the following equations (2) and (3) based on the fact that the probabilities are Gaussian-distributed with reference to the previously stored model.
[0017]
P (y | x, q, λ) = [1 / ((2π))^{N / 2}  ) | Σ_w|^1/2  ] × exp [-(1/2) (y-Ax-bw)_m)^Tσ_w ^-1(Y-Ax-bw_m)] (2)
P (x | q, λ) = [1 / ((2π)^{M / 2}  ) | Σ_x|^1/2  )] × exp [-(1/2) (xx_m)^Tσ_x ^-1(Xx_m)] (3)
N is the order of the vector y, M is the order of the vector x, and both are 42 in the above example.^TRepresents the transpose of a matrix.
[0018]
Further, a transition probability P (q | λ) is obtained, and a product of P (y | x, q, λ), P (x | q, λ) and P (q | λ) is used as a branch metric to obtain each state. The largest branch metric is determined to be the surviving path, and the branch metric is sequentially added to the previous path metric. The maximum state sequence q of the finally obtained path metric maximizes the equation (1).
Next, it is checked whether or not the maximum value of the output probability of the input speech parameter vector y obtained at the time of determining the state sequence q, that is, the value of the maximum path metric has converged (S3). State sequence q and input speech parameter vector sequenceyAnd input articulation parameter vector sequencexAnd
A change point from the model to the model in the state sequence q is detected, and the input speech parameter vector sequence of the phoneme section is detected.yAnd input articulation parameter vector sequencexAgainst
Is reset (S4).
[0019]
For the set of the speech parameter vector and the articulation parameter vector for each of the reset phonemes, each model parameter A, b, w_m, Σ_w, X_m, Σ_x, P_t, That is, a phoneme model is created, the stored corresponding model parameters are updated and stored, and the process returns to step S2 (S5).
By repeating steps S2 to S5 below, the maximum value of the output probabilities of the obtained speech parameter vectors becomes substantially constant, that is, the convergence is detected in step S3, and the process ends.
[0020]
Each model of the HMM obtained in this manner is, for example, as shown in FIG. The state transition probabilities γ (which consist of a total of five probabilities for each loop and the adjacent one as described above) are stored in the transition probability storage unit 27 for each of the storage units 25-1 to 25-J of. A and b for each state are stored in the coefficient storage unit 28, and x_m, W_mIs stored in the average storage unit 29._m, Σ_wIs stored in the covariance storage unit 31. The coefficients A and b are referred to as articulation / acoustic mapping coefficients because they are parameters for associating an approximate value of the speech parameter vector y corresponding to the articulation parameter vector X. Other parameters x_m, W_m, Σ_m, Σ_wIs called the articulatory model. Further, since P (y | x, q, λ) is calculated by equation (2) and P (x | q, λ) is calculated by equation (3), the output of the speech parameter vector y with respect to the articulation parameter vector x It can be said that the probability and the output probability of the articulation parameter vector x are also stored in the model storage unit 25. Although the “Viterbi learning method” has been described as a model creation method, a more accurate learning method “EM learning method” (Expectation-Maximization) may be used.
[0021]
First embodiment
In the first embodiment of the present invention, a speech parameter (spectrum) vector generated by an existing speech generation model is adapted so that an output probability of an input speech parameter (spectrum) vector sequence of an input speaker is maximized, Using the coefficients relating the generated speech parameter vector and the input speech parameter vector sequence, the articulatory / acoustic mapping coefficients in the speech production model are adapted.
Hereinafter, the first embodiment will be described with reference to FIGS. The input voice signal of the speaker is input as a digital signal from the input terminal 11 and is temporarily stored in the signal storage unit 42 (S1). In the speaker input speech signal, an input speech parameter (spectrum) vector y is generated for each frame in a speech parameter vector generation unit 43, and an input speech parameter vector sequenceYIs generated (S2). For example, an input audio signal is subjected to spectrum analysis for each frame, audio parameters are detected (S2-1), and velocity and acceleration parameters as temporal changes in the spectrum are detected (S2-2). Parameter vector, and the time series of the voice parameter vector of each frame is the voice parameter vector sequence.YIt is said. These parameters are the same as the speech parameters used when creating the speech generation model stored in the model storage unit 48 to be adapted, and in the above example, the 16th-order mel-cepstral coefficients excluding the 0th-order term and The speed parameter is detected. This input speech parameter vector sequenceYIs temporarily stored in the storage unit 42 (S3). Note that a phoneme string of a sentence corresponding to the speaker input voice signal is stored in the phoneme string storage unit 45.
[0022]
Average vector y of speech parameter vectors generated from the speech generation model_mjFor the input speech parameter vector sequenceY= (Y₁, ..., Y₂) Output probability P (Y| Q, λ) is adapted to maximize the average vector of the speech generation model. Output probability P (Y| Q, λ) to maximize the average vector y_mjReferring to page 174 of Non-patent Document 2, log likelihood log P (Y| Q, λ) may be determined so as to maximize it. Therefore
log P (Y| Q, λ) = K- (1/2) Σ_tΣ_jγ_t(J) (Y_t-H_sy_mj)^Tσ_yj ^-1(Y_t-H_sy_mj)
H to maximize_sTo
Σ_tΣ_jγ_t(J) σ_yj ^-1Y_ty_mj ^T= Σ_tΣ_jγ_t(J) σ_yj ^-1H_sy_mjy_mj ^T(4)
Can be obtained by calculating Here, t represents the discrete time of the vector sequence, j represents the state number of each phoneme, K is a constant, H_sIs the regression coefficient, γ_t(J) is the probability that the speech parameter vector exists in state j at time t, and γ_t(J) = P (q_t= J | y, λ) and the speech parameter vector y_jMean vector of y_mj= A_jx_mj+ B_jYields the vector y_jIs the covariance matrix of σ_yj= A_jσ_xjA_j ^T+ Σ_wAsk by ()^TRepresents a transposed matrix, and s represents a cluster dividing the acoustic space. That is, the whole phoneme model is divided into several clusters, for example, all the speech generation models λ are made into one cluster, or vowels and consonants are found as separate clusters.
[0023]
That is, as shown in FIG. 4 and FIG.YAnd voice
Using the generated model, the voice-related coefficient calculating unit 46 uses the voice-related coefficient H_sIs calculated (S4). Articulatory / acoustic mapping coefficient A for each model in model storage unit 44_j, B_jAnd the articulatory average vector x_mjAnd the average vector y of speech parameters_mjIn the average vector generation unit 47_mj= A_jx_mj+ B_j(S4-1). The articulation parameter vector x of each model in the model storage unit 44_jCovariance σ of_xjAnd the articulatory average vector x_mjError w_jCovariance σ of_wjAnd the voice parameter vector y_jCovariance matrix σ_yjIn the covariance calculation unit 48_yj= A_jσ_xjA_j ^T+ Σ_wj(S4-2). Further, in the speech relation coefficient calculating section 46, the transition probability γ of the corresponding phoneme model λ in the model storage section 44 for each phoneme according to the phoneme sequence in the storage section 45._jFrom the input speech parameter vector sequenceYAnd the average vector y from the average vector generation unit 47_mjAnd the covariance matrix σ from the covariance calculation unit 48_yjAnd with Σ_tΣ_jγ_t(J) σ_yj ^-1Y_ty_mj ^T, Σ_tΣ_jγ_t(J) σ_yj ^-1y_mjy_mj ^TIs calculated, and a regression coefficient (speech-related coefficient) H that satisfies equation (4)_sIs obtained (S4-3).
[0024]
This voice relation coefficient H_s, The articulatory / acoustic mapping coefficient A of the speech generation model_j, B_jAnd H_sA_j, H_sb_jAnd the coefficient adaptation unit 49 adapts to the input speaker's voice (S5).
Further, if necessary, the average vector y of the speech parameter vectors obtained earlier_mjBy the voice adaptation unit 51_mj= H_sy_mj(S6).
If this adapted speech generation model is used, it is possible to input an articulatory parameter vector sequence and synthesize a speech signal corresponding to the input, which is close to the speech of the input speaker.
In FIG. 5, the control unit 52 sequentially operates each unit, and performs reading and writing for each storage unit.
[0025]
Second embodiment
Next, a second embodiment of the present invention will be described with reference to FIGS. In the second embodiment, the articulatory motion of the input speech is determined using the adaptation target speech generation model, and the speech generation model is adapted so that the output probability of this articulation motion is maximized.
Input speech parameter (spectrum) vector sequence from the input speech signal of the input speaker from the input terminal 41YIs generated by the voice parameter vector generation unit 43 (S2), and is temporarily stored in the storage unit 43 (S3), as in the first embodiment.
[0026]
In the second embodiment, the input speech parameter vector seriesYOutput probability P (YThe state sequence q that maximizes | q, λ) is generated by the state sequence generation unit 61 by, for example, the Viterbi algorithm based on the phoneme sequence in the storage unit 45 (S4). This generation method may be performed in substantially the same manner as the model creation method described above.
Next, the articulatory motion that maximizes the posterior probability P (x | y, q, λ) for this state sequence q, that is, an articulatory parameter vector sequence, is generated by the articulatory parameter vector generator 62 (S5). Articulatory parameter vector sequence that maximizes P (x | y, q, λ)xIs a sequence that minimizes the following equation (5), as is clear from the description in the left column of page 2306 of Non-Patent Document 1.x _eShould be obtained.
[0027]
J = (Y-Ax _e-B)^Tσ_w ^-1(Y-Ax _e-B) (5)
That is, it is obtained by Expression (4) (the following expression) in Non-Patent Document 1.
x _e= (Σ_x ^-1+ A^Tσ_w ^-1A)^-1(Σ_x ^-1x_m+ A^Tσ_w ^-1(Y-b))
The articulation parameter vector sequence generated in this wayx _eAnd the output probability P (x _e| Q, λ) is the maximum articulation parameter vector x_eMean vector x of_emIs calculated by the average relation coefficient calculation unit 63 to obtain the average relation coefficient C_s(S6).
[0028]
Σ_tΣ_jγ_t(J) σ_xj ^-1x_etx_mj ^T= Σ_tΣ_jγ_t(J) σ_xj ^-1C_sx_mjx_mj ^T(6)
In other words, the generated articulation parameter vector sequencex _eEach vector x_et, The transition probability γ of the corresponding phoneme model λ of the speech generation model in the storage unit 44_j, Covariance σ_xj, Average x_mjTake out and Σ_tΣ_jγ_t(J) σ_xj ^-1x_etx_mj, Σ_tΣ_jγ_t(J) σ_xj ^-1x_mjx_mj ^TIs calculated, and equation (6) is calculated to obtain the regression coefficient C_sAsk for. The average relation coefficient C thus obtained_s, The articulation average vector x in the storage unit 44_mjIn the average adaptation unit 64_mj= C_sx_mj(S7).
[0029]
By using this adapted speech generation model to determine the articulatory motion (articulatory parameter vector sequence) of the speech signal of the input speaker, it is possible to obtain the articulatory motion with higher accuracy than when using the model before adaptation. it can.
Further, articulation / acoustic mapping coefficient A_j, B_jWhen adapting also,
Articulatory parameter vector sequence generated in step S15x _eEach articulatory spectrum vector x_etAnd a speech spectrum vector corresponding to the speech generation model while referring to the phoneme sequence._j, B_j, The speech spectrum vector y in the speech vector generation unit 65_j= A_jx_et+ B_jIs generated (S8).
[0030]
For a speech spectrum vector y using this speech generation model, an input speech parameter vector sequenceYOutput probability P (Y| Q, λ) is maximized, as in the case of the first embodiment.
P (Y| Q, λ) = ∫P (Y | x, q, λ) P (x | q, λ) dx log likelihood log P (Y| Q, λ) is maximized. Therefore, similarly to the derivation of equation (4), a regression coefficient (mapping relation coefficient) D satisfying the following equation (7):_sIs calculated by the mapping relation coefficient calculation unit 66 (S9).
[0031]
Σ_tΣ_jγ_t(J) σ_wj ^-1Y_t(A_jx_et+ B_j)^T= Σ_tΣ_jγ_t(J) σ_wj ^-1D_s(A_jx_et+ B) (A_jx_et+ B_j)^T            (7)
That is, in the mapping relation coefficient calculation unit 66, the input speech parameter vector sequenceYAnd the speech vector y from the speech vector generation unit 65_jAnd the error covariance σ in the model storage unit 44_wj, Transition probability γ_jAnd using
Σ_tΣ_jγ_t(J) σ_wj ^-1Y_t  y_j ^T= Σ_tΣ_jγ_t(J) σ_wj ^-1D_sy_jy_j ^T
Fill D_sIs calculated.
[0032]
This mapping relation coefficient D_s, Each articulatory / acoustic mapping coefficient A of each sound generation model in the model storage unit 44._j, B_jIn the coefficient adaptation unit 67_sA_j, D_sb_j(S10).
The generation of the state series in step S4 is performed by the adaptive speech average vector Y generated by the speech adaptation unit 51 in the first embodiment._mj(= H_sy_mj) May be used. In this case, the regression coefficient C_sAnd D_sWhen a common cluster is used, the likelihood is the same as that of the first embodiment due to the redundancy of the regression coefficient, but the adapted speech generation model is different from that of the first embodiment.
[0033]
The speech generation model is adapted to the speaker's speech by the various adaptation methods as described above, and the articulatory motion of the speaker's speech signal using the adapted speech production model is shown in, for example, Non-Patent Document 1. presume.
Experiment
Using speech signals and articulatory data of 356 sentences uttered by three Japanese men, parameter vectors are generated under the conditions described in the section on model creation, models are created for each of the three persons, and from each input speaker, Adaptation was performed for each model of two speakers other than the input speaker, and the evaluation was performed on the average of a total of six tests. The articulation data used this time was observed using a magnetic sensor system that adheres a small receiving coil to the articulation observation point. However, since the position where the receiving coil is bonded differs for each speaker, and the size of the articulatory organ differs for each speaker, it is estimated using the articulatory movement observed by the input speaker and another specific speaker model. Articulatory movements cannot be compared as they are. Therefore, normalization was performed using a linear transformation of the position and size of the observation articulation movement of the input speaker and the observation articulation movement of another speaker, which were obtained in advance, and the evaluation was performed. At the time of adaptation, supervised learning was used, and the number of adapted sentences was 40. In the case of articulatory motion reverse estimation, a condition with phonemes was used (Non-Patent Document 1). Experiments were performed on three adaptation methods: (A) the first embodiment, (B) the second embodiment, and (C) a combination of the first and second embodiments. The number of clusters s was common in all adaptation methods.
[0034]
FIG. 8 shows the log likelihood of the acoustic parameter vector for the learning data when the values of the number of clusters are 1, 3, 5, and 10. It can be seen that the use of the adaptation method increases the likelihood as compared with the speaker independent model. The likelihoods of the (A) method and the (C) method are almost the same, and the likelihood of the (B) method is lower than those.
Note that a model created using speaker's voice and articulatory movement (speaker model) and a model created using speech unrelated to the speaker and its corresponding articulatory movement (speaker independent model) were used. Experimental results for the case used are also shown.
[0035]
FIG. 9 shows the square error of the articulation motion by the adaptation method with respect to the number of clusters. By using the adaptation method, the error is smaller than that of the speaker independent model at all cluster numbers. In the case where the method (A) is used, the adaptation relating to the articulatory movement is not performed, so that the error is substantially constant regardless of the number of clusters. On the other hand, in the case of the method (B), the error decreases as the number of clusters increases. The (C) method has a smaller error than the (B) method when the number of clusters is up to 5, but the error is large when the number of clusters is 10, despite the high likelihood.
[0036]
FIG. 10 shows the square error for each phoneme. The evaluation was performed using 10 clusters. 'Total' is the square error of the entire utterance, and from 'Vowel' to 'Nasal' are the square errors of articulatory organs important in the utterance. By using the adaptation method, all phoneme classes showed improvement over the speaker independent model. Up to about 44.4% improvement was seen. In addition, there was no difference in error between phoneme classes by the adaptation method.
An articulatory trajectory (thick line) and an observed articulatory trajectory estimated by using a speaker-independent model from a speech signal in which the input male speaker uttered a sentence saying "I am doing what I need to do and there is no fault" An example of a vertical signal (thin line) is shown in FIG. 11, and the vertical signal of the articulatory orbit (thick line) estimated using the model adapted by the method (C) and the observed vertical signal of the articulatory orbit (thin line) is used. An example is shown in FIG. By comparing these figures, the thick line in FIG. 12 is closer to the thin line, and it can be understood that the effect of model adaptation is obtained.
[0037]
The estimated articulatory movements well reproduce the characteristics of the uttered phonemes. The spectral distortion between the speech spectrum generated from the estimated articulatory movement and the input speech spectrum was also improved by about 69.0%.
The adaptation device shown in FIGS. 5 and 7 may be caused to function by a computer. In this case, a program for causing the computer to execute each procedure of the adaptation method shown in FIG. 4 or FIG. 7 is installed in the computer from a recording medium such as a CD-ROM or a magnetic disk, or downloaded via a communication line. The program may be executed by a computer. In the above description, a linear function is used as a function for approximating the speech parameter vector using the articulation parameter vector as a variable, but another function may be used. The acceleration component and the velocity component need not be used as the voice parameter vector and the articulation parameter vector.
[0038]
【The invention's effect】
According to the present invention, it is possible to speaker-adapt a speech generation model including a probabilistic dynamic model representing articulatory movement and an articulation / acoustic mapping coefficient that associates an articulation parameter vector and a speech parameter vector. By using the adapted speech generation model, the articulatory motion of the input speaker's speech can be estimated more accurately than when using a model that does not adapt. Further, when speech synthesis is performed from the articulation parameter vector, speech of a desired speaker can be synthesized. Similarly, in the case of speech recognition using this model, high recognition accuracy can be obtained. Further, according to the present invention, a speech generation model can be adapted even from a small amount of speech data.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of a state transition of one phoneme model.
FIG. 2 is a flowchart illustrating an example of a model creation procedure.
FIG. 3 is a diagram showing an example of storage contents of a storage device in which a speech generation model is stored.
FIG. 4 is a flowchart showing an example of a processing procedure according to the first embodiment of the present invention.
FIG. 5 is a block diagram showing a functional configuration example of the first embodiment of the present invention.
FIG. 6 is a flowchart illustrating an example of a processing procedure according to a second embodiment of the present invention.
FIG. 7 is a block diagram showing a functional configuration example according to a second embodiment of the present invention;
FIG. 8 is a graph showing experimental results of log likelihood of a synthesized speech parameter vector with respect to a learning speech parameter vector.
FIG. 9 is a graph showing an experimental result of a square error of an articulation parameter vector estimated for an input audio signal.
FIG. 10 is a diagram showing experimental results of square errors of various types of phonemes of an articulation parameter vector estimated for an input speech signal.
FIG. 11 is a diagram showing an example of an actually measured articulation motion and an articulation motion estimated using a speaker independent model.
FIG. 12 is a diagram showing an example of an actually measured articulation motion and an articulation motion estimated using a speaker-adapted model.

Claims (10)

A speech generation model including a probabilistic dynamic model (hereinafter, referred to as an articulation model) representing articulation movement stored in a storage device and an articulation / acoustic mapping coefficient for associating an articulation parameter vector with a speech parameter vector is input. A method of adapting to a speaker voice signal,
Analyzing the input speaker voice signal for each frame to generate an input voice parameter vector sequence;
A procedure for generating each of the articulatory parameter vector sequences and the respective average vectors of the speech parameter vectors using the respective articulatory / acoustic mapping coefficients for each of the above sound generation models,
A step of relating the average vector sequence of the input voice parameter vector sequence (hereinafter referred to as the input voice average vector) and the sequence of the average vector so as to maximize the output probability of the input voice parameter vector sequence;
Applying the relation to each of the articulatory / acoustic mapping coefficients of each of the above-mentioned speech generation models, and adapting the above-mentioned speech generation model. A method of adapting an articulatory model stored in a storage device and a speech generation model including articulatory / acoustic mapping coefficients from an input speaker audio signal,
Analyzing the input speaker voice signal for each frame to generate an input voice parameter vector sequence;
A procedure for generating an articulation parameter vector sequence having a maximum posterior probability for the input speech parameter vector sequence using the speech generation model;
A procedure for relating the average vector sequence of the articulation parameter vector sequence and the average vector sequence of the speech generation model such that the output probability of the articulation parameter vector sequence is maximized,
Applying the association to the average vector of the speech generation model and adapting the average vector. A procedure for generating a speech parameter vector sequence corresponding to the articulation parameter vector sequence using each articulation / acoustic mapping coefficient of each speech generation model,
A step of relating the generated voice parameter vector sequence and the input voice parameter vector sequence to maximize the output probability of the input voice parameter vector sequence;
3. A method according to claim 2, further comprising the step of giving the association to the articulation / acoustic mapping coefficients of the speech generation model and adapting them. The procedure for generating the articulation parameter vector sequence includes:
For each of the speech generation models, using each of the articulation and acoustic mapping coefficients, a step of generating each average vector of the speech parameter vector corresponding to the articulation parameter vector sequence,
Adapting the average vector to maximize the output probability of the input speech parameter vector sequence,
Generating a sequence of the average vector that maximizes the output probability of the input speech parameter vector;
Generating the articulation parameter vector sequence that maximizes the posterior probability for the input speech parameter vector sequence with respect to the average vector sequence. Method. A speech generation model including a probabilistic dynamic model (hereinafter, referred to as an articulation model) representing articulation movement stored in a storage device and an articulation / acoustic mapping coefficient for associating an articulation parameter vector with a speech parameter vector is input. A device adapted to a speaker voice signal,
A speech parameter vector generation unit that performs spectrum analysis of the input speaker speech signal for each frame and generates an input speech parameter vector sequence;
An average vector generation unit that inputs each articulatory / acoustic mapping coefficient of each of the audio generation models and generates an average vector of the audio parameter vector,
A speech relation that inputs an average vector sequence of the input speech parameter vector sequence (hereinafter referred to as an input speech average vector) and the respective average vectors and relates them so as to maximize the output probability of the input speech parameter vector sequence. A voice-related coefficient calculator for calculating a coefficient,
An audio adaptation unit for multiplying each of the audio relation coefficients and each articulation / acoustic mapping coefficient of each audio production model and outputting an adapted articulation / acoustic mapping coefficient. Person adaptation device. An articulation model stored in a storage device, a device for adapting a speech generation model including articulation and acoustic mapping coefficients from an input speaker speech signal,
A speech parameter vector generation unit that performs spectrum analysis of the input speaker speech signal for each frame and generates an input speech parameter vector sequence;
The speech generation model, the input speech parameter vector sequence, input, the posterior probability for the input speech parameter vector sequence, the articulation parameter vector generation unit that generates an articulation parameter vector sequence that maximizes,
An average relation coefficient calculation unit for inputting the articulation parameter vector series and each articulation average vector of each sound generation vector and calculating an average relation coefficient for relating the articulation parameter vector series so that the output probability of the articulation parameter vector series is maximized; When,
A voice generation model speaker comprising: an average vector adaptation unit configured to generate an adapted articulation average vector by multiplying each of the average relation coefficients and each of the articulation average vectors of each of the speech generation models. Adaptation device. A speech vector generation unit that receives each of the articulation / acoustic mapping coefficients of the speech generation models and the articulation parameter vector sequence and generates a corresponding speech parameter vector sequence;
A mapping relation coefficient calculation for inputting the generated speech parameter vector series and the input speech parameter vector series, and calculating a mapping relation coefficient for relating the input speech parameter vector series to maximize the output probability thereof. Department and
7. A mapping adaptation unit for multiplying each of the mapping relation coefficients and each of the articulation and acoustic mapping coefficients of each of the speech generation models to generate an adapted articulation and acoustic mapping coefficient. The described speech generation model speaker adaptation device. The articulatory parameter vector generator,
A speech average vector generation unit that inputs each articulatory / acoustic mapping coefficient and each articulation average vector of each of the above speech generation models, and generates each average vector of the speech parameter vector,
A voice-related coefficient calculating unit that receives the input voice parameter vector sequence and each of the voice average vectors, and calculates a voice-related coefficient that relates these to maximize the output probability of the input voice parameter vector sequence;
A voice adaptation unit that generates an adapted voice average vector by multiplying the voice relation coefficient and each of the voice average vectors,
A speech average vector sequence generation unit that generates the sequence of the adapted speech average vector that maximizes the output probability of the input speech parameter vector sequence,
8. The parameter sequence generator according to claim 6, further comprising: a parameter sequence generator configured to generate the articulation parameter vector sequence having a maximum posterior probability with respect to the input audio parameter vector sequence with respect to the adaptive audio average vector sequence. 9. Speech generation model speaker adaptation device. A program for causing a computer to execute each procedure of the speech generation model speaker adaptation method according to claim 1. A computer-readable recording medium recording the program according to claim 9.

Images