JPH0981179A - Speaker adaptive device and voice recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the precision in estimating movement vectors and to enhance the voice recognition rate by selecting prescribed higher-order plural vectors in which the value of the distance between an object vector that it to be processed and a vicinity vector is small. SOLUTION: A speaker adaptive control section 31 adaptively learns an initial speaker model 30 which includes speaker's cluster models using speaker adaptive learning data 32, which are the sentence uttering text data, for example, converts the model into an unspecified speaker phoneme model of the phoneme HMM, stores the model in the memory of a hidden Markov network (an HM network) and performs voice recognition based on the network 11. Specifically, the section 31 successively executes the computational process of the movement vectors, the interpolation process of the movement vectors, the smoothing process of the movement vectors and the learning process, in which speaker-adapting is conducted, employing the processed movement vectors. During the interpolation and flattering/smoothing processes of the movement vectors, the selection of vicinity vectors is conducted using the tree construction of the state dividing process by a known sequential state dividing method.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speaker adaptation device for preparing a hidden Markov model (hereinafter referred to as HMM) by performing speaker adaptation on an initial speaker model using learning data for speaker adaptation. , And a voice recognition device for recognizing a voice using the HMM.

[0002]

2. Description of the Related Art Conventionally, in the case of performing speaker adaptation with a small amount of learning material for a speech recognition apparatus using an HMM, it is essential to compensate for the lack of information in order to obtain a stable adaptation effect. For this reason, various speaker adaptation methods that use information included in an initial speaker model such as an unspecified speaker model as prior knowledge have been studied (for example, in conventional document 1, “Kemi Okura et al.,“ Mixed continuous distribution ”). Moving vector field smoothing speaker adaptation method using HMM ", Proceedings of the Acoustical Society of Japan, 2-Q
-17, pp. 191-192, March 1992 ”. ).

For example, a conventional moving vector field smoothing speaker adaptation method (hereinafter referred to as VFS) disclosed in the prior art reference 1.
It is called a method. ), The difference between the estimated value using the speaker adaptation learning data and the initial value for each model parameter is defined as a movement vector. Then, the estimation error is reduced by smoothing each movement vector using the information of the movement vector that is acoustically nearby, and the unlearned model parameters are interpolated by the lack of corresponding learning data. Person adaptation.

[0004]

The VFS method of the conventional example is a method of reducing parameter estimation error caused by a small amount of learning data for speaker adaptation by parameter interpolation and smoothing. In the VFS method of the example, the neighboring vector used when performing this interpolation and smoothing is one having a short Euclidean distance among the vectors learned by the speaker adaptation learning data, and the concept of the phoneme environment is included. It is not. On the basis of the distance, the interpolation process and the smoothing process may be performed by the vectors of different phoneme environments, so that it is not reflected that the movement vectors have unique properties depending on the phoneme environment. However, there is a problem that the estimation accuracy of the movement vector deteriorates.

An object of the present invention is to solve the above-mentioned problems, improve the estimation accuracy of the moving vector as compared with the conventional example, and improve the speech recognition rate, and a speaker adaptation device, and It is to provide a voice recognition device for recognizing a voice using the HMM.

[0006]

A speaker adaptation apparatus according to a first aspect of the present invention is adapted for speaker adaptation by using a movement vector indicating a relationship between feature vectors of hidden Markov models before and after speaker adaptation. A speaker adaptation apparatus for calculating a speaker model of a hidden Markov model for speech recognition by learning by adapting an initial speaker model based on learning data, the learning data for speaker adaptation Exists and the first feature vector of the Hidden Markov Model after the speaker adaptation based on the speaker adaptation learning data is set to the first feature vector and a speaker adaptation in the vicinity thereof after the speaker adaptation. Smoothing means for performing a smoothing process using a plurality of second feature vectors of the hidden Markov model, and speaker adaptation not calculated by the smoothing means because the learning data for speaker adaptation does not exist. The average vector of the Gaussian distribution of the hidden Markov model of the above is smoothed by the learning data for speaker adaptation existing near the average vector of the Gaussian distribution of the hidden Markov model before speaker adaptation corresponding to the average vector. Interpolating means using the moving vector of the mean vector of the Gaussian distribution of the Hidden Markov Model after speaker adaptation calculated by the means, the smoothing means and the interpolating means, Selecting means for selecting a plurality of predetermined high-order vectors having a small distance value between a target vector to be processed and a neighboring vector among vectors in a lower layer from a node in the tree structure using the tree structure of the process It is characterized by having.

Further, the speaker adaptation apparatus according to claim 2,
2. The speaker adaptation device according to claim 1, wherein the selection means extracts a node in the lowest layer to which the state to which the target vector belongs corresponds, and from the extracted node in the lowest layer,
The tree structure is traced back until the number of speaker adaptive learned vectors in a state below a certain node in a layer higher than the node of the lowest layer is equal to or more than the predetermined number, and the certain node is set as the top node, In the vector in the state below the top node, selecting a predetermined plurality of vectors having a small distance value between the target vector and the neighboring vector as a selection vector for the interpolation processing and the smoothing processing. Characterize.

Further, the speaker adaptation apparatus according to claim 3 is the speaker adaptation apparatus according to claim 1 or 2, wherein the smoothing means includes the learning data for speaker adaptation and the smoothing. The mean vector of the Gaussian distribution of the hidden Markov model after speaker adaptation calculated by the smoothing means is calculated by the smoothing means with the mean vector and the speaker adaptation learning data in the vicinity thereof. Using the moving vector of the mean vector of the Gaussian distribution of the Hidden Markov Model after speaker adaptation and the constraint of the continuity of the moving vector On the other hand, smoothing is performed using a smoothing coefficient that indicates a predetermined smoothing strength so that the smoothing strength becomes smaller.

A speech recognition apparatus according to a fourth aspect of the present invention is based on the speaker adaptation apparatus according to any one of the first to third aspects and the input speech signal of the uttered voice sentence. And a voice recognition means for performing voice recognition using a hidden Markov model speaker model adapted by the speaker adaptation device and outputting a voice recognition result.

[0010]

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a voice recognition device according to an embodiment of the present invention. The speech recognition apparatus of the present embodiment adaptively learns an initial speaker model 30 including a speaker cluster model by using speaker adaptation learning data 32 that is, for example, sentence utterance text data, and unidentifies a phoneme HMM. A speaker adaptation control unit 31 for converting into a speaker phoneme model and storing it in a memory of a hidden Markov network (hereinafter referred to as HM network) 11 is provided, and speech recognition is performed based on the HM network 11. To do. In particular, as shown in FIG. 3, the speaker adaptation control unit 31 calculates the movement vector (step S1), interpolates the movement vector (step S2), and smooths the movement vector (step S3). ) And a learning process (step S4) for adapting the speaker using the processed moving vector are sequentially executed. Here, in the interpolation process and the smoothing process of the moving vector, a known sequential state division method ( It is characterized in that the neighborhood vector is selected using a tree structure of the state division process by SSS).

First, the selection of a vector in the VFS interpolation processing and smoothing processing will be described. The VFS method of the conventional example regards the problem of speaker adaptation as a re-learning problem of the HMM with a small number of learning materials (that is, learning data for speaker adaptation), and (1) moving vector calculation processing, (2) moving vector This method is performed by three steps of interpolation processing and (3) movement vector smoothing processing.
Here, the movement vector is the difference between the average vectors of the Gaussian distributions corresponding to the initial model and the adaptive model.
(2) Movement vector interpolation processing and (3) movement vector smoothing processing are performed using K neighboring vectors of the vector to be interpolated and smoothed. The movement vector interpolation process is performed on the vector p that has not been learned by the speaker adaptation learning data, and first, the distance d _{p, k} between the vector _p and the vector k learned by the speaker adaptation learning data is used as a reference. , K vectors are selected in order from the vector having the smallest distance value. Then, the movement vector is estimated by interpolation and extrapolation using these movement vectors. Similarly, the smoothing of the moving vector is performed by K vectors having a small distance d _{p, k} from the vector p to be smoothed. However,
This is simply the selection of vectors by distance in vector space, and the phoneme environment is not considered. It is considered that movement vectors have high similarity between vectors having the same phoneme environment or high similarity. Therefore, even if different phonemes are close to each other in the vector space, they are used for interpolation processing and smoothing processing in the conventional example.
In the FS method, it is considered that the error of the estimated movement vector is large. Therefore, in the present invention, as will be described in detail below, a method of selecting a neighborhood vector used for VFS interpolation processing and smoothing processing according to the similarity of the phoneme environment is used. Junichi and others,
State by known sequential state division method (SSS) disclosed in "Automatic Generation of Hidden Markov Network by Sequential State Division with respect to Phoneme Context and Time", IEICE Technical Report, SP91-88, December 1991. Vector selection is performed using the tree structure of the division process.

The sequential state division method (SSS) is an algorithm devised to generate a hidden Markov network (hereinafter referred to as an HM network) that simultaneously obtains a set of context-dependent phonemes expressed by connecting a plurality of states. Is. Starting from a model of the initial state with one state, the state with the largest distribution is repeatedly divided into the phoneme environment direction or the time direction to form a network.

The principle of the sequential state division method (SSS) will be described. The basic principle of Sequential State Division (SSS) is
For the stochastic model that expresses the temporal transition of the speech feature parameters by the stochastic transition between the stochastic stationary signal sources (states) assigned in the phoneme feature space Based on this, by repeating operations such as dividing individual states in the context direction or the time direction, the model is refined sequentially. Thereby, the unit determination of the model, the structure determination of the model, and the parameter estimation of each state can be simultaneously realized under a common evaluation criterion. A flow of processing in the sequential state division method (SSS) is shown in FIG.
The principle of the sequential state division method (SSS) will be described with reference to FIG.

First, as an initial model, a model consisting of only one state and one path connecting the state from the start end to the end is formed from all voice samples, and this state is divided. The division of the state at a certain time point is performed either in the context direction with the division of the path or in the time direction without the division of the path. Especially when splitting in the context direction,
As the paths are divided, the context class assigned to each path is also divided at the same time. As the actual division method, the one that gives the largest sum of likelihoods when applied to a voice sample is adopted from all possible division methods at that time, including the context class division method. By repeating such state division, an efficient model that can achieve high likelihood with a small number of states is generated.

State distribution by the sequential state division method (SSS)
Following the splitting process, a tree structure can be constructed as shown in Fig. 8.
You. In FIG. 8, the state S0 is changed from (a) to (b).
When the state S0 and the state S1 are divided into two, the original node N₀
Node N branched from₁From state S0 to state S1
It branches in parallel because of the relationship. Then, from (b) to (c)
Divides the state S1 into the state S1 and the state S2,
Node N₁Node N branched from₂To state S1 and state S
It is parallelly branched to 2. Furthermore, from (c)
In (d), the state S0 is divided into two states S0 and S3.
Then, node N ₁Node N branched from_ThreeTo state S0
To the state S3 in parallel with each other. Again
In addition, in (d) to (e), the state S2 is changed to the state S2.
If S2 and S4 are divided into two, node N₂A node branched from
N_FourFrom state S2 to state S4 in parallel in a parallel relationship.
It is. In the same way, from one node to two branches
The state is divided into and the tree structure is constructed.

The sequential state division method (SS
In the tree structure according to the state division process of S), the states below an arbitrary node are considered to have a high similarity in the phoneme environment, since the states at one stage were one state at a certain stage. Then, the similarity is considered to be higher in the lower layer node. The method of the present invention uses a tree structure according to the state division process of this sequential state division method (SSS), and V
By selecting the neighborhood vector of the FS method, the property of the movement vector peculiar to the phoneme environment is added to improve the estimation accuracy of the movement vector.

Next, a method of selecting a neighborhood vector used for the above interpolation processing and smoothing processing according to the present invention will be described below. The selection of the neighboring vector k for the target vector p for the VFS method interpolation processing and smoothing processing is performed in the following procedure. (1) Extract the lowest node corresponding to the state to which the target vector p belongs. (2) The tree structure is traced back from the extracted lowermost layer node until the number of speaker adaptive learned vectors in a state below a certain node in a layer higher than the lowermost layer node becomes K or more. , The node above is the top node. (3) Among the vectors in the state below the top node, the upper K vectors having a smaller value of the distance d _{p, k} between the target vector p and the neighboring vector k are selected for the interpolation processing and smoothing processing. Vector. That is, in this embodiment, the sequential state division method (S
(SS) using a tree structure formed by the state division process, a distance d between a target vector p and a neighboring vector k among vectors in a lower layer from a node in the tree structure.
It is characterized in that the upper K vectors having small values of _{p, k} are selected.

FIG. 9 shows an example of the selection range of nodes and vectors in the tree structure when selecting a neighborhood vector for a vector in state j. From the node N _{j0 in the} lowest layer to which the state j corresponds to the nodes N _j1 and N in the tree structure.
_j2 , N _j3 ,. . . As we go back, the selection ranges of the vectors at these nodes have the following groups respectively, and the neighborhood vectors are selected in these selection ranges. (A) A group G0 including the state Sj of the node N _j0 ,
(B) A group G1 including states Sj, Sk, and Sl included in the tree structure from the node N _j1 to the bottom layer, and (c) States Sj and S included in the tree structure from the node N _j2 to the bottom layer.
Group G2 including k, Sl and Sm, (d) Node N _j3
States Sj, Sk, S included in the tree structure from the bottom to the bottom
The group G3, which includes l, Sm, Sn, So. . . .

The HM network (the lowest one in FIG. 6) 11 which is automatically generated by the sequential state division method (SSS) and is connected to the phoneme collation unit 4 can be represented as a network of a plurality of states. Each state can be regarded as one stochastic stationary signal source in the voice space, and each holds the following information. (A) state number, (b) acceptable context class, (c) list of preceding and succeeding states,
(D) Parameters of the probability distribution assigned on the feature space of the speech, (e) self-transition probabilities and transition probabilities to subsequent states. In the HM network 11, when input data and its context information are given, a model for the input data is uniquely determined by linking states that can accept the context within the constraints of the preceding and succeeding state lists. be able to. This model is as shown in Figure 7,
Since a plurality of states are connected in cascade and each state is equivalent to an HMM having a self-loop, a forward path algorithm for likelihood calculation and a Baum-Welch (Baum- Wel
The algorithm of ch) can be used as it is. Here, the output probability density function is a mixed Gaussian distribution (hereinafter referred to as Gaussian distribution) having a 34-dimensional diagonal covariance matrix, and each Gaussian distribution is controlled by the speaker adaptation control using the initial speaker model 30. Learned by the unit 31.

It is generally known that when speaker adaptation is performed on a continuous distribution HMM model with a small amount of adaptation data, adaptation of the average value of the Gaussian distribution is more effective than adaptation of other parameters. (For example, refer to the conventional document 1.). In the present embodiment, only the mean value of each Gaussian distribution is adapted, the variance value, the state transition probability, and
The weighting coefficient of the Gaussian mixture distribution is not adapted.

In the present embodiment, the buffer memory 3
The HM network 11, the LR table 13, the context-free grammar database 20, the initial speaker model 30, and the speaker adaptation learning data 32 are stored in a storage device such as a hard disk memory.

A specific speaker adaptation process in the speaker adaptation control unit 31 will be described below with reference to FIGS. 2 and 3. In this speaker adaptation processing, first, an initial speaker model 30 which is an initial phoneme HMM is learned by using speaker adaptation learning data (hereinafter, referred to as learning data) 32 including sentence utterance text data. Here, the phoneme HMM is generated according to the phoneme label series corresponding to the sentence utterance text data.
Are connected to create a sentence HMM, the sentence HMM is learned by using the sentence utterance data that is the speaker adaptation learning data, and then the sentence HMM is separated again into units of the phoneme HMM. Learn.

That is, in this speaker adaptation processing, with respect to the phonemes included in the voice of the unknown speaker, the phoneme H of the standard speaker is obtained.
Re-learn the average value of MM. First, the phoneme HM of the standard speaker
Let M be the initial speaker model of the phoneme HMM of an unknown speaker. Then, the HMMs of the unknown speaker are connected so as to correspond to the phoneme sequence of the input voice of the unknown speaker, and only the average of the transition probability of the HMM, the average and variance of the appearance probabilities, and the branch probability is connected and learned. Specifically, the difference between the average vectors of the HMMs before and after the connected learning is regarded as the moving vector, and the moving vector of the unlearned HMM average vector is interpolated to move the average vector.

First, in step S1, the movement vector is calculated as follows. A set of average vectors (C ^I = c ₁ ^I , ..., C _K ^I ) of a Gaussian distribution of all phoneme HMMs of an unknown speaker in the initial speaker model, where K is the number of all Gaussian distributions. ), The k-th average vector c _k ^I (k ∈ K ₁ , K ₁ : a set of numbers of average vectors of HMMs of phonemes existing in the learning speech) and the learning data for speaker adaptation The difference vector vk of the mean vector is calculated from the corresponding c _k ^R in the set of mean vectors CR of the Gaussian distribution of the standard speaker, and this is used as the movement vector in the speaker space.

[0025]

## EQU1 ## v _k = c ^I _k −c ^R _k , _k ∈ K ₁

Here, K ₁ is a set of Gaussian distributions having learning data. This is shown in FIG. As shown in FIG. 3, for example, three Gaussian distributions exist in the acoustic space AS1 of the initial speaker model before adaptive learning, while three Gaussian distributions exist in the acoustic space AS2 of the speaker model after adaptive learning. When it exists, the average vector c _k ^R of the Gaussian distribution before adaptive learning is adaptively learned to the average vector c _k ^I of the Gaussian distribution after adaptive learning.

Then, in step S2 of FIG.
The movement vector interpolation processing is executed as follows. That is, of the set C ^I of average vectors of Gaussian distribution of all phoneme HMM of unknown speaker, average vector c _n ^{I of} Gaussian distribution belonging to unlearned HMM for phonemes for which speaker adaptation learning data did not exist. Here, n ∈ K ₂ , and K ₂ is a set of Gaussian distributions in which learning data for speaker adaptation did not exist.
Move to c _n ^I by using the movement vector v _{k 1),} the motion vector v _n obtained from the fuzzy grade function mu _{n, k} between the mean vectors c _n ^R mean vectors c _n ^k. Where the kth (k
The movement vector v _k of εK ₁ ) is the movement vector selected using the tree structure of the state division process by the sequential state division method (SSS) as described above.

[0028]

[Equation 2]

## EQU3 ## c _n ^I = c _n ^R + vn

(Equation 4)

Here, d _{n, k} represents the distance between the average vector c _n ^R and the average vector c _k ^R. The above-described movement vector calculation processing and interpolation processing will be described with reference to FIG. Figure 4 (a)
And (b) show the case where the total number of Gaussian distributions included in all HMMs is four. The case where the average vectors c ₁ ^R , c ₂ ^R , and c ₃ ^R are respectively moved to the average vectors c ₁ ^I , c ₂ ^I , and c ₃ ^I by the connection learning and the average vector c _n ^R is not learned is shown. ing. The average vector c _n ^{I in} this case is c ₁ ^R , c ₂ ^R , c ₃ ^R and movement vectors v ₁ , v ₂ , v ₃ and fuzzy class functions μ _{n, 1} , μ _{n, 2} , μ.
Calculated using _{n, 3} .

As shown in FIG. 4A, three mean vectors c ₁ ^R and c ₂ are present in the vicinity of the mean vector c _n ^R of the unlearned Gaussian distribution for which the learning data for speaker adaptation did not exist. ^R , c
₃ ^R exists. Then, as shown in (b) of FIG. 4, based on these movement vectors v _n (n = 1, 2, 3), the movement vector v _n of the average vector c _n ^R is calculated using Equation 3.
Then, the moving vector is interpolated to obtain the average vector c _n ^I of the unlearned Gaussian distribution.

The model obtained in the above steps contains an estimation error when a sufficient number of adaptive words are not obtained. It is considered that the direction of the movement vector obtained from the one including such an estimation error has a discontinuous movement. Therefore, a constraint condition of continuity is put in the movement vector for moving in the speaker space, and the directionality of the movement vector is made uniform, that is, smoothing is performed to absorb the estimation error.

Further, in the smoothing process of step S3, the difference vector v _m between the average vector c _k ^I and the m-th average vectors c _m ^I and c _m ^{R in} the vicinity thereof is obtained. Next, the fuzzy class function μ _{k, m} is used to perform the smoothing process on the difference vector v _m, and the smoothing movement vector v _k ^s is obtained using the following equation 5.

[0033]

(Equation 5)

Here, N (k) is k of the average vector c _k ^R.
-The number of the average vector in the neighborhood, α _m is a constant giving the reliability of v _m , and when k = m, μ _{k, m} = 1. Here, the average vector in the _k -neighborhood of the average vector c _k ^R means the sequential state division method (SS
It is the average vector selected using the tree structure of the state division process according to S).

[0035] Finally, in step S4, by using the movement vector v _k ^S after treatment the mean vector c _k ^R, as shown in the following equation 6, the unknown mean vector c _k ^R of the initial speaker model Adapt the speaker to the speaker. That is, by using the calculated movement vector v _k ^S , the initial speaker model 30 is learned by adapting the speaker, and the speaker model of the phoneme HMM is calculated and stored in the memory of the HM network 11.

[0036]

(6) c _S ^k = c _k ^R + v _k ^S

Here, c _S ^k is the average vector of the Gaussian distribution of the speaker-adapted phoneme HMM obtained by smoothing. In the present embodiment, α _m = 1 (mεK ₁ ), α
_{It was set to m} = 0 (mεK ₂ ). Also, fuzzy grade function _{μ k, m: (k ≠} m) , the average vector c _m is m∈K1
^R was calculated using all.

The above processing will be described with reference to FIG. FIG.
Shows the case where the total number of Gaussian distributions included in all HMMs is four. The average vectors c ₁ ^R , c ₂ ^R , c ₃ ^R , and c _k ^{R are} processed by the processing in steps S3 to S5.
Are moved to c ₁ ^I , c ₂ ^I , c ₃ ^I and c _k ^I , respectively.
Now consider the movement vector v _k corresponding to c _k ^I. Movement vector vk _{is, v 1, v 2, v} 3, v k and smoothed by the weight coefficient αm of reliability for fuzzy grade function and the motion vector corresponding to v _k ^S is calculated.

Next, SSS-LR (left-to-right rightmost) using the speaker adaptation method of the present embodiment described above.
(Type) An unspecified speaker continuous speech recognition device will be described. This device uses a phoneme environment-dependent efficient HMM representation format stored in the memory of the HM network 11. Further, in the above SSS, the likelihood is compared with the stochastic model in which the temporal transition of the speech parameter is expressed by the stochastic transition between the stochastic stationary signal sources (states) assigned in the phoneme feature space. The model refinement is performed sequentially by repeating the operation of dividing each state in the context direction or the time direction based on the maximization criterion.

In FIG. 1, the speaker adaptation control unit 31 uses an initial speaker model 30 including a speaker class model as shown in FIG. 2 by using speaker adaptation learning data 32 which is, for example, sentence utterance text data. The moving vector is calculated by the person adaptation process, and adaptive learning is performed using the calculated moving vector to convert it into an unspecified speaker phoneme model of the HMM and store it in the memory of the HM network 11. On the other hand, the uttered voice of the speaker is input to the microphone 1 and converted into a voice signal, and then input to the feature extraction unit 2. The feature extracting unit 2 performs, for example, LPC analysis after A / D conversion of the input voice signal, and performs logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 1
A 34-dimensional feature parameter including a 6th-order Δ cepstrum coefficient is extracted. The time series of the extracted characteristic parameters is input to the phoneme matching unit 4 via the buffer memory 3.

The phoneme matching unit 4 executes a phoneme matching process in response to a phoneme matching request from the phoneme context dependent LR parser 5. Then, the likelihood of the data in the phoneme matching section is calculated using the speaker model of the phoneme HMM stored in the memory of the HM network 11, and the value of this likelihood is returned to the LR parser 5 as the phoneme matching score. At this time, the forward pass algorithm is used.

On the other hand, a predetermined context-free grammar (CFG) in the context-free grammar database 20 is automatically converted, as is known, to create an LR table and stored in the memory of the LR table 13. The LR parser 5 is the LR table 1 described above.
3, the input phoneme prediction data is processed from left to right without backtracking. In the case of syntactic ambiguity, the stack is split and parsing of all candidates is processed in parallel. The LR parser 5 predicts the next phoneme from the LR table 13 and outputs the phoneme prediction data to the phoneme matching unit 4. In response to this, the phoneme collation unit 4 collates by referring to the information in the HM network 11 corresponding to the phoneme, returns the likelihood to the LR parser 5 as a speech recognition score, and sequentially connects the phonemes. By going through, recognition of continuous voice is performed. When a plurality of phonemes are predicted in the continuous speech recognition, the existence of all of them is checked, and a pruning is performed by using a beam search method to leave a partial tree having a high likelihood of partial speech recognition. To achieve high-speed processing.

In the above embodiment, the reliability weighting coefficient α _m for each movement vector is used, but the present invention is not limited to this, and the weighting coefficient α _m will be described in detail below as the weighting coefficient λ _{a, b.} It may be controlled to do so. In the present embodiment, the number of neighbors K is 6, and the weighting factor λ _{a, b} is
A Gaussian window calculated by can be used.

[0044]

[Formula 7] λ _{a, b} = exp (-d _{a, b} / fp)

Here, d _{a, b} is the distance between the average vector c _a ^I and the average vector c _b ^I , and fp is a predetermined positive number of weight control parameters, which indicates the strength of smoothing. It is a smoothing coefficient and is expressed by the following equation 8.

[0046]

[Expression 8] fp = (f · α) / (n _p + α)

Here, f is a predetermined initial value of the smoothing coefficient fp commonly given to all parameters,
n _p represents the data amount of the p-th Gaussian distribution learning data for adaptation.

That is, in the present embodiment, in controlling the smoothing coefficient fp, it is considered that the data amount of the adaptive learning data for each parameter is biased depending on the content of the adaptive learning data, and the number of states and the mixture are mixed. In order to perform control on the basis that does not depend on the structure of the model such as the number, each parameter, that is, the average value of the Gaussian distribution is independently controlled. By using the equation (8), the smoothing strength for each parameter is weakened as the adaptive learning data amount increases, and when the data amount n _p becomes infinite,
It can be seen that the state converges to the same state as when smoothing is not performed. Further, the convergence speed at this time is determined by the coefficient α, but in the present embodiment, the coefficient α uses a value obtained experimentally.

Finally, in step S4, the movement vector calculated and processed in step S2 or S3 is used to learn by speaker adaptation of the initial speaker model stored in the memory 30, whereby The speaker model of the HM network is calculated and stored in the memory of the HM network 11.

In the present embodiment, the smoothing coefficient fp represented by the equation 8 is used, but the present invention is not limited to this, and
At least, the smoothing coefficient fp indicating the smoothing strength determined in advance so that the smoothing strength becomes smaller with the increase in the data amount of the speaker adaptation learning data having the Gaussian distribution may be used. For example, the smoothing coefficients fp1 to fp4 shown in the following Expressions 9 to 12 may be used instead of the smoothing coefficient fp.

[0051]

Equation 9] _{fp1 = f {1- (n i} / α)}, when fp1 = 0, n _i ≧ α when n _i <alpha

Equation 10] _{fp2 = f {1- (n i} / α)} 2, n i < When alpha fp2 = 0, when n _i ≧ alpha of

[Mathematical formula-see original document] fp3 = f * exp (-n _i / α)

Mathematical Expression 12 fp4 = f · exp (−n _i / α) ²

In the above embodiment, the speaker adaptation control unit 31, the phoneme matching unit 4, and the LR parser 5 are constituted by, for example, a digital electronic computer. In the above embodiment, the phoneme HMM is a H represented by a network.
Although the M network 11 is used, the present invention is not limited to this, and the HM
A phoneme HMM may be used instead of the network 11.

[0053]

EXAMPLE The present inventor conducted the following experiment in order to evaluate the speech recognition apparatus of this embodiment. A 200-state HM network was used for this experiment. As an initial speaker model in an initial state before speaker adaptation, an unspecified speaker model (Kosaka et al., “Method for creating unspecified speaker model using clustering method”, Acoustical Society of Japan, 1-R-12 , 19
See November 1994. ) (A model created by synthesizing from 285 unspecified speaker models),
The number of mixtures in each state was 5. Further, the number of neighboring vectors used when performing the VFS method of the conventional example is set to 6. Table 1 shows analysis conditions, parameters used, and adaptive / recognition data.
Shown in In the experiment, the evaluation with different selection phrases was repeated three times for each number of adaptive phrases, and the average phoneme recognition rate was obtained.

[0054]

[Table 1] Experimental conditions ─────────────────────────────────── Analysis conditions Sampling frequency = 12 KHz Hamming window = 20 ms frame period = 5 ms ─────────────────────────────────── Working parameter 16th LPC cepstrum + 16th Δ cepstrum + Logarithmic power + Δ logarithmic power ─────────────────────────────────── Learning data 146 men + 139 women (50 sentences for each speaker) ─────────────────────────────────── Adaptation / recognition data ──── ──────────────────────────────── (a) Four speakers (MAU, MMY, MSH, MTM) ( ) Adaptive data n clauses randomly picked from 598 clauses (SB1, SB2, SB4 task owned by the applicant of the present patent) (c) Recognition data 279 clauses (SB3 task owned by the applicant of the present patent) ─── ────────────────────────────────

Table 2 shows the results of the phoneme recognition experiment conducted by four men. For comparison, the result of the conventional VFS method in which the neighborhood vector used for the interpolation process and the smoothing process of the moving vector is selected by the distance d _{p, k} is also shown.

[0056]

[Table 2] Speaker adaptation result-phoneme recognition error rate (%) Upper row: VFS method of conventional example Lower row: method of this embodiment ──────────────────── ─────────────── Number of adaptive phrases Speaker name Before adaptation 10 20 30 40 50 ─────────────────────── ───────────── MAU 19.1 17.4 14.8 13.4 12.7 11.9 17.3 14.2 13.3 12.7 12.1 ───────────────────────── ────────── MMY 20.8 17.5 16.1 15.3 14.5 14.3 18.6 16.3 15.1 14.4 14.0 ──────────────────────────── ─────── MSH 26.9 19.4 17.5 17.2 16.8 16.5 20.2 17.8 16.8 15.9 15.6 ──────────────────────────────── ──── MTM 18.7 14.2 12.2 10.8 10.7 10.5 15.1 12.1 10.6 9.8 10.1 ─ ───────────────────────────────── Average 21.4 17.1 15.2 14.2 13.6 13.3 17.8 15.1 13.9 13.2 13.0 ──── ───────────────────────────────

As is clear from Table 2, when the number of adaptive clauses is small, the conventional VFS method uses the tree structure of the state division process by the sequential state division method (SSS) according to the present invention. The recognition rate is higher than that of the method of the embodiment (see the case where the number of adaptive clauses is 10). However, when the number of adaptive phrases is large, the method of the present embodiment according to the present invention shows a small but high recognition rate (see when the number of adaptive phrases is 20, 30, 40, 50). .
When the number of adaptive clauses is small, the reason why the recognition rate of the method of the present embodiment according to the present invention is inferior is that since the number of adaptively learned vectors is small, the tree structure is traced back to the upper layer node. It is conceivable that a vector with a low phoneme environment similarity is selected. When the number of adaptive clauses is large, vectors are selected within the state below the node in the tree structure of the state division process by the sequential state division method (SSS), and the recognition rate is higher than that of the conventional VFS method. Is shown. Therefore, it is not necessary to perform movement vector interpolation processing and smoothing processing using vectors in states belonging to nodes under the tree structure in the lower layer of the state division process of the sequential state division method (SSS) according to the present invention. It turns out that the environment is taken into consideration and is more effective than the selection of the distance between the vectors.

In the above experiment, the smoothing coefficient of the VFS method of the conventional example is not controlled, but as shown in the modified example at the end of the above-mentioned embodiment, it depends on the similarity of the phoneme environment. It is considered that the voice recognition rate can be further improved by performing the smoothing coefficient control.

As described above, the neighborhood vector used when performing the interpolation process and the smoothing process of the conventional VFS method is selected by using the tree structure formed by the state division process of the sequential state division method (SSS). I decided to do so,
In the above-described interpolation processing and smoothing processing, the similarity of the phoneme environment is taken into account, the accuracy of estimating the motion vector can be improved, and the speech recognition rate can be improved, as compared with the conventional example.

[0060]

As described above in detail, according to the speaker adaptation apparatus of claim 1 of the present invention, the movement vector indicating the relationship between the feature vectors of the hidden Markov model before and after the speaker adaptation is used, A speaker adaptation apparatus for calculating a speaker model of a hidden Markov model for speech recognition by learning by speaker adaptation of an initial speaker model based on learning data for speaker adaptation, wherein the speaker The first feature vector of the hidden Markov model after the adaptation learning data is present and is speaker-adapted based on the speaker adaptation learning data is the first feature vector and the speaker adaptation in the vicinity thereof. Smoothing means for performing a smoothing process using the plurality of second feature vectors of the hidden Markov model after being processed, and the speaker adaptation learning data does not exist and is not calculated by the smoothing means. The mean vector of the Gaussian distribution of the hidden Markov model after speaker adaptation, the learning data for speaker adaptation that exists near the mean vector of the Gaussian distribution of the hidden Markov model before speaker adaptation that corresponds to the mean vector exists. And interpolating means for interpolating using a moving vector of the average vector of the Gaussian distribution of the Hidden Markov Model after speaker adaptation calculated by the smoothing means. By using the tree structure of the state division process by the method, a plurality of predetermined upper vectors with a small distance value between the target vector to be processed and the neighboring vector among the vectors in the lower layer from a node in the tree structure A selection means for selecting is provided. Therefore, in the above-mentioned interpolation processing and smoothing processing, the processing takes into account the similarity of the phoneme environment, the estimation accuracy of the movement vector can be improved compared to the conventional example, and the speech recognition rate can be improved. .

In the speaker adaptation device, the selecting means extracts a node in the lowest layer to which the state to which the target vector belongs corresponds, and the node in the lowest layer is extracted from the extracted nodes in the lowest layer. The tree structure is traced back until the number of speaker adaptive learned vectors in a state below a certain node in a higher layer becomes equal to or more than the predetermined plurality.
In the vector in the state below the top node, the certain node is set as the top node, and a plurality of predetermined upper vectors having a small value of the distance between the target vector and the neighboring vector are set in the interpolation process and the smoothing process. As a selection vector for As a result, in the above-mentioned interpolation processing and smoothing processing, the similarity of the phoneme environment is introduced, the estimation accuracy of the moving vector can be improved compared to the conventional example, and the speech recognition rate can be improved. it can.

Further, in the speaker adaptation device, the smoothing means calculates the Gaussian distribution of the Hidden Markov Model after speaker adaptation calculated by the smoothing means in the presence of the speaker adaptation learning data. An average vector is the average vector and a moving vector of the average vector of the Gaussian distribution of the hidden Markov model after speaker adaptation, which is calculated by the smoothing means with the speaker adaptation learning data existing in the vicinity thereof. Based on the constraint condition of the continuity of the movement vector, the smoothing strength determined in advance so that the strength of the smoothing becomes smaller as the data amount of the speaker adaptation learning data of the Gaussian distribution increases. Smoothing is performed using a smoothing coefficient indicating strength. Therefore, by effectively smoothing the estimation error by smoothing for the movement vector with a small amount of the learning data, and weakening the smoothing for the parameter with a small estimation error learned by a large amount of learning data, It is possible to prevent the performance from decreasing. As a result, it is possible to always obtain good adaptation performance for a large amount of adaptation learning data. Also, in order to control the smoothing strength individually for each movement vector, even if there is a bias in the phonemes included in the learning data for adaptation, smoothing control that considers that bias should be performed. You can Therefore, by performing voice recognition using the speaker model adapted to the speaker using the calculated and smoothed movement vector, the apparatus according to claim 1 or 2 is compared with the conventional example. By comparison, a high voice recognition rate can be obtained.

Further, according to the speech recognition apparatus of the fourth aspect of the present invention, the speaker adaptation apparatus and the speaker adaptation apparatus speaks based on the inputted voice signal of the uttered voice sentence. And a voice recognition unit that outputs voice recognition results by performing voice recognition using a speaker model of a hidden Markov model that is personally adapted. Therefore, a higher voice recognition rate can be obtained as compared with the conventional example.

[Brief description of drawings]

FIG. 1 is a block diagram of a voice recognition device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a speaker adaptation process executed by a speaker adaptation control unit 31 of FIG.

FIG. 3 is a diagram illustrating conversion of an acoustic space AS1 of an initial speaker model before adaptive learning into an acoustic space AS2 of a speaker model after adaptive learning when the speaker adaptation process of FIG. 2 is executed using a movement vector. It is a conceptual diagram which shows.

4 (a) is a conceptual diagram showing a movement vector calculation process executed in step S1 of FIG. 2, and FIG.
FIG. 3 is a conceptual diagram showing a movement vector interpolation process executed in step S2 of FIG.

FIG. 5 is a conceptual diagram showing a movement vector smoothing process executed in step S3 of FIG.

6 is a diagram showing the principle of the sequential state division method (SSS) executed by the speaker adaptive control unit 31 of FIG.

7 is a state transition diagram showing an individual model structure of an HM network used in the speech recognition apparatus of FIG.

8 is a conceptual diagram showing a tree structure of a state division process by a sequential state division method (SSS) executed by the speaker adaptive control unit 31 of FIG.

9 is a conceptual diagram showing a selection range of neighboring vectors selected in the processing in the speaker adaptive control unit 31 of FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Microphone, 2 ... Feature extraction part, 3 ... Buffer memory, 4 ... Phoneme matching part, 5 ... LR parser, 11 ... Hidden Markov network (HM network), 13 ... LR table, 20 ... Context-free grammar database, 30 ... Initial speaker model, 31 ... Speaker adaptation control unit, 32 ... Learning data for speaker adaptation, S1 ... Movement vector calculation processing, S2 ... Movement vector interpolation processing, S3 ... Movement vector smoothing processing, S4 ... The process of speaker adaptation using the processed motion vector.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Masahiro Tonomura Inoue Masahiro, Soka-cho, Kyoto Prefecture No. 5 Mihiradani, Osamu Osamu, Kyoto, Japan (72) Inventor Shoichi Matsunaga Kyoto 5 Seiraya-cho, Seiji-cho, Seika-cho, Oita Prefecture San-tani Valley, Inc.

Claims (4)

[Claims] 1. A voice is obtained by speaker-adaptive learning of an initial speaker model based on speaker adaptation learning data using a movement vector indicating a relationship between feature vectors of a hidden Markov model before and after speaker adaptation. In the speaker adaptation device for calculating the speaker model of the hidden Markov model for recognition, after the speaker adaptation learning data exists and the speaker adaptation is performed based on the speaker adaptation learning data, The first feature vector of the hidden Markov model is the first feature vector,
Smoothing means for performing a smoothing process using a plurality of second feature vectors of the Hidden Markov Model after speaker adaptation in the vicinity thereof, and the learning data for speaker adaptation does not exist, and The average vector of the Gaussian distribution of the hidden Markov model after speaker adaptation that has not been calculated by the smoothing means is in the vicinity of the average vector of the Gaussian distribution of the hidden Markov model before speaker adaptation that corresponds to the average vector. Person learning data exists and interpolation means for interpolating using the moving vector of the average vector of the Gaussian distribution of the hidden Markov model after speaker adaptation calculated by the smoothing means is provided, and the smoothing means The interpolating means uses a tree structure of a state division process by the sequential state division method to process a vector in a lower layer from a node in the tree structure. Speaker adaptation apparatus characterized by comprising a selection means for selecting the upper plurality of vectors predetermined value is less of the distance between the elephant vector and neighboring vectors. 2. The selecting means extracts a node in the lowest layer to which the state to which the target vector belongs corresponds, and a node in a layer higher than the node in the lowest layer from the extracted node in the lowest layer. The tree structure is traced back until the number of speaker adaptive learned vectors in the following states is equal to or more than the predetermined number, the certain node is set as the top node, and the vector in the state below the top node is 2. The speaker adaptation apparatus according to claim 1, wherein a plurality of predetermined upper vectors having a small distance value between the target vector and the neighborhood vector are selected as selection vectors for the interpolation processing and the smoothing processing. . 3. The smoothing means sets the average vector of the Gaussian distribution of the Hidden Markov Model after speaker adaptation calculated by the smoothing means when the learning data for speaker adaptation exists to the average vector. , The movement vector of the mean vector of the Gaussian distribution of the hidden Markov model after the speaker adaptation, which has the learning data for speaker adaptation in its vicinity, and is calculated by the smoothing means, and the continuation of the movement vector Based on the constraint of sex, a smoothing coefficient is used that indicates a predetermined smoothing strength so that the smoothing strength decreases as the data amount of the speaker adaptation learning data of the Gaussian distribution increases. 3. The speaker adaptation apparatus according to claim 1, wherein the speaker adaptation apparatus is characterized by smoothing. 4. The speaker adaptation device according to claim 1, wherein the speaker adaptation device adapts the speaker based on a voice signal of an input uttered voice sentence. A voice recognition device, comprising: a voice recognition means for performing voice recognition using a hidden Markov model speaker model and outputting a voice recognition result.