JP3035239B2 - Speaker normalization device, speaker adaptation device, and speech recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the speaker normalizing device, the speaker adaptive device and the speech recognizer while improving the estimating precision of adaptive parameters compared with the conventional ones and improving the speech recognition rate. SOLUTION: A speaker normalization control section 20 computes the transformation coefficient, which includes a transformation matrix and a constant term vector for every speaker. The matrix transforms an average vector based on a multiple regression mapping model employing a maximum likelihood linear regression method against an initial hidden Markov model(HMM) 31 based on the voice data that depend on plural speakers. The constant term vector is subtracted from the voice data, normalized voice data are computed, an initial HMM is learned and the HMM, which is speaker normalized, is obtained. A speaker adaptive control section 21 computes the transformation coefficient against a speaker normalized HMM 33 based on speaker adaptive learning data. Then, the transformation coefficient for the transformation based on the model, which is made speaker adaptive by a MAP estimating method, is computed and linear transformation processed to compute the average vector of the HMM after the speaker adaptive process. Then, an HMM 11, which is made speaker adaptive, is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speaker-normalized hidden Markov model (hereinafter, referred to as a speaker-normalized model) by performing speaker normalization on an initial speaker model using speaker-dependent speech data feature parameters. A speaker normalizing device that creates HMM), performs speaker adaptation on the speaker-normalized HMM using the speaker adaptation learning data, and performs speaker-adapted H.
The present invention relates to a speaker adaptation device that creates an MM, and a speech recognition device that recognizes speech using a speaker-normalized or speaker-adapted HMM.

[0002]

2. Description of the Related Art When considering a speech recognition application, there is a strong demand for an unspecified speaker speech recognition system that can be used without prior speaker registration. However, the current unspecified speaker speech recognition performance is lower than that of the specific speaker speech recognition, and the difference is about two to three times the error rate. In order to improve the performance of the speaker-independent speaker recognition, a speaker adaptation process (for example, using a small amount of adaptation data uttered by a particular speaker) to bring the acoustic model of the speaker-independent speaker recognition closer to the particular speaker.
Prior art document 1 “CLLeggetter et al.,“ MaximumLike
lihood Linear Regression for Speaker Adaptation of
Continuous Density Hidden Markov Models ", Computer
Speech and Language, Vol. 9, pp. 171-185, 1995. " However, a large amount of training adaptation data is required until the performance is equivalent to that of the specific speaker speech recognition.

[0003]

Generally, learning of an independent speaker-independent HMM (hereinafter, SI-HMM) independent of a speaker is performed using voice data of a plurality of speakers.
In spite of the fact that the learning data contains not only differences depending on speakers, but also differences such as situations (contexts) where units to be learned are placed, an acoustic model for specific speaker speech recognition (HMM depending on speakers) (Hereinafter referred to as SD-HMM).). Thereby, SI-H
In the MM, both the variation caused by the difference of the speaker and the variation of the phonemic context coexist, resulting in a model having a large spread. This is considered to be one of the factors of the degradation of the identification performance. In the case of a speech recognition system based on a continuous mixture distribution type HMM, the variance of the Gaussian distribution is large, and there is a problem in that recognition units are overlapped with each other, making identification difficult.

[0004] In particular, when speaker adaptation is performed using the conventional multiple regression mapping model disclosed in the prior art document 1, when the adaptation data for learning is small, estimation of adaptation parameters is performed. There is a problem that the accuracy is relatively poor and the speech recognition rate is relatively low.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above problems and to improve the estimation accuracy of adaptation parameters as compared with the prior art, and to improve the speech recognition rate. A device, a speaker adaptation device, and a speech recognition device.

[0006]

According to a first aspect of the present invention, there is provided a speaker normalizing apparatus which is training data for learning an initial model of a predetermined hidden Markov model, and which depends on a plurality of speakers. A storage device for storing a feature vector of speech data to be processed, and a multiple regression mapping for the initial model of the hidden Markov model based on the feature vector of the speech data stored in the storage device by a maximum likelihood linear regression method. A first transform coefficient for each speaker is calculated, the first transform coefficient including a transform matrix for transforming an average vector based on a model and a constant term vector representing an individual difference common to spectra.
And the characteristic of the voice data normalized by subtracting the constant term vector calculated by the first calculating means for each speaker from the feature vector of the voice data stored in the storage device. A second calculating means for calculating the vector, and an initial model of the hidden Markov model based on a feature vector of the normalized speech data calculated by the second calculating means, using a predetermined learning algorithm. And a third calculating means for calculating model parameters of the speaker-normalized hidden Markov model by learning.

According to a second aspect of the present invention, there is provided a speaker normalizing apparatus according to the first aspect, based on a feature vector of voice data of a speaker to be speaker-adapted. A second transformation coefficient including a transformation matrix and a constant term vector for transforming an average vector based on the multiple regression mapping model is calculated by the maximum likelihood linear regression method for the Hidden Markov Model calculated by the third calculation means. Multiple regression speaker-adapted by a maximum a posteriori probability estimating method based on a fourth calculating means, and a second conversion coefficient including a conversion matrix and a constant term vector calculated by the fourth calculating means. Fifth computing means for computing a third transformation coefficient including a transformation matrix for transforming an average vector based on a mapping model and a constant term vector, and a transformation matrix computed by the fifth computing means and a constant term vector For the third transform coefficient including,
By performing a predetermined linear transformation process, a sixth vector for calculating an average vector of the hidden Markov model after speaker adaptation is calculated.
And arithmetic means.

[0008] Further, the speech recognition apparatus according to claim 3 is
Using the hidden Markov model calculated by the third calculation means of the speaker normalization apparatus according to claim 1, perform voice recognition based on the voice signal of the input uttered voice sentence and output a voice recognition result. Voice recognition means.

Further, the voice recognition device according to the fourth aspect of the present invention,
Speech recognition based on a speech signal of an input uttered speech sentence, using a hidden Markov model including an average vector of the hidden Markov model calculated by the sixth calculation means of the speaker adaptation apparatus according to claim 2. And voice recognition means for outputting a voice recognition result.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a speech recognition device according to one embodiment of the present invention. This embodiment is characterized in that a speaker normalization control unit 20 and a speaker adaptation control unit 21 are provided.

In this case, the speaker normalization control section 20 has the following steps:
Speech data 32-1 depending on each of a plurality of M speakers
Based on the feature vectors of
Of the initial model (hereinafter referred to as an initial HMM) 31 by a maximum likelihood linear regression method, a first conversion coefficient A including a conversion matrix for converting an average vector based on a multiple regression mapping model and a constant term vector. _c ^(m) and b _c ^{( m)} are expressed by the following _equation ⁽ 6).
, M is calculated for each speaker m (m = 1, 2,..., M) by using Equation 11;
Speech data 32-1 to 32-depending on human speakers
The calculated constant term vector b _c ^(m) is subtracted for each speaker ^m from the 2-M feature vector o _t ^(m) to calculate a normalized speech data feature vector oh _t. , (C)
Based on the computed normalized feature vectors oh _t of the audio data, the initial model 3 of Hidden Markov Models
1 by using a predetermined learning algorithm to calculate speaker-normalized model parameters of the HMM 33. Here, the model parameters include HMM model parameters such as mean vector, variance of Gaussian distribution, and state transition probability.

The speaker adaptation control unit 21 further comprises (d) speaker adaptation learning data 3 which is speech data for speaker adaptation.
Based on the feature vector of No. 4, the speaker-normalized HMM 33 calculated by the speaker normalization device 20 is:
Using the maximum likelihood linear regression method, a second conversion coefficient A _c , b _c including a constant matrix and a conversion matrix for conversion of an average vector based on the multiple regression mapping model is calculated using Equation 6 described below, (E) On the basis of the calculated transformation matrix and the second transformation coefficients A _c and b _c including the constant term vector, the maximum posterior probability estimation method is used to obtain the following equations 14 and 15,
A third matrix including a transformation matrix and a constant term vector for transformation of an average vector based on a speaker-adapted multiple regression mapping model
The coefficients of the transformed A _{c, k ^MAP,} b _c, calculates the _k ^MAP, the third conversion coefficient A _c containing (f) the calculated transformation matrix and the constant term _{vector, k ^MAP,} b _c, the _k ^MAP On the other hand, by performing a predetermined linear conversion process using Expression 13 described later, the average vector μh _k, ^MAP of the HMM after speaker adaptation is calculated.

Further, the speech recognition apparatus shown in FIG. 1 uses the above-described speaker-adapted HMM 11 to perform speech recognition based on the speech signal of the input uttered speech sentence, and outputs a speech recognition result. Further, using the speaker-normalized HMM 33, speech recognition may be performed based on the speech signal of the input uttered speech sentence, and the speech recognition result may be output.

In the embodiment according to the present invention, generation of an acoustic model by a speaker normalization method for removing speakerness has been studied. By performing speaker normalization, the spread of the model is reduced, and an improvement in discrimination performance can be expected. Also, if a model is obtained by such speaker normalization in which the variation can be considered to be mainly based on the phonemic context, it is considered to be effective as an initial model for speaker adaptation. The normalization process uses a constant term of the multiple regression mapping model. The constant term is considered to represent an individual difference common to a wide range of spectrums such as an outline of a vocal cord sound source spectrum and line characteristics. The constant term is considered as an individual difference vector, and normalization is performed by subtracting the constant term from the learning data. Further, here, the speaker adaptation using the speaker-normalized HMM trained by the speaker-normalized speech data as the initial model is performed by using the speaker adaptation by the multiple regression mapping model and the maximum posterior probability estimating method (hereinafter, referred to as “the maximum posterior probability estimation method”). , MAP estimation method).

First, a multiple regression mapping model used in the present embodiment will be described. The speaker adaptation by the multiple regression mapping model is performed by converting the average vector μ _k (the number of dimensions n) of the k-th Gaussian distribution of the initial model into the average vector μh _k based on the speaker adaptation model by the following equation. Done.

[0016]

Μh _k = A _c μ _k + b _c

Where A _c is an n × n transformation matrix,
b _c is an n-dimensional constant term vector, which is obtained for each class Ω _{c of the} shared Gaussian distribution. Here, a maximum likelihood linear regression method (Maximum likelihood linear regression) for estimating the conversion coefficients A _c and b _c on the learning adaptation data based on the maximum likelihood is used.
gression; hereinafter, referred to as the MLLR method. See, for example, Prior Art Document 1. ) Will be described. In the MLLR method, an input vector o _t in a k-th Gaussian distribution (hereinafter, referred to as a Gaussian distribution k) at time t.
The observed probability density function b _k a (o _t) Suppose as follows.

[0018]

[Number 2] _{b k (o t) = 1} / {(2π) n / 2 | Σ k | 1/2}
× exp [- (1/2) { o t - (A c μ k + b c)} 'Σ k -1 {o t
− (A _c μ _k + b _c )}]

Where Σ _k is the diagonal covariance matrix diag
[Σ ² _k1 , σ ² _k2 ,..., Σ ² _kn ]. 'Represents a transposed matrix. Further, Σ _k ^-1 represents an inverse matrix of the matrix Σ _k . The transform coefficients are obtained by maximizing the Baum auxiliary function:

[0020]

(Equation 3)

Here, O represents a sequence (o ₁ , o ₂ ,..., O _T ) of feature vectors of the adaptation data having the frame length T. Λ and λb are model parameters before and after adaptation. θ is a state sequence (θ ₁ , θ ₂ ,..., θ _T ), and Θ represents a set of all possible state sequences. F
(O, θ | λ) and F (O, θ | λb) are likelihoods before and after adaptation in the state sequence θ, respectively.

Conversion coefficients A _c , b _{c at} which the auxiliary function indicates the maximum value
Is obtained by partially differentiating the auxiliary function with A _c and b _c , and setting the partially differentiated value to zero in the shared class Ω _{c as} in the following equation.

[0023]

(Equation 4)

(Equation 5)

Here, γ _k (t) is an expected value at which an input vector is observed at time t in Gaussian distribution k. Μ _k ′ is a transposed matrix of the average vector μ _k . Therefore, from Equations 4 and 5, the element a in the p-th row of the transformation matrix A _{c
cp, i,} and p th element b _cp constant term b _c is given by the following equation.

[0025]

(Equation 6)

Here,

(Equation 7)

(Equation 8)

(Equation 9)

(Equation 10)

[Equation 11]

Here, μ _ki is the i-th element of the mean vector, σ _kp is the (p, p) element of the diagonal covariance matrix, and o _tp is the p-th element of the input vector at time t. Is represented. The above is the description of the multiple regression mapping model.

Next, the creation of an acoustic model by speaker normalization using a multiple regression mapping model will be described. Constant term b _c multiple regression mapping model is considered to represent the individual differences that are common to a wide range of the spectrum, such as general shape and line characteristics of glottal source spectrum. Therefore, in the present embodiment, the constant term b _c
Is a personal difference vector, and speaker normalization is performed. FIG.
5 and 5 are conceptual diagrams of the invented speaker normalization method. FIG. 2 is a flowchart of a speaker normalization process executed by the speaker normalization control unit 20 of FIG. 1 to create a speaker normalization model using the voice data of M speakers. FIG. 7 is a block diagram thereof. In FIG. 1, a speaker normalization control unit 20, a speaker adaptation control unit 21, a feature extraction unit 2, a phoneme collation unit 4, and an LR parser 5 are configured by an arithmetic and control unit such as a digital computer, for example.
Is, for example, a hard disk memory, and the initial HMM 31
The feature parameter vectors of the voice data of the speakers 1 to M, the speaker-normalized HMM 33, the speaker adaptation learning data 34, the speaker-adapted HMM 11, the LR table 12, and the context-free grammar 13 are, for example, a hard disk. Stored in memory. The speech data 32-1 to 32-M of each speaker is a vector of feature parameters extracted from the speech waveform signal of each speaker, that is, a feature vector. In this specification, audio data refers to a feature vector. Hereinafter, the procedure for creating the speaker normalization model will be described with reference to FIGS.

Referring to FIGS. 1, 2 and 7, first,
In step S1 of FIG. 2, an initial H which is an unspecified speaker HMM
The MM (initial model of the HMM) 31 is read out and set as the HMM to be processed. Next, in step S2, as shown in FIG. 4, the conversion coefficient A _c ^{(m of the} multiple regression mapping model for each of the speakers 1 to M using the MLLR method with respect to the processing target HMM by using the equations 6 to 11. ⁾ , B _c ^(m) , m = 1, 2,..., M. Further, in step S3, as shown in FIG. 5, the voice data o _t ^{(m) of} each of the speakers 1 to M is obtained by using Expression 12.
By subtracting the constant term vector b _c ^(m) of the multiple regression mapping model from 32-1 to 32-M, the normalized speech data o
_ht is calculated.

Equation 12] ^{_{oh t = o t (m)}} -b c (m), 1 ≦ m ≦ M

[0030] Next, the re-trained using a learning algorithm Balm Welch (Baum-Welch) for text data-normalization audio data oh _t in step S4. Then, in step S5, it is determined whether or not a predetermined number of repetitions has been reached.
In step S6, the HMM after the re-learning is set as the processing target HMM, and the process returns to step S2 again to execute the above processing. On the other hand, when the predetermined number of repetitions is reached (three in the preferred embodiment) in step S5, the HMM after the re-learning is stored in the memory as the speaker-normalized HMM 33 in step S7. Then, the speaker normalization processing ends.

Next, the speaker adaptation processing of the MLLR method using the MAP estimation method will be described. Since the MLLR method estimates the average vector for the learning adaptation data on the basis of the maximum likelihood, it is not speaker adaptation that effectively uses prior knowledge of the initial model. Therefore, even if the speaker normalization model has good prior knowledge, there is a possibility that it cannot be fully utilized. Therefore, MA, which is a method to effectively use prior knowledge,
P estimation method (for example, see Prior Art Document 2 “CHLee et al.,
“A Study on Speaker Adaptation of the Parameters
of Continuous Density Hidden Markov Models ", IEEE T
ransactions onSignal Processing, Vol.39, No.4, pp.806
-814, 1991. " ) Applied to the MLLR method (hereinafter referred to as the MAP-MLLR method.
Processing by the LLR method is called MAP-MLLR processing. )
We have invented the following speaker adaptation. Here, the average vector μh _k ^MAP of the Gaussian distribution k after speaker adaptation by the MAP-MLLR method is given by the following equation.

[0032]

Μh _k ^MAP = A _{c, k} ^MAP μ _k + b _{c, k} ^MAP

(Equation 15)

Here, I is an n × n unit matrix, and τ
_k is a constant related to the certainty of prior knowledge. In the preferred embodiment, τ _k = 4.0 is set.

The estimation of the average vector by the MAP estimation method is a linear combination of the average vector (prior knowledge) based on the initial model and the average vector based on the maximum likelihood estimation. FIG.
FIG. 3 is a conceptual diagram of estimation of an average vector by an MLLR method using a MAP estimation method. The thickness of the arrow in FIG. 6 indicates the magnitude of the expected value at which the learning data is observed in the Gaussian distribution. As in the example of FIG. 6, a Gaussian distribution having a large expected value at which learning data is observed is represented by the MLLR
It is estimated around the average vector estimated by the method.
Conversely, in the Gaussian distribution where the observed expected value is small, the value is estimated near the average vector based on the initial model.
By introducing the MAP estimation method in this manner, speaker adaptation using information of prior knowledge is appropriately performed in consideration of the reliability of average vector estimation by speaker adaptation by the MLLR method. Here, the method of the present embodiment estimates all coefficients and obtains transform coefficients for each Gaussian distribution. For this reason, the method of the present embodiment can perform speaker adaptation more precisely than the conventional example.

FIG. 3 is a flowchart of the speaker adaptation process executed by the speaker adaptation control unit 21 of FIG. 1, and FIG. 8 is a block diagram thereof. In FIG. 3, first, in step S11, the speaker-normalized HMM 33 and the speaker adaptation learning data 34 including the feature vector of the speech data of the speaker to be speaker-adapted are read. Next, in step S12, the conversion coefficients A _c and b _c are calculated by the MLLR method using the equations 6 to 11. Then, step S13
Then, the conversion coefficients A _{c, k} ^MAP and b _{c, k} ^MAP are calculated by the MAP method using Expressions 14 and 15. Further, a linear transformation process is performed using Expression 13 to obtain a speaker-adapted HMM 11. Finally, the speaker-adapted HMM 11 is stored in the memory. Thus, the speaker adaptation processing by the MAP-MLLR method ends.

The speaker-adapted HMM 11 is connected to the phoneme matching unit 4 and can be represented as an HM network as a network in a plurality of states. Each state in the HMM 11 can be regarded as one stochastic stationary signal source in the sound space, and each has the following information. (A) state number, (b) acceptable context class, (c)
A list of preceding and following states, (d) probability distribution parameters assigned on the speech feature space,
(E) Self transition probability and transition probability to the succeeding state. In the speaker-adaptive HMM 11, when input data and its context information are given, a model for the input data is connected by concatenating states capable of accepting the context within constraints of the preceding and succeeding state lists. It can be determined uniquely. Here, the output probability density function is a Gaussian mixture distribution having a 34-dimensional diagonal covariance matrix (hereinafter, referred to as a Gaussian distribution).
Each Gaussian distribution is speaker-normalized by the speaker normalization control unit 20 using the initial HMM 31, and is speaker-adapted by the speaker adaptation control unit 21 by using the speaker-normalized HMM 33. . Note that the speaker-normalized HMM 33 may be connected to the phoneme matching unit 4 and used for phoneme detection.

It is generally known that when speaker adaptation is performed on a model based on a continuous distribution type HMM with a small amount of adaptation data, adaptation of the average value of the Gaussian distribution is more effective than adaptation of other parameters. (For example, Prior Art Document 3, "Kumi Okura et al.," Moving vector field smoothing speaker adaptation method using mixed continuous distribution HMM ", Proc. Of the Acoustical Society of Japan, 2-Q-17, pp. 191-192, 1992
March ". ). In the present embodiment, only the average value of each Gaussian distribution is applied, and the variance, the state transition probability, and the weight coefficient of the mixed Gaussian distribution are not applied.

Next, an SSS-LR (left-t) using the above-described speaker normalization method and speaker adaptation method of the present embodiment.
An o-right rightmost type) speaker-independent continuous speech recognition device will be described. This device is a phoneme environment-dependent efficient HM stored in the memory of the HM network including the HMM 11.
M expression format is used. In the SSS, the likelihood of a stochastic model expressing a temporal transition of a speech parameter by a stochastic transition between stochastic stationary signal sources (states) assigned to a feature space of a phoneme is calculated. The refinement of the model is performed sequentially by repeating the operation of dividing each state in the context direction or the time direction based on the criterion of maximization.

In FIG. 1, a speaker's uttered voice is input to a microphone 1 and converted into a voice signal, and then input to a feature extracting unit 2. After performing A / D conversion on the input audio signal, the feature extraction unit 2 performs, for example, LPC analysis, and performs 34-dimensional feature parameters including logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. Is extracted. The time series of the extracted feature parameters is input to the phoneme matching unit 4 via the buffer memory 3.

The phoneme matching unit 4 executes a phoneme matching process in response to a phoneme matching request from the phoneme context-dependent LR parser 5. Then, the likelihood for the data in the phoneme matching section is calculated using the speaker model of the phoneme HMM stored in the memory of the speaker-adapted HMM 11, and the value of the likelihood is used as the phoneme matching score in the LR parser 5. Is returned to
At this time, a forward path algorithm is used.

On the other hand, a predetermined context-free grammar (CFG) in the context-free grammar database 13 is automatically converted, as is well known, to create an LR table 12, which is stored in its memory. The LR parser 5 refers to the LR table 12 and processes the input phoneme prediction data from left to right without regression. If there is syntactic ambiguity, the stack is split and the analysis of all candidates is processed in parallel. The LR parser 5 uses the LR table 1
It predicts the next phoneme from 2 and outputs phoneme prediction data to the phoneme matching unit 4. In response, the phoneme matching unit 4
The matching is performed with reference to the information in the HMM 11 corresponding to the phoneme, the likelihood is returned to the LR parser 5 as a voice recognition score, and the continuous voice recognition is performed by sequentially connecting the phonemes. If multiple phonemes are predicted in the above continuous speech recognition, check for the presence of all of them,
By the beam search method, high-speed processing is realized by performing pruning to leave a partial tree having a high likelihood of partial speech recognition.

[0042]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present inventor conducted an evaluation experiment on a speech recognition apparatus having the above-described configuration as follows. As an evaluation experiment, a phoneme typewriter-type phoneme recognition experiment for 26 phonemes without using language constraints was performed. Table 1 shows the acoustic analysis conditions and the audio data used.

[0043]

[Table 1] Experimental conditions ─────────────────────────────────── Analysis conditions Sampling frequency 12 kHz 20 ms Hamming window Frame Period 5 ms 使用 Parameter used 16th order LPC cepstrum + 16th order cepstrum + log power + Δlog Power ─────────────────────────────────── Learning data 9 men selected from 146 men and 139 women Name, 6 women (50 sentences each) 適 応 Adaptation / recognition data Person 3 men (MAU, MMY, MTM) 3 women (FAF, FMS, FYM) Adaptation data 598 clauses (SB1 , Clauses SB2, SB4 tasks) n clauses randomly extracted from the recognition data 279 clauses (SB3 task) ─────────────────────────── ────────

The shared structure of the state of the acoustic model before adaptation (H
The M network uses the word utterance of one male speaker and uses the sequential state division method (for example, the prior art document 4 “J. Takami et al., A Su
ccessive State Splitting Algorithm for Efficient A
llophone Modeling ", Proceedings of CASSP'92, pp.573-
576, 1992 ". ). 20 states
A model to which a silence model of 1 state (5 mixtures) and 1 state (10 mixtures) was added was used. The number of shared classes of the MLLR method used for speaker normalization and speaker adaptation was set to one.
That is, all the Gaussian distributions are shared, and the transform coefficients are estimated.

A speaker normalization model and a conventional SI-HMM model for comparison are prepared using speech data of 15 speakers, and are created by Baum-Welch.
Learning was performed with the algorithm. These 15 speakers are 285
As a representative speaker from a human model, a clustering method (for example, see Prior Art Document 5 “T.Kosaka et al.,“ Tree-S
tructured Speaker Clustering For Fast Speaker Adap
tation ", Proceedingsof ICASSP'94, pp.245-248,1994
See year. ). The number of repetitions of the speaker normalization process in step S5 is three. Further, in the speaker adaptation processing, the constant τ _k relating to the certainty of the prior knowledge of the MAP estimation method was set to the same value in all Gaussian distributions, and an experimentally determined value of 4.0 was used. Supervised speaker adaptation is performed according to the procedures shown in FIGS.
The average phoneme recognition rate was obtained by repeating the evaluation in which the selected phrase was changed for each number of adaptive phrases three times.

First, in order to confirm the improvement of the discrimination performance by the speaker normalization, a phoneme recognition experiment was performed using the speaker normalized HMM 33 without any adaptive processing. Table 2 shows the results. For comparison, the recognition result of the conventional SI-HMM model is also described.

[0047]

[Table 2] Speech recognition results using speaker-normalized HMM Phoneme error rate (%) Upper: Speaker normalized model, Lower: Unspecified speaker model ─────────────────────── MAU MMY MTM FAF FMS FYM average ─────────────────── １５ 15.2 15.2 12.0 20.2 18.4 29.5 18.4 15.5 17.0 13.3 21.9 25 .2 33.4 21.1}

As is clear from Table 2, the recognition rate of the speaker-normalized model is high in all six evaluation speakers, and the average phoneme error rate decreases from 21.1% to 18.4% (12.8%).
Error reduction rate). In particular, the improvement effect is large for speakers (FMS, FYM) with a low recognition rate in the conventional SI-HMM model. It is considered that the speaker normalization reduces the variance of the Gaussian distribution, clarifies the discrimination between recognition units, and improves the performance.

Next, Table 3 shows the case where the speaker-normalized HMM 33 is used as the initial model and the conventional SI-H
4 shows recognition results of speaker adaptation by the MAP-MLLR method when the MM model is used.

[0050]

[Table 3] Speech recognition result using speaker-adapted HMM Phoneme error rate (%) Upper: Speaker normalized model, Lower: Unspecified speaker model {Speaker adaptation number of clauses 3 5 7 10 20} {MAU 15.8 15.0 14.9 15.2 13.7 16.4 15.7 14.9 15.3 14.3} ＭＭ MMY 15.3 14.6 14.4 14.2 13.6 17.3 16.0 16.0 15.3 14.6 Ｍ MTM 11.8 11.8 11.0 10.9 9.9 13.3 13 .2 12.8 12.3 10.6 ───────────────────────────── FAF 19.0 16.8 15.6 14.9 14.1 21.8 19. 8 18.5 16.5 15.1 FMS 19.5 18.5 17.7 16.6 13.9 26.3 23.9 22.4 20.0 15.6 {FYM 26.6 23.9 23.2 21.4 19.4 29.6 24.0 25.4 24.2 19.6} {Average 18.0 16.8 16.1 15.6 14.1 20.8 18.8 18.3 17.2 14.9} ──────────────────── ───

As is apparent from Table 3, the speaker adaptation using the speaker-normalized HMM 33 as the initial model shows a high recognition rate for all speakers and the number of phrases. The speaker normalization model is an initial model having prior knowledge suitable for speaker adaptation, and realizes accurate speaker adaptation.

As described above, according to the present embodiment, a method for creating a speaker normalization model using a multiple regression mapping model has been invented. The acoustic model 33 created by this speaker normalization method obtained higher performance in phoneme recognition than the conventional SI-HMM model. Also, the speaker-normalized HMM 33 is used as an initial model, and MAP-MLLR
Even when speaker adaptation was performed by the method, the prior knowledge of the initial model was reflected and accurate speaker adaptation was realized. Further, even when the amount of training application data is small, the accuracy of estimating the parameters of the speaker-normalized or speaker-adapted HMM can be greatly improved as compared with the conventional example.

[0053]

As described above in detail, according to the speaker normalizing apparatus of the first aspect of the present invention, the learning data for learning the initial model of a predetermined hidden Markov model is a plurality of learning data. A storage device for storing a feature vector of voice data depending on a speaker, and a maximum likelihood linear regression method for an initial model of the hidden Markov model based on the feature vector of the voice data stored in the storage device. A first conversion coefficient including a conversion matrix for converting the average vector based on the multiple regression mapping model and a constant term vector representing an individual difference common to the spectrum is calculated for each speaker. The constant vector obtained by subtracting the constant term vector calculated by the first calculating means for each speaker from the calculating means and the feature vector of the voice data stored in the storage device is normalized. A second calculating means for calculating a feature vector of the voice data, and an initial model of the hidden Markov model based on a normalized feature vector of the voice data calculated by the second calculating means, the predetermined model A third calculating means for calculating model parameters of the speaker-normalized hidden Markov model by learning using an algorithm. Therefore, the estimation accuracy of the parameters of the Hidden Markov Model can be greatly improved by the speaker normalization device as compared with the conventional example.
By performing speech recognition using the speaker-normalized hidden Markov model obtained by the speaker normalization device, speech recognition can be performed at a higher speech recognition rate than in the related art.

According to the speaker adapting apparatus according to the second aspect of the present invention, the speaker normalizing apparatus according to the first aspect is based on the feature vector of the voice data of the speaker to be speaker-adapted. A second transformation coefficient including a transformation matrix for transforming an average vector based on a multiple regression mapping model and a constant term vector by a maximum likelihood linear regression method on the hidden Markov model calculated by the third calculation means. And a second conversion coefficient including a conversion matrix and a constant term vector calculated by the fourth calculation means,
A fifth transformation coefficient including a transformation matrix for transforming an average vector based on a speaker-adapted multiple regression mapping model and a constant term vector is calculated by a maximum posterior probability estimation method.
By performing a predetermined linear transformation process on the third transformation coefficient including the transformation matrix and the constant term vector computed by the fifth computing means, the hidden matrix after speaker adaptation is obtained. A sixth calculating means for calculating an average vector of the Markov model. Therefore, the estimation accuracy of the parameters of the speaker adaptation can be greatly improved by the speaker adaptation device as compared with the conventional example, and the speaker adaptation obtained by the speaker adaptation device can be improved. By performing voice recognition using the hidden Markov model, voice recognition can be performed with a higher voice recognition rate than the conventional example.

According to a third aspect of the present invention, an input uttered voice sentence is obtained by using the hidden Markov model calculated by the third calculating means of the speaker normalizing device according to the first aspect. Voice recognition means for performing voice recognition on the basis of the voice signal and outputting a voice recognition result. Therefore, by performing the speech recognition using the speaker-normalized hidden Markov model obtained by the above-described speaker normalization apparatus, speech recognition can be performed at a higher speech recognition rate than the conventional example.

Further, according to the speech recognition apparatus of the fourth aspect, the hidden Markov model including the average vector of the hidden Markov model calculated by the sixth calculation means of the speaker adaptation apparatus of the second aspect is used. Voice recognition means for performing voice recognition based on the voice signal of the input uttered voice sentence and outputting a voice recognition result. Therefore, by performing speech recognition using the speaker-adapted hidden Markov model obtained by the speaker adaptation apparatus, speech recognition can be performed at a higher speech recognition rate than the conventional example.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice recognition device according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a speaker normalization process performed by a speaker normalization control unit in FIG. 1;

FIG. 3 is a flowchart illustrating a speaker adaptation process performed by a speaker adaptation control unit in FIG. 1;

FIG. 4 is a diagram illustrating an MLLR process executed by a speaker normalization control unit in FIG. 1;

FIG. 5 is a diagram illustrating a speaker normalization process performed by a speaker normalization control unit in FIG. 1;

FIG. 6 is a diagram illustrating a speaker adaptation process performed by a speaker adaptation control unit in FIG. 1;

FIG. 7 is a block diagram illustrating a speaker normalization process performed by a speaker normalization control unit in FIG. 1;

FIG. 8 is a block diagram illustrating a speaker adaptation process performed by the speaker adaptation control unit in FIG. 1;

[Description of Signs] 1 ... Microphone, 2 ... Feature extraction unit, 3 ... Buffer memory, 4 ... Phoneme collation unit, 5 ... LR parser, 11 ... HMM with speaker adaptation, 12 ... LR table, 13 ... Context free Grammar database, 20: speaker normalization control unit, 21: speaker adaptation control unit, 31: initial HMM, 32-1 to 32-M: voice data of speakers 1 to M, 33: speaker normalized HMM, 34... Learning data for speaker adaptation.

Continuation of the front page (56) References Acoustical Society of Japan Autumn Research Conference 1996 Annual Meeting I-3-3-17 “Study of speaker adaptation method based on multiple regression model” p. 119-120 (September 25, 1996) Proceedings of 1995 IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 1 "Speaker Adoption based on Spectral Normalization and Dynamic HMM Parameter adaptation" p. 704-707 Proceedings of the Acoustical Society of Japan 1997 Spring Meeting, ▲ I ▼ 2-6-16, "Speaker normalization method for speaker adaptation using multiple regression models" p. 75-76 (March 17, 1997) Proceedings of the Fall Meeting of the Acoustical Society of Japan in 1995 (I) 3-2-9 “Examination of unspecified speaker model using state-based speaker clustering” p. 123-124 (September, 1995) Proceedings of the Acoustical Society of Japan Spring Meeting, 1995, I, 2-5-6, "Effect of smoothing coefficient control on MA P-VFS speaker adaptation method" p . 41-42 (March 1995) IEICE Technical Report [Voice] Vol. 94 No. 271 SP94-51 "Speaker adaptation method integrating maximum a posteriori probability estimation method and moving vector field smoothing method" p. 25-30 (October 13, 1994) Proceedings of the Acoustical Society of Japan Spring Meeting, 1996, I, 1-5-22, "Study on speaker adaptation using restricted multiple regression model" p. 51-52 (issued on March 26, 1996) Proceedings of 1996 IEEE International Conference on Spokane Language Processing, "Novell Training Technology Association for Associates Classification for Associates." 2119-2122, 1996 Proceedings of 1996 IEEE International Conference on Spokane Language Processing, "Compact Model for Speaker-Adaptive Training." 1137-1140, 1996 Proceedings of 1996 IEEE International Conference on Acoustics, Speech and Signal Processing Vol. 2, "Normalized Discriminant Analysis with Application to a Hybrid Speed Maker-Verification System", p. 681-684 Processings of 1996 IEEE International Conference on Acoustics, Speech and Signal Processing Vol. 1, "Speaker Backpack Model Models for Connected Digit Password Speaker Verification" p. 81-84 Proceedings of 1981 IEEE International Conference on Acoustics, Speech and Signal Processing Vol. 1/3, "Speaker Identification and Verification Combined with Speaker Independent Word Recognition" p. 184-187 Proceedings of 1997 IEEE International Conference on Acoustics, Speech and Signal Processing Vol. 2, "Speaker-Adapted Training on the Switchboard Corpus" p. 1059-1062 Proceedings of 1997 IEEE International Conference on Acoustics, Speech and Signal Processing Vol. 2, "Speaker-Adactive Training: A Maximum Likelihood Approach to Speaker Normalization" p. 1043-1046 (58) Field surveyed (Int. Cl. ⁷ , DB name) G10L 15/14 G10L 15/06 G10L 15/10 JICST file (JOIS)

Claims (4)

(57) [Claims] 1. A storage device for storing an initial model of a predetermined Hidden Markov Model, wherein the storage device stores a feature vector of voice data dependent on each of a plurality of speakers. Based on the feature vector of the voice data, the initial model of the Hidden Markov Model is subjected to the maximum likelihood linear regression method, using a transformation matrix for transforming the average vector based on the multiple regression mapping model, and individual differences common to the spectrum. A first calculating means for calculating, for each of the speakers, a first conversion coefficient including a constant term vector representing the following, and for each of the speakers from the feature vector of the voice data stored in the storage device: A second computing means for subtracting the constant term vector computed by the first computing means to compute a feature vector of the speech data normalized, and the second computing means By learning the initial model of the hidden Markov model using a predetermined learning algorithm based on the feature vector of the normalized speech data calculated by the above, model parameters of the speaker-normalized hidden Markov model And a third calculating means for calculating the following. 2. A hidden Markov model calculated by the third calculating means of the speaker normalization apparatus according to claim 1, based on a feature vector of the voice data of the speaker to be speaker-adapted. Fourth arithmetic means for calculating a second conversion coefficient including a conversion matrix and a constant term vector for conversion of an average vector based on the multiple regression mapping model by a likelihood linear regression method; A transformation matrix and a constant term for transforming an average vector based on a speaker-adapted multiple regression mapping model by a maximum posterior probability estimation method based on the transformed transformation matrix and a second transformation coefficient including a constant term vector A fifth calculating means for calculating a third conversion coefficient including the vector, and a predetermined linear conversion processing for the third conversion coefficient including the conversion matrix and the constant term vector calculated by the fifth calculating means. The by executing speaker adaptation apparatus characterized by comprising a sixth calculating means for calculating an average vector of the hidden Markov model after speaker adaptation. 3. A speech recognition apparatus using the hidden Markov model computed by the third computation means of the speaker normalization apparatus according to claim 1, and performing speech recognition based on the speech signal of the input uttered speech sentence. And a voice recognition unit for outputting a recognition result. 4. A speech signal of an input uttered speech sentence using a hidden Markov model including an average vector of a hidden Markov model computed by the sixth computing means of the speaker adaptation apparatus according to claim 2. And a voice recognition unit for performing voice recognition based on the voice recognition result and outputting a voice recognition result.