JP2003131687A - System, method, and program for voice recognition sound model learning and adaptation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate adapted HMM by providing an HMM estimating method based upon a framework for MAP estimation which can be extracted from learning data and constituted and estimating a weight belonging to a cluster of a previous distribution for a new speaker. SOLUTION: An initial value setting part 111 of a data processor 101 sets initial values of variables in a cluster weight variable storage part 122, an HMM variable storage part 123, and a cluster variable storage part 124, a cluster weight variable optimization part 112 optimizes cluster weight variables by speakers, and an HMM variable optimization part 113 optimizes the HMM variables of the respective learners. A convergence decision part 114 decides whether a cluster weight variable or HMM variable converges and instructs the process to be repeated when not. A cluster variable optimization part 115 optimizes the cluster variable and a convergence decision part 116 decides whether the cluster variable is optimized and instructs the process to be repeated when not.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition acoustic model learning / adapting system, a speech recognition acoustic model learning / adapting method, and a speech recognition acoustic model learning / adaptation program. The present invention relates to a speech recognition acoustic model learning / adaptation system, a speech recognition acoustic model learning / adaptation method, and a speech recognition acoustic model learning / adaptation program for constructing and adapting a speaker.

[0002]

2. Description of the Related Art Conventionally, as one of acoustic model creating methods in a speech recognition apparatus, maximum posterior probability estimation (MAP estimation) has been performed.
There is a Hidden Markov Model (HMM) estimation method based on the framework of <April 1994, IEE Transactions on Speech and Audio Processing, Volume 2, No. 2, No. 291. −
Item 298 (IEEE TRANSACTIONS ON
SPEECH ANDAUDIO PROCESSESI
NG, VOL. 2, NO. 2, pp 291-298, A
PRIL 1994)> (Reference 1).

The method described in the above-mentioned document 1 estimates the variables constituting the HMM using a model in which they are distributed in advance, and also estimates the variables constituting the prior distribution.

The HMM estimation method based on the framework of MAP estimation shown in the above Document 1 and the prior distribution construction method will be described by using equations.

FIG. 5 is an explanatory diagram showing a list of equations (1) to (7).

Equation (1) is an HMM based on MAP estimation.
Represents the variable λ estimation method. Here, the HMM variable λ is HM
A model parameter that completely describes M. G (λ)
Is a prior distribution representing the appearance distribution of the HMM variable λ. The prior distribution G is described by the prior distribution variable that is a model parameter of the prior distribution.

F (x | λ) is the likelihood that the learning data x appears under the HMM described by the HMM variable λ. Formula (1) determines λ so that the posterior probability f (x | λ) G (λ) becomes maximum with respect to the learning data x under the known prior distribution G, and the value is determined as ˜λ. This is expressed (expressed as argmax).

That is, it is an HMM variable λ estimation method based on MAP estimation. The variable estimation formula in the case of the HMM having the mixed Gaussian distribution is shown in Chapter 4 of Reference 1. The prior distribution construction method is a method in which the formula (1) and the formula (2) are repeatedly used.

Equation (2) represents that when the HMM variable λ is fixed, the prior distribution variable φ is determined so that its appearance probability becomes maximum, and its value is set to Φ. Here, the prior distribution belongs to the mathematical classification described in Chapter 2 of Reference 1 in order to simplify the mathematical handling. Mathematical formula (1) is used for a certain initial value to determine λ, and then λ
Then, φ is determined by using the equation (2).

It is determined that this procedure has converged when φ and λ are invariant with respect to repetition or when the change is sufficiently small. That is, the HMM estimation method based on the prior distribution construction method and the MAP estimation framework.

In Reference 1, the prior distribution consisted of one distribution. On the other hand, based on the method of Reference 1, a method in which the prior distribution is extended to a linear combination of a plurality of functions is <2000, iEeEe
Proceedings IC SA S
P00, Volume 2, Item II993 (Proc. IE
EE ICASSP-00, VOL. 2, II993
2000)> (Reference 2).

The HMM estimation method by learning based on the framework of the MAP estimation using the conventional technique shown in the above-mentioned Document 2 and the HMM adaptation method will be described using figures and mathematical formulas.

FIG. 6 is an explanatory diagram for explaining a conventional learning method (HMM estimation method based on a MAP estimation framework).

Referring to FIG. 6, this conventional technique comprises a data processing device 501 and a storage device 502.

The storage device 502 has a learning data storage unit 521 for storing large-scale learning data of a large number of speakers, a cluster weight variable storage unit 522 for representing a weight of belonging to a prior distribution cluster, and an HMM for an unspecified speaker. And an HMM variable storage unit 523 that configures the above and a cluster variable storage unit 524 that determines the structure of the prior distribution cluster. The prior distribution cluster refers to each Gj in which the prior distribution G is composed of a plurality of functions Gj. For example, G-cell, G-fixed, etc. when the learning data is composed of different data sets such as mobile phone data and fixed phone data.

The cluster weight variable is a weighting coefficient of each prior distribution cluster Gj with respect to the prior distribution G, and is represented by qj in Expression (3). The cluster variable is a model parameter of each prior distribution cluster Gj and is represented by φj.

All the learning data are composed of a plurality of clusters, and the cluster weight variable and the cluster variable are given in advance so as to follow the existence weights of the clusters in the all learning data. Formula (3)
Is a cluster G with J prior distributions G that the HMM variable λ follows.
It is composed of j and represents that the weight variable to each cluster is qj. Gj is determined by each cluster variable φj.

The data processing device 501 comprises an HMM variable optimizing section 511 for optimizing HMM variables, and a convergence judging section 512 for judging whether the HMM variables have converged.
In the HMM variable optimization unit 511, regarding the HMM variable,
The MAP estimation is performed using the data and variables stored in the storage device 502, the HMM variable is repeatedly updated, and when the convergence determination unit 512 determines that this HMM variable has converged, the update ends. H with mixed Gaussian distribution
The MM variable estimation formula is shown in Chapter 2 of Reference 2.

Next, the prior art M shown in Document 2 is used.
The HMM adaptation method based on the framework of AP estimation will be described. H
What is the MM adaptation method? HMM is applied to new speakers who do not belong to the learning speaker.
Adapting variables.

FIG. 7 is a block diagram for explaining an HMM adaptation method based on the conventional MAP estimation framework.

Referring to FIG. 7, in this conventional technique, an input device 600, a data processing device 601, and a storage device 60.
2 and.

The input device 600 is a device for inputting the utterance of a new speaker. The storage device 602 includes a new speaker data storage unit 621 that stores input data of a new speaker, a cluster weight variable storage unit 622, an HMM variable storage unit 623, and a cluster variable storage unit 624. The cluster weight variable is given by reflecting the weight belonging to the cluster of the learning data when the new speaker learns. The data processing device 601 is an HM
HMM variable optimization unit 611 for optimizing M variables
And a convergence determination unit 612 that determines whether the HMM variables have converged.
Consists of. The data processing device 601 of FIG. 7 operates similarly to the data processing device 501 of FIG. Prior art HM
At the time of M adaptation, an estimation method similar to the HMM estimation method is performed on the new speaker data to adapt the HMM variable to the new speaker.

The invention described in Japanese Patent Application Laid-Open No. 10-207485 is a speech recognition technique using the maximum posterior probability estimation method.

[0024]

The first problem of the above-mentioned prior art is that the cluster weight variable and the method of configuring the cluster variable at the time of learning are not flexible.

The reason is that the cluster weight variable and the method of determining the cluster variable are given to all the learning data by the existence weights of the respective clusters that are distinguished in advance.

The second problem is that the HMM adaptation method is not flexible.

The reason is that the cluster variable determined for the reason pointed out in the first problem is used, and the cluster weight variable is determined by the weight of the adapted new speaker belonging to the cluster of the original learning data. This is because a sufficient adaptation effect cannot be expected when this weight is not known.

It is an object of the present invention to provide an HMM estimation method based on a MAP estimation framework which can be constructed by extracting a flexible and effective prior distribution from learning data at the time of HMM adaptation, that is, this learning method. Is to provide.

Another object of the present invention is to adapt to HMM adaptation.
It is to provide a method of estimating a weight belonging to this prior distribution for a new speaker and a method of estimating an adapted HMM by using a flexible and effective prior distribution configured at the time of learning.

[0030]

A first speech recognition acoustic model learning / adapting system of the present invention reads learning data from a learning data storage unit for storing learning data and the learning data storage unit. An optimization means for optimizing the clustered prior distribution based on the learning data and optimizing the HMM variable based on the optimized prior distribution.

Second speech recognition acoustic model learning of the present invention
The adaptation system includes a learning data storage unit that stores learning data, a cluster variable storage unit that stores a variable that determines a clustered prior distribution, a cluster weight storage unit that stores a cluster weight variable, and an acoustic model. HMM
An HMM variable storage unit that stores variables, a cluster weight variable optimization unit that optimizes cluster weight variables for each learning data, and an HMM variable that optimizes HMM variables for each learning data.
An MM variable optimizing means and a cluster variable optimizing means for optimizing cluster variables for all learning data are provided.

Third speech recognition acoustic model learning of the present invention
The adaptation system uses the prior distribution expressed as a linear combination of speaker clusters when adapting the acoustic model to the new speaker,
It is characterized by comprising means for optimizing the cluster weight variables and adapting the HMM variables.

Fourth speech recognition acoustic model learning of the present invention
The adaptation system includes a new speaker data storage unit that stores the adaptation data of a new speaker, a cluster variable storage unit that stores a cluster variable that determines a prior distribution expressed as a linear combination of speaker clusters, and a cluster A cluster weight variable storage unit that stores the weight variable, an HMM variable storage unit that stores the HMM variable that determines the acoustic model, and optimizes the weight variable to the new speaker cluster based on the adaptation data of the new speaker. A cluster weight variable optimizing means and an HMM variable optimizing means for optimizing the HMM variable based on the adaptation data of the new speaker are provided.

First speech recognition acoustic model learning of the present invention
The adaptation method reads learning data from the learning data storage unit that stores learning data, optimizes a clustered prior distribution based on the read learning data, and determines an HMM variable based on the optimized prior distribution. It is characterized by including an optimization procedure for optimizing.

Second speech recognition acoustic model learning of the present invention
The adaptation method includes a cluster weight variable optimization procedure for optimizing the cluster weight variable in the cluster weight storage unit for each learning data in the learning data storage unit and an HMM for optimizing the HMM variable in the HMM variable storage unit for each learning data. The method is characterized by including a variable optimization procedure and a cluster variable optimization procedure for optimizing the cluster variables in the cluster variable storage unit for all learning data.

Learning the third speech recognition acoustic model of the present invention
The adaptation method is characterized by including a procedure of optimizing a cluster weight variable and adapting an HMM variable by using a prior distribution expressed as a linear combination of speaker clusters when an acoustic model is adapted to a new speaker. To do.

Fourth speech recognition acoustic model learning of the present invention
The adaptation method includes a cluster weight variable optimization procedure for optimizing a weight variable to a cluster of a new speaker based on the adaptation data of the new speaker in the new speaker data storage unit, and a new weight of the new speaker data storage unit. And an HMM variable optimizing procedure for optimizing the HMM variable based on the speaker's adaptation data.

Learning the first speech recognition acoustic model of the present invention
The adaptation program reads the learning data from the learning data storage unit that stores the learning data, optimizes the clustered prior distribution based on the read learning data, and determines the HMM variable based on the optimized prior distribution. It is characterized by causing a computer to execute an optimization procedure for optimization.

Second speech recognition acoustic model learning of the present invention
The adaptation program includes a cluster weight variable optimization procedure for optimizing the cluster weight variable in the cluster weight storage unit for each learning data in the learning data storage unit, and H for each learning data.
It is characterized by causing a computer to execute an HMM variable optimization procedure for optimizing HMM variables in the MM variable storage section and a cluster variable optimization procedure for optimizing cluster variables in the cluster variable storage section for all learning data. And

Third speech recognition acoustic model learning according to the present invention
The adaptation program uses a prior distribution expressed as a linear combination of speaker clusters when adapting the acoustic model to a new speaker, optimizes cluster weight variables, and causes a computer to execute a procedure for adapting HMM variables. Is characterized by.

Fourth speech recognition acoustic model learning of the present invention
The adaptation program includes a cluster weight variable optimization procedure for optimizing a weight variable for a new speaker cluster based on the new speaker adaptation data in the new speaker data storage unit, and a new speaker data storage unit new optimization unit. And a HMM variable optimization procedure for optimizing HMM variables based on speaker adaptation data.

Fifth speech recognition acoustic model learning of the present invention
The adaptation system uses an initial value of a cluster weight variable, an initial value of an HMM variable, and an initial value of a cluster variable that determines the structure of a speaker cluster in all learning data for each learning speaker as a cluster weight variable storage unit, An HMM variable storage unit, an initial value setting unit set in the cluster variable storage unit, and a cluster for optimizing each speaker cluster weight variable for a speaker in the cluster weight variable storage unit and updating the contents of the cluster weight variable storage unit A weight variable optimization unit and the HM
An HMM variable optimizing unit that optimizes the HMM variables for the speakers in the M variable storage unit and updates the contents of the HMM variable storage unit, a cluster weight variable for the learning speaker, and
If it is determined whether or not the HMM variable has converged, and if it is determined that the HMM variable has not converged, the process is returned to the cluster weight variable optimization unit and the HMM variable optimization unit, and further processed for all learning speakers. If it is determined that it is not processed, the process returns to the cluster weight variable optimization unit and the HMM variable optimization unit for the next learning speaker.
And the first convergence determination unit has determined that all the learning speakers have been processed, the cluster variables of the cluster variables are optimized to optimize the speaker cluster configuration for all the speakers in the cluster variable storage unit. A cluster variable optimizing unit that performs optimization and updates the contents of the cluster variable storage unit, and whether or not the cluster variables have converged. If it is determined that the cluster variables have not converged, the cluster variable optimizing unit performs the A cluster weight variable optimizing unit and a second convergence determining unit for returning the processing to the HMM variable optimizing unit are provided so as to perform optimization using the cluster variables stored in the cluster variable storage unit. Characterize.

Fifth speech recognition acoustic model learning of the present invention
The adaptation method is such that, for each learning speaker, the initial value of the cluster weight variable, the initial value of the HMM variable, the initial value of the cluster variable that determines the structure of the speaker cluster in all the learning data, An initial value setting procedure to be set in the HMM variable storage unit and the cluster variable storage unit, and a cluster for optimizing each speaker cluster weight variable for the speaker in the cluster weight variable storage unit and updating the contents of the cluster weight variable storage unit Weight variable optimization procedure and the HMM
An HMM variable optimization procedure for optimizing the HMM variables for the speaker in the variable storage unit and updating the contents of the HMM variable storage unit, a cluster weight variable for the learning speaker, and
If it is determined whether or not the HMM variables have converged, and if it is determined that they have not converged, the processing is returned to the cluster weight variable optimization procedure and the HMM variable optimization procedure, and further processed for all learning speakers. If it is determined that it is not processed, the first convergence determination procedure for returning the processing to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker; When it is determined that the convergence determination procedure of (1) has been processed for all the learning speakers, the cluster variables are optimized so that the speaker cluster configuration for all the speakers in the cluster variable storage unit is optimized, and the cluster variable storage is performed. The cluster variable optimization procedure for updating the contents of the section, and whether or not the cluster variables have converged. If it is determined that the cluster variables have not converged, the cluster variable optimization procedure performs the cluster variable optimization procedure. A cluster weight variable optimization procedure and a second convergence determination procedure for returning the processing to the HMM variable optimization procedure so that optimization is performed using the cluster variables stored in the variable storage unit. To do.

Fifth speech recognition acoustic model learning of the present invention
The adaptation program stores, for each learning speaker, an initial value of a cluster weight variable, an initial value of an HMM variable, and an initial value of a cluster variable that determines the structure of a speaker cluster in all learning data. An initial value setting procedure to be set in the HMM variable storage unit and the cluster variable storage unit;
A cluster weight variable optimization procedure for optimizing each speaker cluster weight variable for the speaker in the cluster weight variable storage unit and updating the contents of the cluster weight variable storage unit, and an HMM variable for the speaker in the HMM variable storage unit. And a HMM variable optimization procedure for updating the contents of the HMM variable storage unit, a cluster weight variable for a learning speaker,
Further, it is determined whether or not the HMM variable has converged, and if it is determined that the HMM variable has not converged, the process is returned to the cluster weight variable optimization procedure and the HMM variable optimization procedure, and further, for all learning speakers. If it is determined that it has not been processed, and if it is determined that it has not been processed, the first convergence determination procedure for returning the processing to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker, When it is determined that the first convergence determination procedure has been processed for all learning speakers,
The cluster variables are optimized so that the speaker cluster configuration for all speakers in the cluster variable storage unit is optimal,
A cluster variable optimization procedure for updating the contents of the cluster variable storage unit, and whether or not the cluster variables have converged, and if it is determined that the cluster variables have not converged, the cluster variable storage unit is stored in the cluster variable storage unit by the cluster variable optimization procedure. The computer is made to execute the cluster weight variable optimization procedure and the second convergence determination procedure for returning the processing to the HMM variable optimization procedure so as to perform the optimization using the stored cluster variable. .

[0045]

BEST MODE FOR CARRYING OUT THE INVENTION Next, features of the first embodiment of the present invention will be described.

At the time of learning, a clustered prior distribution by speaker characteristics is constructed from a large-scale database. When the HMM is adapted to the new speaker, an acoustic model adapted with the new speaker's utterance data is created using a prior distribution composed of sufficient data at the time of learning.

The clustered prior distribution configuration can be performed in advance using sufficient learning data at the time of learning, and this prior distribution configuration method uses variable optimization of each learning speaker and all data. Since global cluster variable optimization is repeatedly executed hierarchically, it is equivalent to the variable optimization of each learning speaker during new speaker adaptation and constructs a prior distribution cluster that is effective during speaker adaptation. can do.

Therefore, it is expected that sufficient speaker adaptation can be realized only by determining the cluster weight variable and the HMM variable with a small number of utterances during HMM adaptation to a new speaker. In addition, since the recognition unit of the acoustic model that did not exist during speaker adaptation is also optimized for the speaker cluster weight, it can be effectively speaker-adapted with the weighted prior distribution. Expected

Next, the difference from the prior art will be described.

In the conventional learning method, the prior distribution cluster, that is, the cluster variable, and the cluster weight variable which is a weight for the prior distribution cluster, directly reflect the cluster of the learning data itself. In the conventional adaptation method, the cluster weight variable is given as a known value when adapting to a new speaker.

On the other hand, in the learning method according to the embodiment of the present invention, the variable optimization of each learning speaker and the global cluster variable optimization by all data are hierarchically repeatedly executed, and the story inherent in the data is The cluster variables are automatically determined so that the prior distribution that reflects personality constitutes a prior distribution cluster that is effective during speaker adaptation. Further, in the adaptation method according to the present embodiment, the cluster weight variable is optimized, and the HMM adaptation to a new speaker is performed using the prior distribution that reflects the effective speaker characteristic.

Next, a first embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the first embodiment of the present invention.

Referring to FIG. 1, the first embodiment of the present invention comprises a data processing device 101 which operates under program control and a storage device 102 which stores information.

The storage device 102 stores a learning data storage unit 121 for storing large-scale learning data of a large number of speakers, a cluster weight variable storage unit 122 for indicating the weight of belonging to a prior distribution cluster, and each story in the learning data. H that constitutes the person's HMM
The MM variable storage unit 123 and the cluster variable storage unit 124 that represents speaker clusters in all learning data as clusters of prior distribution and determines the structure of each cluster. Here, the speaker cluster means that each cluster is a set of speakers having close speaker characteristics, and this is not given in advance but is extracted as a prior distribution cluster.

The data processing device 101 includes an initial value setting unit 111 for setting initial values of variables, a cluster weight variable optimization unit 112 for each learning speaker, an HMM variable optimization unit 113 for each learning speaker, A convergence determination unit 114 that determines whether the cluster weights and the HMM variables have converged, a cluster variable optimization unit 115, and a convergence determination unit 116 that determines whether the speaker cluster optimization has converged for all learning data. Including.

Next, the operation of the first embodiment of the present invention will be described with reference to the drawings.

In particular, the operation of the learning part which constitutes a clustered prior distribution based on speaker characteristics from a large-scale database of many speakers will be described.

FIG. 2 is a flow chart showing the operation of the first embodiment of the present invention.

Referring to FIG. 2, first, the initial value setting unit 1
11 is the initial value of the cluster weight variable (q, the numerical value for all clusters), the initial value of the HMM variable (λ, a set of numerical values), and the structure of the speaker cluster in all the learning data for each learning speaker. Initial values of the cluster variables (φ, a set of numerical values) that determine
(From recording medium or the like), cluster weight variable storage unit 122, H
The values are set in the MM variable storage unit 123 and the cluster variable storage unit 124 (step A1 in FIG. 2). In the initial value setting unit 111, if there is no prior knowledge, it is assumed that the cluster weight variables are evenly distributed, and the cluster variables also have a single prior distribution function for the unspecified HMM variable, that is, the prior distribution determination in Literature 1. It shall be decided by law.

For the HMM variables, M
The unspecified speaker HMM variable is determined by the maximum likelihood estimation framework instead of the AP estimation framework and used as the initial value. It is assumed that the functional form of the prior distribution Gj in the mathematical expression (3) belongs to the same mathematical classification as the literature 1 and the literature 2 for easy mathematical handling. Here, it is not necessary to set constraints on all cluster variables, or a constraint condition may be set to reduce free variables. If you have knowledge in advance, you can use that information as the initial value. For example, it is possible to construct a speaker cluster of static data based on the likelihood criterion from the learning data, estimate the HMM for each data cluster, and use the result as prior knowledge.

Here, it is assumed that the learning data has been set in the learning data storage unit 121 in advance.

Regarding steps A2 to A4,
All learning speakers are executed for each learning speaker. Here, the explanation will be given by focusing on the speaker α.

Next, the cluster weight variable optimization unit 112
, Optimizes the weight variable q belonging to each speaker cluster for the speaker α in the cluster weight variable storage unit 122, and updates the contents of the cluster weight variable storage unit 122 (step A2 in FIG. 2). This is represented by equation (4). {Q}
Represents the set for all j.

[Mathematical formula-see original document] Equation (4) is such that the speaker α becomes the MAP optimum when the cluster weight variable {q ^α } for each speaker cluster is observed as the learning data x ^α with the HMM variable λ ^α of the speaker α. It means to decide. For example, in the cluster weight variable optimization unit 112, {q
such as the steepest descent method or the conjugate gradient method, which uses ^α } as a variable (in this step, all other variables are fixed)
A numerical function maximization algorithm is used.

Next, the HMM variable optimizing unit 113 causes the HM
The HMM variable λ is optimized for the speaker α in the M variable storage unit 123, and the contents of the HMM variable storage unit 123 are updated (step A3 in FIG. 2). This is represented by equation (5).

Equation (5) is the HMM variable λ ^{α of the} speaker ^α.
Represents that MAP estimation is performed when the learning data x ^α is observed using the prior distribution in which the speaker α is cluster-weighted. For example, the HMM variable optimizing unit 113 uses an estimation formula based on the EM algorithm used in Reference 2. In this step, since the HMM variables are fixed except the HMM variable, the HMM variable optimization formula of Chapter 2 of Reference 2 can be used as it is.

Next, the convergence determination unit 114 determines whether or not the cluster weight variable {q ^α } and the HMM variable λ ^α have converged for the speaker α (step A in FIG. 2).
4) If it is determined that it has not converged (step A4 in FIG. 2).
/ NO), step A2 and step A3 are repeated.

If the convergence determination unit 114 determines that the processing has converged (step A4 / YES in FIG. 2), it further checks whether or not all speakers have been processed (step A5 in FIG. 2). If judged (Fig. 2 step A5 / N
O), for the next speaker (for example, α '), start from the process of step A2.

If it is determined that all speakers have been processed (step A5 / YES in FIG. 2), the cluster variable optimization unit 11
5 optimizes the cluster variables so that the speaker cluster configuration for all speakers in the cluster variable storage unit 124 is optimized, and updates the contents of the cluster variable storage unit 124 (step A6 in FIG. 2). This is represented by equation (6). {Φ} represents a set for all j.

Expression (6) represents that the cluster variable φ is determined so that the sum of the posterior probabilities for each learning speaker is maximized. For example, in the cluster variable optimizing unit 115, a numerical function maximizing algorithm (which is general in this step because all other variables are fixed) such as the steepest descent method and the conjugate gradient method using {φ} as a variable is used. To use. Here, it is not necessary to use constraint conditions for all the cluster variables {φ}, or it is possible to reduce the free variables by using constraint conditions that are restricted by using the variational function Ψ as shown in Expression (7). good. For example, if Ψ is the number of clusters as Ψ, then J rows x J
It can be limited to those related by a linear transformation matrix of columns.
For example, φ′1 = cos θ * φ1 + sin θ * φ2.

Φ'2 = -sin θ * φ1 + cos θ * φ
If it is 2, the conversion from the cluster variable {φ} to the cluster variable {φ ′} will be limited by θ.

Next, the convergence judgment unit 116 judges whether or not the cluster variables have converged (step A7 in FIG. 2), and if it does not converge (step A7 / N in FIG. 2).
O), using the cluster variable {φ} newly stored in the cluster variable storage unit 124, the variable updating procedure from step A2 is repeated. If the convergence determination unit 116 determines that the convergence has occurred (step A7 / YES in FIG. 2), the process ends.

In this way, the optimized adaptive system is constructed.

Next, for the optimized adaptive system,
The case of implementing adaptation to a new speaker will be described in detail with reference to the drawings.

FIG. 3 is a block diagram showing the configuration of the first embodiment of the present invention when the adaptive system is used for a new speaker.

Referring to FIG. 3, the adaptive system for a new speaker according to the first embodiment of the present invention includes an input device 300 for inputting a new speaker's utterance and a data processing device 301 operated by program control. And a storage device 30 for storing information
2 and.

The storage device 302 includes a new speaker data storage unit 321 for storing input data of a new speaker, and a cluster weight variable storage unit 322 for representing a weight of belonging to a prior distribution cluster.
And an HMM variable storage unit 323 that constitutes the HMM of each speaker in the learning data, and a cluster variable storage unit 324 that represents the speaker clusters in all learning data as clusters of prior distribution and determines the structure of each cluster. Consists of.

The data processing device 301 includes an initial value setting unit 311 for setting initial values of variables for a new speaker, a cluster weight optimization unit 132 for the new speaker, and an HMM variable optimization for the new speaker. The conversion unit 313 and the convergence determination unit 314 that determines whether the cluster weight and the HMM variable have converged.

Next, the operation will be described with reference to the drawings.

FIG. 4 is a flowchart showing the operation of the first embodiment of the present invention when the adaptive system is used for a new speaker.

Referring to FIG. 4, first, the operator inputs new speaker data from the input device 300 to the new speaker data storage unit 32.
1 (step B0 in FIG. 4).

Next, the initial value setting section 311
The initial values of the cluster weight variables (q, numerical values for all clusters) and the initial values of the HMM variables (λ, a set of numerical values) are respectively input from the operator (from keyboard, recording medium, etc.), cluster weights. The variables are set in the variable storage unit 322 and the HMM variable storage unit 323 (step B1 in FIG. 4). In step B1, if the initial value setting unit 311 has no prior knowledge about the new speaker, the cluster weight variables are evenly distributed. If there is prior knowledge, it may be used as a cluster weight variable. Regarding the HMM variable, an unspecified HMM learned in advance is stored in the HMM variable storage unit 323 to be an initial value.

Next, the cluster weight variable optimization unit 312
Optimizes the weight variable q belonging to the new speaker cluster for the new speaker α in the cluster weight variable storage unit 322, and updates the contents of the cluster weight variable storage unit 322 (step B2 in FIG. 4).

Next, the HMM variable optimization unit 313 causes the HM
The HMM variable λ is optimized for the new speaker α in the M variable storage unit 323, and the contents of the HMM variable storage unit 323 are updated (step B3 in FIG. 4).

HMM variable optimization unit 313 in step B3
Then, the estimation formula based on the EM algorithm used in Reference 2 is used. Further, in this step, since the HMM variables other than the HMM variables are fixed, the HMM variable optimization formula in Chapter 2 of Reference 2 can be used as it is.

Next, the convergence determination unit 314 determines whether or not the cluster weight variable {q ^α } and the HMM variable λ ^α have converged for the new speaker α (step B in FIG. 4).
4) If it is determined that it has not converged (step B4 in FIG. 4).
/ NO), step B2 and step B3 are repeated.

If the convergence determination unit 314 determines that the convergence has occurred (step B4 / YES in FIG. 4), the process ends.

Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 2, according to the second embodiment of the present invention, the initial value of the cluster weight variable, the initial value of the HMM variable, and the structure of the speaker cluster in all learning data are set for each learning speaker. The initial values of the cluster variables to be decided are respectively set in the cluster weight variable storage unit 122, the HMM variable storage unit 123, and the cluster variable storage unit 124 (step A1 in FIG. 2), and the cluster weight variable storage unit 122 is described. Cluster weight variable optimization procedure for optimizing each speaker cluster weight variable for each speaker and updating the contents of the cluster weight variable storage unit 122 (step A2 in FIG. 2),
HM for optimizing HMM variables for the speaker in the HMM variable storage unit 123 and updating the contents of the HMM variable storage unit 123
With the M variable optimization procedure (step A3 in FIG. 2), it is determined whether the cluster weight variable and the HMM variable have converged for the learning speaker (step A4 in FIG. 2), and if it is determined that they have not converged, The process is returned to the cluster weight variable optimization procedure and the HMM variable optimization procedure (step A4 / NO in FIG. 2), and further it is checked whether or not all learning speakers have been processed (step A5 in FIG. 2). If not, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker (step A5 / NO in FIG. 2), the first convergence determination procedure, and When it is determined that the first convergence determination procedure has been processed for all the learned speakers (step A5 / YES in FIG. 2), the configuration of the speaker cluster for all the speakers in the cluster variable storage unit 124 is optimized. A cluster variable optimizing procedure of optimizing the cluster variables and updating the contents of the cluster variable storage unit 124 (step A6 in FIG. 2), determining whether the cluster variables have converged (step A7 in FIG. 2), and converging If not determined, the cluster weight variable optimization procedure and the HMM variable optimization procedure are performed so that optimization is performed using the cluster variables stored in the cluster variable storage unit 124 by the cluster variable optimization procedure. This is a method including a second convergence determination procedure for returning the processing (step A7 / NO in FIG. 2).

Next, a third embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 2, according to the third embodiment of the present invention, the initial value of the cluster weight variable, the initial value of the HMM variable, and the structure of the speaker cluster in all learning data are set for each learning speaker. The initial values of the cluster variables to be decided are respectively set in the cluster weight variable storage unit 122, the HMM variable storage unit 123, and the cluster variable storage unit 124 (step A1 in FIG. 2), and the cluster weight variable storage unit 122 is described. Cluster weight variable optimization procedure for optimizing each speaker cluster weight variable for each speaker and updating the contents of the cluster weight variable storage unit 122 (step A2 in FIG. 2),
HM for optimizing HMM variables for the speaker in the HMM variable storage unit 123 and updating the contents of the HMM variable storage unit 123
With the M variable optimization procedure (step A3 in FIG. 2), it is determined whether the cluster weight variable and the HMM variable have converged for the learning speaker (step A4 in FIG. 2), and if it is determined that they have not converged, The process is returned to the cluster weight variable optimization procedure and the HMM variable optimization procedure (step A4 / NO in FIG. 2), and further it is checked whether or not all learning speakers have been processed (step A5 in FIG. 2). If not, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker (step A5 / NO in FIG. 2), the first convergence determination procedure, and When it is determined that the first convergence determination procedure has been processed for all the learned speakers (step A5 / YES in FIG. 2), the configuration of the speaker cluster for all the speakers in the cluster variable storage unit 124 is optimized. A cluster variable optimizing procedure of optimizing the cluster variables and updating the contents of the cluster variable storage unit 124 (step A6 in FIG. 2), determining whether the cluster variables have converged (step A7 in FIG. 2), and converging If not determined, the cluster weight variable optimization procedure and the HMM variable optimization procedure are performed so that optimization is performed using the cluster variables stored in the cluster variable storage unit 124 by the cluster variable optimization procedure. It is a program that causes the computer (for example, the data processing device 101) to execute the second convergence determination procedure for returning the process (step A7 / NO in FIG. 2).

[0093]

The first effect is that a clustered prior distribution based on speaker characteristics can be constructed from a large-scale database.

The reason is that the variable optimization of each learning speaker and the global cluster variable optimization by all data are repeatedly executed hierarchically.

The second effect is that sufficient speaker adaptation can be achieved by simply determining cluster weight variables and HMM variables with a small number of utterances during speaker adaptation.

The reason is that a clustered prior distribution based on speaker characteristics is constructed from a large-scale database during learning, and HMM adaptation to a new speaker is equivalent to variable optimization of each learning speaker. , Because the prior distribution cluster that is effective at the time of adaptation can be used.

The third effect is that the recognition unit of the acoustic model, which was not present in the adaptation data when the speaker was adapted to the new speaker, can be flexibly adapted.

The reason is that the speaker cluster weight is optimized by using the recognition unit existing in the adaptation data, and thus the speaker prioritization can be effectively performed by the weighted prior distribution cluster. is there.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of the first exemplary embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration for adapting an acoustic model for a new speaker according to the first embodiment of this invention.

FIG. 4 is a flowchart showing an operation of adapting an acoustic model for a new speaker according to the first embodiment of this invention.

FIG. 5 is an explanatory diagram showing mathematical expressions.

FIG. 6 is a block diagram showing a configuration of an HMM estimation method based on a conventional MAP estimation framework.

FIG. 7 is a block diagram showing a configuration of an HMM adaptation method based on a conventional MAP estimation framework.

[Explanation of symbols]

101 Data processing device 301 Data processing device 501 data processor 601 Data processing device 102 storage device 302 storage device 502 storage device 602 storage device 300 input device 600 input device 121 learning data storage unit 321 New speaker data storage 521 Learning data storage unit 621 New speaker data storage unit 122 cluster weight variable storage unit 322 cluster weight variable storage unit 522 cluster weight variable storage unit 622 cluster weight variable storage unit 123 HMM variable storage 323 HMM variable storage 523 HMM variable storage 623 HMM variable storage 124 cluster variable storage 324 cluster variable storage 524 cluster variable storage 624 cluster variable storage 111 Initial value setting section 311 Initial value setting section 112 cluster weight variable optimization unit 312 cluster weight variable optimization unit 113 HMM variable optimization unit 313 HMM variable optimization unit 511 HMM variable optimization unit 611 HMM variable optimization unit 115 Cluster Variable Optimization Unit 114 Convergence determination unit 116 Convergence determination unit 314 Convergence determination unit 512 Convergence determination unit 612 Convergence determination unit

Claims (15)

[Claims] 1. A learning data storage unit for storing learning data, and reading the learning data from the learning data storage unit, optimizing a clustered prior distribution based on the read learning data, and optimizing the prior distribution. A speech recognition acoustic model learning / adaptation system comprising: an optimization means for optimizing an HMM variable based on the above. 2. A learning data storage unit that stores learning data, a cluster variable storage unit that stores variables that determine a clustered prior distribution, a cluster weight storage unit that stores cluster weight variables, and an acoustic model is determined. HMM variable storage unit for storing HMM variables, cluster weight variable optimizing means for optimizing cluster weight variables for each learning data, and HMM for optimizing HMM variables for each learning data
A speech recognition acoustic model learning / adaptive system comprising a variable optimization means and a cluster variable optimization means for optimizing a cluster variable for all learning data. 3. A means for optimizing a cluster weight variable and adapting an HMM variable using a prior distribution expressed as a linear combination of speaker clusters when adapting an acoustic model to a new speaker. Speech recognition acoustic model learning and adaptation system. 4. A new speaker data storage unit for storing new speaker adaptation data, a cluster variable storage unit for storing a cluster variable for determining a prior distribution expressed as a linear combination of speaker clusters, and a cluster. A cluster weight variable storage unit that stores the weight variable, an HMM variable storage unit that stores the HMM variable that determines the acoustic model, and optimizes the weight variable to the new speaker cluster based on the adaptation data of the new speaker. A speech recognition acoustic model learning / adapting system comprising: a cluster weight variable optimizing means; and an HMM variable optimizing means for optimizing an HMM variable based on new speaker adaptation data. 5. The learning data is read from the learning data storage unit that stores the learning data, the clustered prior distribution is optimized based on the read learning data, and the HMM variable is calculated based on the optimized prior distribution. A method for learning and adapting a voice recognition acoustic model, characterized by including an optimization procedure for optimizing. 6. A cluster weight variable optimization procedure for optimizing a cluster weight variable of a cluster weight storage unit for each learning data of a learning data storage unit, and an HMM for optimizing an HMM variable of an HMM variable storage unit for each learning data. A speech recognition acoustic model learning / adaptation method comprising: a variable optimization procedure; and a cluster variable optimization procedure for optimizing a cluster variable in a cluster variable storage unit for all learning data. 7. A method of optimizing a cluster weight variable and adapting an HMM variable by using a prior distribution expressed as a linear combination of speaker clusters when adapting an acoustic model to a new speaker. Speech recognition acoustic model learning and adaptation method. 8. A cluster weight variable optimization procedure for optimizing a weight variable for a cluster of a new speaker based on the adaptation data of the new speaker in the new speaker data storage unit, and a new procedure for the new speaker data storage unit. HMM variable optimization procedure for optimizing HMM variables based on speaker adaptation data. 9. The learning data is read from the learning data storage unit that stores the learning data, the clustered prior distribution is optimized based on the read learning data, and the HMM variable is calculated based on the optimized prior distribution. A speech recognition acoustic model learning / adapting program characterized by causing a computer to execute an optimizing procedure for optimizing. 10. A cluster weight variable optimization procedure for optimizing a cluster weight variable in a cluster weight storage unit for each learning data in the learning data storage unit, and an HMM for each learning data.
It is characterized by causing a computer to execute an HMM variable optimization procedure for optimizing HMM variables in a variable storage section and a cluster variable optimization procedure for optimizing cluster variables in a cluster variable storage section for all learning data. Speech recognition acoustic model learning / adaptation program. 11. A method of optimizing a cluster weight variable and causing a computer to perform a procedure of adapting an HMM variable by using a prior distribution expressed as a linear combination of speaker clusters when adapting an acoustic model to a new speaker. A speech recognition acoustic model learning / adaptation program characterized by. 12. A cluster weight variable optimization procedure for optimizing a weight variable to a cluster of a new speaker based on the adaptation data of the new speaker in the new speaker data storage unit, and a new procedure for the new speaker data storage unit. A speech recognition acoustic model learning / adaptation program for causing a computer to execute an HMM variable optimizing procedure for optimizing an HMM variable based on speaker adaptation data. 13. A cluster weight variable storage unit, an initial value of a cluster weight variable, an initial value of an HMM variable, and an initial value of a cluster variable that determines a structure of a speaker cluster in all learning data for each learning speaker, respectively. HMM variable storage,
An initial value setting unit to be set in the cluster variable storage unit, and a cluster weight variable optimization unit to optimize each speaker cluster weight variable for the speakers in the cluster weight variable storage unit and update the contents of the cluster weight variable storage unit And optimize the HMM variables for the speaker in the HMM variable storage unit,
An HMM variable optimization unit that updates the contents of the MM variable storage unit, a cluster weight variable, and H for the learning speaker.
If it is determined whether or not the MM variable has converged, and it is determined that the MM variable has not converged, the cluster weight variable optimization unit, the H
The process is returned to the MM variable optimizing unit, whether or not all the learning speakers have been processed is checked, and if it is determined that they have not been processed, the cluster weight variable optimizing unit for the next learning speaker, A first convergence determination unit that returns the processing to the HMM variable optimization unit, and when it is determined that the first convergence determination unit has processed all the learning speakers, the cluster variable storage unit talks to all the speakers. Cluster variables are optimized so that the cluster configuration is optimized, and a cluster variable optimization unit that updates the contents of the cluster variable storage unit and a judgment whether the cluster variables have converged. If the determination is made, the cluster weight variable optimization unit and the H variable optimization unit perform optimization by using the cluster variables stored in the cluster variable storage unit by the cluster variable optimization unit.
A speech recognition acoustic model learning / adaptation system, comprising: a second convergence determination unit that returns processing to the MM variable optimization unit. 14. An initial value of a cluster weight variable, an initial value of an HMM variable, and an initial value of a cluster variable that determines a speaker cluster structure in all learning data for each learning speaker, respectively. HMM variable storage,
An initial value setting procedure to be set in the cluster variable storage unit, and a cluster weight variable optimization procedure for optimizing each speaker cluster weight variable for the speaker in the cluster weight variable storage unit and updating the contents of the cluster weight variable storage unit And the HM
The HMM variable optimization procedure for optimizing the HMM variables for the speaker in the M variable storage unit and updating the contents of the HMM variable storage unit, and for the learning speaker, the cluster weight variable and whether the HMM variable has converged If it is determined that it has not converged, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure, and
If it is determined that all the learning speakers have not been processed, and if it is determined that they have not been processed, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker. And the first convergence determination procedure is performed for all the learning speakers, cluster variables for all speakers in the cluster variable storage unit are optimized so that the configuration of the speaker cluster is optimized. A cluster variable optimizing procedure of performing optimization and updating the contents of the cluster variable storage section, and determining whether or not the cluster variables have converged, and determining that they have not converged, the cluster variable optimizing procedure Second convergence returning processing to the cluster weight variable optimization procedure and the HMM variable optimization procedure so that optimization is performed using the cluster variables stored in the cluster variable storage unit. A method for learning and adapting a speech recognition acoustic model, characterized by including a decision procedure. 15. An initial value of a cluster weight variable, an initial value of an HMM variable, and an initial value of a cluster variable that determines the structure of a speaker cluster in all learning data for each learning speaker, respectively. HMM variable storage,
An initial value setting procedure to be set in the cluster variable storage unit, and a cluster weight variable optimization procedure for optimizing each speaker cluster weight variable for the speaker in the cluster weight variable storage unit and updating the contents of the cluster weight variable storage unit And the HM
The HMM variable optimization procedure for optimizing the HMM variables for the speaker in the M variable storage unit and updating the contents of the HMM variable storage unit, and for the learning speaker, the cluster weight variable and whether the HMM variable has converged If it is determined that it has not converged, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure, and
If it is determined that all the learning speakers have not been processed, and if it is determined that they have not been processed, the process returns to the cluster weight variable optimization procedure and the HMM variable optimization procedure for the next learning speaker. And the first convergence determination procedure is performed for all the learning speakers, cluster variables for all speakers in the cluster variable storage unit are optimized so that the configuration of the speaker cluster is optimized. A cluster variable optimizing procedure of performing optimization and updating the contents of the cluster variable storage section, and determining whether or not the cluster variables have converged, and determining that they have not converged, the cluster variable optimizing procedure Second convergence returning processing to the cluster weight variable optimization procedure and the HMM variable optimization procedure so that optimization is performed using the cluster variables stored in the cluster variable storage unit. A speech recognition acoustic model learning / adaptation program characterized by causing a computer to execute a judgment procedure.