JP2003177785A - Linear transformation matrix calculation device and voice recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice recognition device having a higher performance for derivation of an LDA transformation matrix. <P>SOLUTION: When a linear transformation matrix for linear discrimination analysis is obtained, an inter-class covariance matrix is calculated. At this time, confusion matrix coefficients are multiplied. The confusion matrix coefficients are determined in accordance with frequencies of combination of pairs of phonemes, pairs of syllables or the like easy to be confused by previous voice recognition. With respect to the linear transformation matrix calculated in this process, feature vectors can be subjected to such transformation that classes easy to be confused can be more separated. As a result, a high recognition performance is obtained. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an LDA conversion matrix calculation device and a voice recognition device, a voice recognition system for an unspecified speaker, a voice recognition method, and a recording medium recording a voice recognition program. In particular, the present invention relates to a method for creating a linear transformation matrix for linear discriminant analysis to improve discriminability, a program in which the method is described, and a recording medium recording the program. The present invention also relates to a voice recognition system using the linear conversion matrix, a voice recognition program, and a recording medium recording the program.

[0002]

2. Description of the Related Art Conventionally, as one of techniques for improving the performance of speech recognition, page 115 of "Pattern Recognition" (hereinafter referred to as Reference 1) written by Kenichiro Ishii published by Ohmsha. To Linear Discriminant Analysis (hereinafter referred to as L
(Referred to as DA) is known. LDA is a method of performing a linear transformation so that the variance of the feature amount within a class is small and the variance of the feature amount between classes is large, and performing a linear transform on the feature vector space (hereinafter, such a transform is performed by LDA. Referred to as conversion). In many cases, LDA conversion in speech recognition is applied to a multidimensional feature amount, and in that case, the LD
The operator for A conversion is a matrix (hereinafter, such a matrix for LDA conversion is referred to as an LDA conversion matrix).
Feature vectors in speech recognition were introduced in NT in 1995.
Mel cepstrum and its variation are used, as shown on pages 134 to 142 of the basics of speech recognition (above) published by T Advanced Technology Co., Ltd. (above) (above). . Also, in speech recognition, to represent the equivalent of the above class, 1
Hidden Markov Mod (Hidden Markov Mod) as shown on page 102 to page 138 of the basics of speech recognition (below) referred to by Director Furui, published by NTT Advanced Technology Co., Ltd.
el: hereinafter referred to as HMM), the output distribution, etc. As a result of such LDA conversion, the degree of separation between classes is larger than in the space in which the original feature vector is distributed. For example, when the symbol representing the i-th class and omega _1, if the feature vector closest to the class omega ₁ is input, when the likelihood of other classes (or distance) is not performed LDA transformation It becomes larger (smaller for distance). As a result, it can be expected that the discriminating ability is enhanced and the recognition performance is also enhanced.

An example of a conventional LDA conversion matrix calculation device for obtaining an LDA conversion matrix using a conventional technique and a speech recognition device using the LDA conversion matrix will be described with reference to the drawings and mathematical formulas. Figure 4
FIG. 6 is a diagram for explaining a conventional LDA conversion matrix calculation device 3 using a conventional technique. The conventional LDA conversion matrix calculation device 3 includes an interclass covariance matrix calculation unit C002, an intraclass covariance matrix calculation unit C003, and a conversion matrix calculation unit C00.
It is composed of 4. In-class covariance matrix calculation unit C002
Calculates the in-class covariance matrix Σ _W using the learning feature vector x and the mathematical expression shown in the following Expression 5 (described in Reference Document 1) and outputs it. In the formula shown in the above equation 5, ω ₁ is the i-th class, c is the number of classes, μ ₁
Is the average value of the feature vector for each class, N _i is the class ω ₁
Shows the number of data of.

[0004]

[Equation 5]

[0005] x∈ω _1, out of the feature vector x, ω ₁
Indicates that it belongs to. P (ω ₁ ) is the prior probability that the class ω ₁ appears, and as shown on page 128 of Reference 1, a value depending on the number of data or a uniform value may be set regardless of the class. Many. The mathematical expression shown in the above equation 5 calculates the sum of the mean squares of the differences of the feature vectors belonging to each class (in-class variance) from μ _i obtained by averaging the feature vectors x belonging to each class. . The denser the feature vectors included in the class, that is, Σ
_The smaller the value of _W, the higher the degree of belonging to the class, which seems to be advantageous for recognition performance. The inter-class covariance matrix calculation unit C002 calculates the inter-class covariance matrix by using the learning feature vector x and the mathematical formula (described in Reference 1) shown in the following Expression 6. Output.

[0006]

[Equation 6]

The mathematical formula shown in the above equation 6 is a special feature for each class.
Mean value of characteristic vector x₁Calculate the sum of squared distances for each
There is. The conversion matrix calculation unit C003^TΣ_BA is big
A, ^TΣ_WThe LDA conversion matrix A is set so that A becomes small.
Ask.

As a method for obtaining such an LDA conversion matrix A, it is conceivable to maximize the mathematical formula as shown in the above equation 5, for example. The analysis device C001 for calculating the learning feature vector input to the conventional LDA conversion matrix calculation device 3 from the learning speech signal S will be described.
The analysis device C001 cuts out the audio signal S at regular time intervals (hereinafter, referred to as a frame), calculates a time series of the feature vector x composed of the mel cepstrum and its variation, and outputs the calculated time series. Here, a learning voice signal S collected in advance for learning is used as a voice signal (hereinafter, a feature vector obtained by analyzing the learning voice signal S is referred to as a learning feature vector x).

Next, the calculation operation of the conventional LDA conversion matrix calculation device 3 will be described. The analysis unit C001 converts all of the learning voice signal S into the learning feature vector x. The learning feature vector x is input to the conventional LDA conversion matrix calculation device 3. Using all learning feature vectors x,
The in-class covariance matrix calculation unit C003 calculates the in-class covariance matrix Σ _W according to the mathematical formula shown in the above equation 5.
Using all the learning feature vectors x, the interclass covariance matrix calculation unit C002 calculates the interclass covariance matrix Σ _B according to the mathematical formula shown in the above equation 5. Transformation matrix calculation unit C
004 is such that A ^T Σ _B A is large and A ^T Σ
_The LDA conversion matrix A is obtained according to the mathematical formula shown in the following Expression 7 so that _W A becomes small.

[0010]

[Equation 7]

FIG. 5 is a diagram for explaining a conventional speech recognition apparatus using the LDA conversion matrix A. The conventional speech recognition apparatus includes an analysis apparatus C001 and an LDA conversion matrix storage unit C
005, an HMM storage unit C008, a recognition unit C007, a feature vector matrix multiplier C006, and an HMM matrix multiplier C
009. LDA conversion matrix storage unit C00
5 is the LD calculated by the conventional LDA conversion matrix calculation device
The A conversion matrix A is stored. The HMM storage unit C008 stores H
Store the MM. Various forms can be selected for the HMM, such as a state series as a recognition unit such as a phoneme or a syllable.

The form of the distribution can be either a discrete distribution or a continuous distribution, and here a continuous normal distribution is used. At this time, the class can be represented by using an average vector and its corresponding variance. The feature vector matrix multiplier C006 uses the LDA conversion matrix A
And the feature vector x are multiplied. HMM matrix multiplier C
007 multiplies the LDA conversion matrix A and the HMM mean vector, variance, and the like. The recognition unit C007 calculates the likelihood of the feature vector x and the HMM, and outputs the one having the highest likelihood as the recognition result. As a method of collating the HMM, the Viterbi algorithm shown on pages 125 to 128 of Reference 3 can be considered.

Next, the operation of the conventional speech recognition apparatus using the LDA conversion matrix A will be described. The analyzer C001
The input voice signal S is analyzed at fixed time intervals, and the feature vector x is output. At this time, the voice signal is not a voice signal for learning but a voice signal for recognition. L stored in the LDA conversion matrix storage unit C005 by the feature vector matrix multiplier C006.
The DA conversion matrix A and the feature vector x for each frame (hereinafter,
This feature vector is referred to as a transformed feature vector x ′). The HMM matrix multiplier C009 allows the HM
The LDA conversion matrix A is applied to the average vector of the HMMs and the variance corresponding thereto stored in the M storage unit C008 (hereinafter, this HMM is referred to as a converted HMM). The recognition unit C007 collates the converted feature vector x ′ with the converted HMM, and outputs the one with high likelihood as a recognition result.

[0014]

In the conventional LDA conversion matrix calculation device using the conventional technique, it seems that the separability of each class in the feature space is improved by increasing the interclass variance. . On the other hand, in the voice recognition, since the likelihood is used as a reference, even if the separation degree is improved by using the mathematical formula shown in the above Expression 6, the recognition performance does not always improve. The prior probabilities P (ω ₁ ) in the class ω ₁ can be specified by the mathematical formulas shown in the above formulas 5 and 6, but they are independent appearance probabilities for each class and do not directly contribute to the recognition result, so the performance There is no guarantee that will increase. Although there is a conventional technique that aims at improving the speech recognition performance by performing linear discriminative learning, there is a problem in the degree of improvement in the speech recognition performance.

An object of the present invention is to provide a confusion matrix coefficient rooted in the recognition result in the calculation of the interclass variance in the derivation of the LDA transformation matrix, so that L which is more directly connected to the recognition result.
It is to provide a speech recognition device with higher performance by obtaining a DA conversion matrix.

[0016]

An LDA conversion matrix calculation device according to the present invention includes a confusion matrix coefficient storage unit that stores phoneme pairs, syllable pairs and the like that are easily confused as confusion matrix coefficients.
Shows the weighted inter-class covariance matrix calculation unit that finds the weighted inter-class covariance matrix that indicates the degree of separation between classes from the learning feature vectors, etc., taking into consideration easily confused phoneme pairs, etc. An intra-class covariance matrix calculation unit that finds the inter-class covariance matrix from the learning feature vectors, etc., and a weighted transformation matrix that finds the matrix for linear transformation using the weighted inter-class covariance matrix and the intra-class covariance matrix The calculator is provided. Further, the confusion matrix coefficient storage unit stores, as confusion matrix coefficients, phoneme pairs, syllable pairs, and the like that are easily confused and are created from pairs of previously recognized results and corresponding teacher data.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings and mathematical expressions. FIG. 1 shows the first aspect of the invention.
FIG. 3 is a diagram for explaining the embodiment of 1. The LDA conversion matrix calculation device 1 according to the first exemplary embodiment of the present invention includes a confusion matrix coefficient storage unit A001, a weighted interclass covariance matrix calculation unit A002, an intraclass covariance matrix calculation unit C003, and a weighted conversion matrix calculation unit. It is composed of part A003. The in-class covariance matrix calculation unit C008 is the same as the conventional one.

The confusion matrix coefficient storage unit A001 has a class ω
confusion matrix coefficient indicating the degree _j and ω _i is confused P (ω _i,
ω _j ) is stored. The confusion matrix coefficient P (ω _i , ω _j ) is
Set a value that emphasizes the relationships between classes that are easily confused.

An example of a method of setting the confusion matrix coefficient P (ω _i , ω _j ) will be described. For example, ω ₁ = “yes”, ω ₂
Assume a class of three syllables, such as = “ki”, ω ₃ = “shi”. Usually, P (ω _i , ω _j ) = for all combinations
Like α, it is set to a constant value such as α. Empirically, if you think that “ki” and “shi” are often confused,
P (ω ₃ = “shi”, ω ₂ = “ki”) = α ± α ₂₃ .

Here, the sign of ± is set appropriately according to the distance scale of recognition and the like. The value of α ₂₃ is "ki" and "shi"
May be empirically determined according to the degree of confusion, or may be calculated from the confusion frequency of the previously recognized result. Further, α ₂₃ = α ₃₂ may be set, or α ₂₃ ≠ α ₃₂ may be set.

Weighted interclass covariance matrix calculation unit A00
2 uses the preset confusion matrix coefficient P (ω _i , ω _j ) and the learning feature vector x to calculate the weighted interclass covariance matrix Σ _B 'according to the mathematical formula shown in the above-mentioned equation 1. . The weighted transformation matrix calculation unit A003 uses the weighted inter-class covariance matrix Σ _B ′ and the intra-class covariance matrix Σ _W according to the mathematical formula shown in the above-mentioned Equation 2
Compute the A matrix A '.

The mathematical formula shown in the above equation 2 is an example,
A '^TΣ_BMaximize'A ', A'^TΣ _WMinimize A '
What is an evaluation function that can obtain A '
It can be something like this.

The operation of the first embodiment of the present invention will be described below. The feature vector x in which the learning voice signal S is analyzed by the analysis unit device C001 or the like is input to the weighted inter-class covariance matrix calculation unit A002 and the intra-class covariance matrix calculation unit C003. The in-class covariance matrix calculation unit C003 uses the learning feature vector x
The in-class covariance matrix Σ _W is calculated according to the formula shown in.

Weighted interclass covariance matrix calculation unit A00
2 is shown in Equation 1 above using the learning feature vector x.
Weighted interclass covariance matrix Σ_B'Calculate
And output. The weighted transformation matrix calculation unit A003
A '^TΣ_B"A" is big, A "^TΣ _WA'becomes smaller
Then, the weighted transformation matrix A ′ is obtained and output.

FIG. 2 is a diagram for explaining the second embodiment of the present invention. Second Embodiment, which is the second embodiment of the present invention
The LDA conversion matrix calculation device 2 is composed of an LDA conversion matrix calculation device 1, a replacement frequency counting unit A004, a recognition result storage unit A005, and a teacher data storage unit A006. The recognition result storage unit A005 is a result of recognition of a voice by an arbitrary voice recognition device or the like in advance.
Is remembered.

The recognition result R may be a phoneme unit, a syllable unit, or any expression form such as a word or a sentence. The teacher data storage unit A006 stores the teacher data V that is the correct answer corresponding to the recognition result R. The teacher data V needs to correspond to the recognition result R on a one-to-one basis.

The recognition result R and the teacher data V may be collected from any utterance. Replacement frequency counting section A
004, using the recognition result R and the teacher data V, calculates the degree of confusion for each class pair represented by a state or distribution such as HMM, and determines the degree of confusion by the confusion matrix coefficient P (ω _i ,
output as ω _j ). The LDA conversion matrix calculation unit 1 outputs P (ω _i , ω) output by the replacement frequency aggregation unit A004.
_j ) and the learning feature vector x, and weighted LD
The A conversion matrix A ′ is calculated and output.

FIG. 3 is a diagram for explaining the third embodiment of the present invention. The speech recognition apparatus according to the third embodiment of the present invention includes an LDA conversion matrix calculation apparatus 1, an analysis apparatus C001, an HMM storage unit C008, a recognition unit C007, a feature vector matrix multiplier C006, and an HMM matrix multiplier C009. It is composed of. Analyzer C001 and HMM
The storage unit C008, the recognition unit C007, the feature vector matrix multiplier C006, and the HMM matrix multiplier C009 have the same functions as conventional ones. The LDA conversion matrix calculation device 1 has a first
This is the same as the embodiment.

The operation of this speech recognition apparatus will be described below. The LDA conversion matrix calculation device 1 outputs the weighted LDA conversion matrix A ′ calculated using the learning feature vector. The analyzer C001 analyzes the audio signal S for each frame and outputs the feature vector x. The feature vector matrix multiplier C006 calculates the feature vector x and the weighted LDA.
The conversion matrix A ′ is multiplied. HMM matrix multiplier C00
9 multiplies the HMM stored in the HMM storage unit C008 by the weighted LDA conversion matrix A ′. Recognition unit C0
07 collates the feature vector x multiplied by the weighted LDA conversion matrix A ′ with the HMM multiplied by the weighted LDA conversion matrix A ′, and outputs the one having the highest likelihood as the recognition result.

[0030]

According to the present invention, the confusion matrix coefficient used in obtaining the interclass distance is empirically or from the recognition result,
It was given as a confusion matrix coefficient that emphasizes phoneme pairs that are difficult to recognize and easily confused, and was linked to the calculation of the LDA conversion matrix. Therefore, since it is possible to emphasize and separate phoneme pairs that are originally difficult to recognize and are easily confused, it is possible to expect improvement in recognition performance.

[Brief description of drawings]

FIG. 1 is a diagram showing a first exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a second exemplary embodiment of the present invention.

FIG. 3 is a diagram showing a third exemplary embodiment of the present invention.

FIG. 4 is a diagram showing a linear transformation matrix calculation device using a conventional technique.

FIG. 5 is a diagram for explaining a speech recognition device using a linear transformation matrix using a conventional technique.

[Explanation of symbols]

1 LDA conversion matrix calculation device 2 2nd LDA conversion matrix calculation device 3 Conventional LDA conversion matrix calculation device C001 Analysis device C002 Interclass covariance matrix calculation unit C003 In-class covariance matrix calculation unit C004 Conversion matrix calculation unit C005 LDA conversion matrix storage unit Part C006 Feature vector matrix multiplier C007 Recognition part C008 HMM storage part C009 HMM matrix multiplier A001 Confusion matrix coefficient storage part A002 Weighted interclass covariance matrix calculation part A003 Weighted conversion matrix calculation part A004 Swap frequency calculation part A005 the recognition result storage unit A006 teacher data storage unit S phonetic x feature vectors A LDA transformation matrix A 'weighted LDA transformation matrix sigma _W intraclass covariance matrix sigma _B interclass covariance matrix sigma _B' weighted inter-class covariance matrix P (ω _i , ω _j ) Confusion matrix coefficient

Claims (8)

[Claims] 1. A confusion matrix coefficient storage unit that stores a phoneme pair, a syllable pair, and the like that are easily confused as a confusion matrix coefficient, and a weighted inter-class covariance matrix that uses the confusion matrix coefficient and a learning feature vector. Inter-class covariance matrix calculation unit, intra-class covariance matrix calculation unit that obtains the intra-class covariance matrix from the learning feature vector, and linear transformation using the weighted inter-class covariance matrix and the intra-class covariance matrix A linear transformation matrix calculation device comprising a weighted transformation matrix calculation unit for obtaining a weighted linear transformation matrix for. 2. The confusion matrix coefficient storage unit stores a confusion matrix coefficient for emphasizing a phoneme pair, a syllable pair, or the like, which is empirically and intuitively determined and is likely to be confused. Linear transformation matrix calculator. 3. The confusion matrix coefficient storage unit stores, as a confusion matrix coefficient, a phoneme pair, a syllable pair, or the like that is easily confused and is created from a pair of a previously recognized result and corresponding teacher data. Item 1. A linear conversion matrix calculation device according to item 1. 4. The weighted inter-class covariance matrix calculation unit uses the confusion matrix coefficient P (ω _i , ω _j ) and the learning feature vector (x) according to an equation shown in the following Expression 1. , The weighted interclass covariance matrix (Σ _B ') is calculated, and The weighted transformation matrix calculation unit uses the weighted inter-class covariance matrix (Σ _B ′) and the intra-class covariance matrix (Σ _W ) according to the mathematical formula shown in the following Equation 2.
Compute the weighted linear transformation matrix (A ′) The linear conversion matrix calculation device according to any one of claims 1 to 3, characterized in that. 5. A confusion matrix coefficient storage unit for storing a phoneme pair, a syllable pair, etc., which are easily confused as a confusion matrix coefficient, and a weighted interclass covariance matrix for obtaining a weighted interclass covariance matrix using the confusion matrix coefficient and a learning feature vector. An interclass covariance matrix calculation unit, an intraclass covariance matrix calculation unit that obtains an intraclass covariance matrix from a learning feature vector, and a linear transformation using the weighted interclass covariance matrix and the intraclass covariance matrix An LDA conversion matrix calculation apparatus having a weighted conversion matrix calculation unit for obtaining a weighted linear conversion matrix for the following, an analysis apparatus for analyzing a speech signal for learning to obtain a feature vector, the feature vector and the weighted linear A matrix multiplier for a feature vector that multiplies a transformation matrix and outputs a transformed feature vector, and a Hidden Markov model (hereinafter referred to as HMM) Matrix transformation for HMM that outputs the transformed HMM and the transformed feature vector and the transformed HMM, and the one with the highest likelihood is used as the recognition result. A voice recognition device comprising a recognition unit for outputting. 6. A confusion matrix coefficient storage unit for storing a confusion matrix coefficient for emphasizing a phoneme pair, a syllable pair, or the like which is empirically and intuitively determined and which is easily confused, and the confusion matrix coefficient and the learning feature vector. A weighted inter-class covariance matrix calculation unit for obtaining a weighted inter-class covariance matrix, an intra-class covariance matrix calculation unit for obtaining an intra-class covariance matrix from a learning feature vector, and the weighted inter-class covariance matrix An LDA transformation matrix calculation device having a weighted transformation matrix calculation unit for obtaining a weighted linear transformation matrix for linear transformation using the in-class covariance matrix, and an analysis for obtaining a feature vector by analyzing a speech signal for learning An apparatus, a matrix multiplier for a feature vector that outputs a transformed feature vector by multiplying the feature vector by the weighted linear transformation matrix, and Hidden Markov Moh. The HM converted by performing multiplication of Dell (hereinafter referred to as HMM) and the weighted linear conversion matrix.
An HMM matrix multiplier for outputting M, and a recognition unit for collating the converted feature vector with the converted HMM and outputting the one having the highest likelihood as a recognition result. Speech recognizer. 7. A confusion matrix coefficient storage unit for storing, as a confusion matrix coefficient, a phoneme pair, a syllable pair, etc., which are easily confused and which are created from a pair of preliminarily recognized results and corresponding teacher data, and a confusion matrix coefficient and for learning. A weighted inter-class covariance matrix calculation unit that obtains a weighted inter-class covariance matrix using a feature vector, an intra-class covariance matrix calculation unit that obtains an intra-class covariance matrix from the learning feature vector, and the weighted class An LDA conversion matrix calculation device having a weighted conversion matrix calculation unit for obtaining a weighted linear conversion matrix for linear conversion using the inter-covariance matrix and the intra-class covariance matrix, and analyzing a speech signal for learning. An analyzing device for obtaining a feature vector, and a feature vector matrix for multiplying the feature vector by the weighted linear transformation matrix and outputting the transformed feature vector. And adder performs multiplication of the HMM and the weighted linear transformation matrix,
An HMM matrix multiplier that outputs a converted HMM, and an authentication unit that collates the converted feature vector with the converted HMM and outputs the one having the highest likelihood as a recognition result. Characteristic voice recognition device. 8. The weighted inter-class covariance matrix calculation unit uses the confusion matrix coefficient P (ω _i , ω _j ) and the learning feature vector (x) in accordance with the mathematical formula shown below. , The weighted interclass covariance matrix (Σ _B ') is calculated, and The weighted transformation matrix calculation unit uses the weighted inter-class covariance matrix (Σ _B ′) and the intra-class covariance matrix (Σ _W ) according to the mathematical expression shown in the following Equation 4.
The weighted linear transformation matrix (A ′) is calculated. [Equation 4] The voice recognition device according to claim 5, wherein the voice recognition device is a voice recognition device.