JP2008309945A - Pattern matching method and device, and its feature amount normalizing method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pattern matching method and a device suitable for collating a feature amount including a vector quantization (VQ) distortion after vector quantization and its decoding, to its probability model; and to provide its normalizing method and device suitable for the feature amount. <P>SOLUTION: A mean-variance calculation section 51 calculates a mean y and a variance x1 of an acoustic feature amount (a feature vector) decoded by a VQ decoding section 21. A calculation result of the mean y is subtracted from the sound feature amount in an adder section 52. The variance x1 is added to a variance x2 of the VQ distortion stored beforehand in a VQ distortion variance storage section 27, in an adder section 53, and the variance x with the VQ distortion taken into consideration is calculated. In a normalizing section 54, the sound feature amount is normalized by using the variance x, and the sound feature amount after mean and variance normalization (MVN) is calculated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a pattern matching method and apparatus and a feature quantity normalization method and apparatus thereof, and more particularly to a probability model obtained by decoding a vector quantity once extracted from an input signal. And a feature amount normalizing method and apparatus suitable for normalizing the feature amount.

  In pattern matching used in the fields of information retrieval, voice, and image recognition, the similarity between an unknown input pattern and many known standard patterns is calculated, and the standard pattern with the highest similarity is used as the pattern matching result. . At this time, both the input pattern to be compared and the standard pattern are normalized, and uncertain elements such as disturbance are removed in advance.

  In the speech recognition apparatus to which this pattern matching is applied, a time-series acoustic feature amount extracted from an input speech signal is converted into an acoustic model in which a probability density distribution in an acoustic feature amount space is learned in advance in units of phonemes such as vowels and consonants. The recognition result is obtained by collating with. The acoustic model, which is a probabilistic model, outputs a score (acoustic likelihood) of the phoneme-likeness with respect to the input of the acoustic feature amount. The speech recognition device accumulates phoneme-likeness scores (acoustic likelihood) over the entire utterance according to the restrictions of the grammar and the word dictionary, and outputs a word sequence having the highest accumulated score as a recognition result.

  Here, the acoustic feature amount is time-series data of multidimensional vectors, and when the frequency distribution of data corresponding to each phoneme in each dimension is aggregated, it becomes a shape close to a normal distribution or a shape close to the sum of a plurality of normal distributions. . In order to express such a distribution of acoustic features, the probability density distribution of the acoustic model is represented by a multidimensional normal distribution or a plurality of multidimensional normal distributions.

  However, in actual matching, there is a gap between the distribution of input acoustic features and the probability density distribution of the acoustic model due to variations in microphone characteristics, speaker differences, background noise, etc. It becomes.

  In pattern matching between input acoustic features and acoustic models, a method called Cepstral Mean Normalization (CMN) has been widely used as a method to eliminate such deviations, and CMN has been further developed. Mean value and variance normalization (MVN) has been proposed as a method.

  CMN is the difference between the input acoustic feature distribution and the probability density distribution of the acoustic model by subtracting the average value of the entire utterance from the acoustic feature quantity at each time of utterance to zero the average acoustic feature. This is a technique for reducing the shift by aligning them. When the acoustic feature quantity of each dimension before CMN is x (t) and the acoustic feature quantity after CMN is x ′ (t), the operation of CMN is expressed by the following equation (1).

  MVN, on the other hand, normalizes the acoustic feature value at each time of utterance with the average value and variance of the entire utterance and aligns it with the normal distribution N (average 0, variance 1) of the reference system, so that the microphone characteristics, etc. This is a technique for reducing the difference between the distribution of the input acoustic feature quantity by the sound and the probability density distribution of the acoustic model. If the acoustic feature quantity of each dimension before MVN is x (t) and the acoustic feature quantity after MVN is x ′ (t), the operation of MVN is expressed by the following equation (2).

  However, for CMN and MVN that use the average value and variance of the entire utterance, the start of the matching process is delayed because the normalized acoustic features are not obtained until the utterance is completed, and the waiting time from the end of the utterance to the output of the recognition result There is a demerit that makes it longer.

  As a technique for reducing this processing delay, a technique has been proposed in which an average value and variance are calculated from a part (several hundred milliseconds) from the beginning of the utterance instead of the entire utterance and used for normalization. This method lowers the accuracy of the average value and variance instead of reducing the recognition processing delay. Hereinafter, the MVN for calculating the average value and variance from the entire utterance is expressed as “batch MVN”, and the MVN for calculating the average value and variance from a part of the utterance is expressed as “segmental MVN”.

  However, when performing segmental MVN, if the average value of the utterance head and the interval for calculating the variance are silent or stationary, and the quantization vector becomes one type, the variance becomes zero and normalization In this case, division by zero occurs.

In order to solve such a technical problem, in Patent Document 1, when the variance value calculated in the mean value / variance normalization (MVN) without assuming the quantization of the feature amount is zero or a small value close to zero, A technique that does not perform distributed normalization has been proposed.
JP 2002-278586 A

  Conventional average value / variance feature normalization method (MVN) is based on the assumption that there is no quantization effect on the acoustic features, and input acoustic features caused by variations in microphone characteristics, differences among speakers, background noise, etc. The deviation between the quantity distribution and the probability density distribution of the acoustic model was reduced.

  However, in a distributed (client / server type) speech recognition system in which acoustic features to be recognized are transmitted to a remote speech recognition unit by communication, the feature values of speech detected by the microphone of the client are converted by vector quantization. Sent to the server after the amount of data has been reduced. On the server side, the feature quantity received from the client is decoded to reproduce the feature quantity, and a recognition result is obtained based on the pattern matching between the feature quantity and the probability model.

  In vector quantization, both the client and the server have a finite vector quantization (VQ) codebook that expresses multidimensional features, and in the client, the input feature vector (feature) is the nearest code vector. And send the index to the server. Since the server also has a common vector quantization codebook, a code vector is specified based on the index transmitted from the client, and the feature quantity is decoded by using the code vector as an acoustic feature quantity. That is, the feature vector of the input speech signal is approximated to the nearest code vector through vector quantization and decoding.

  FIG. 7 is a diagram showing a vector quantization codebook for distributed speech recognition standardized by the European standardization organization ETSI (European Telecommunications Standards Institute). In the standard method of ETSI, a total 14-dimensional feature quantity consisting of a cepstrum region feature quantity MFCC (Mel-Frequency Cepstrum Coefficient) and a logarithmic power is quantized with two types of VQ codebooks in two dimensions. The primary and secondary spaces of the MFCC are expressed by 64 code vectors as shown in FIG.

  In such a configuration, the feature quantity to be recognized includes vector quantization distortion (VQ distortion) for vector quantization and decoding thereof, whereas the acoustic model learning speech prepared in advance on the server The feature amount of data does not include VQ distortion when it has not undergone vector quantization and decoding. In this case, there is a difference between the variance of the acoustic feature quantity to be recognized and the variance of the acoustic model, and the effect of improving the recognition rate by MVN is diminished.

  If the variance normalization is not performed when the variance value is zero or a small value close to zero as in Patent Document 1, the probability model includes the feature amount normalization, whereas the input feature Since the quantity does not include feature quantity normalization, a mismatch occurs in pattern matching and the accuracy of speech recognition is reduced.

  An object of the present invention is to solve the above-described problems of the prior art, and a pattern matching method and apparatus suitable for collating a feature quantity including VQ distortion with a probability model after vector quantization and decoding thereof, and the above It is an object of the present invention to provide a feature amount normalization method and apparatus suitable for feature amount normalization.

  Considering MVN for acoustic features that have undergone vector quantization, the variance of the input acoustic features before vector quantization is the sum of the variance after vector quantization and the variance of vector quantization distortion (VQ distortion). expressed. This is shown in the following mathematical formula.

  The m-th dimension value of the acoustic feature of the k-th frame of the input speech is xm (k), and the m-th dimension value of the selected VQ codebook is qm (k), xm (k) and qm (k) If the difference between is Δxm (k), the following equation (3) is established.

  Further, if the average value and variance of the entire utterance of the variable x (k) are expressed by E (x) and V (x), the following equation (4) is established.

  Here, assuming that E (Δxm) = 0, the following equation (5) is established.

  Here, if q (k) m−E (qm) and Δxm (k) are uncorrelated, the following equation (7) is established from the following equation (6).

  That is, the variance of the input acoustic feature quantity before VQ is represented by the sum of the variance of the acoustic feature quantity after VQ and the variance of the VQ distortion. Therefore, in order to estimate the variance of the input acoustic feature quantity before VQ from the acoustic feature quantity after VQ that has arrived at the speech recognition server, it is only necessary to add the variance of VQ distortion calculated in advance for each dimension.

  The feature quantity normalization device and the pattern matching device of the present invention are made based on the above-described knowledge, and are characterized in that the following measures are taken.

  (1) The feature quantity normalization apparatus of the present invention corrects the means for calculating the average value and variance of the decoded feature quantities, and adds the variance of the vector quantization distortion to the variance of the decoded feature quantities A post-correction variance calculation unit that calculates post-variance, and a normalization unit that normalizes the decoded feature amount using the average value and the post-correction variance.

  (2) The pattern matching apparatus according to the present invention includes a decoding unit that decodes a feature quantity extracted from an input signal and subjected to vector quantization, a feature quantity normalization unit that normalizes the decoded feature quantity, and the normalization And a recognition processing means for outputting a pattern matching result by comparing the normalized feature quantity with a probability model, and the feature quantity normalizing means is a means for calculating an average value and variance of the decoded feature quantity, A post-correction distribution calculating unit for calculating the post-correction variance by adding the variance of the vector quantization distortion to the variance of the obtained feature amount, and normalizing the decoded feature amount using the average value and the post-correction variance And normalizing means.

According to the present invention, the following effects are achieved.
(1) According to the feature quantity normalization method and apparatus of the present invention, when the feature quantity of the input signal includes VQ distortion after vector quantization and decoding thereof, the feature quantity takes VQ distortion variance into account. Since normalization is performed, it is possible to reduce a deviation between the distribution of the feature quantity and the probability density distribution of the probability model constructed based on the feature quantity that has not undergone vector quantization and decoding.
(2) According to the feature amount normalization method and apparatus of the present invention, when the feature amount of the input signal includes VQ distortion after vector quantization and decoding thereof, the feature amount is determined by the distribution of the feature amount and the VQ. Normalization is based on the post-correction variance calculated as the sum of the variance of the distortion, so even if a single code vector is selected and the feature variance is zero, avoid division by zero during normalization can do.
(3) According to the pattern matching method and apparatus of the present invention, in the normalization unit that normalizes the feature amount, the feature amount including VQ distortion is normalized in consideration of the variance of the VQ distortion through vector quantization and decoding thereof. Therefore, it is possible to reduce a deviation between the distribution of the feature quantity and the probability density distribution of the probability model constructed based on the feature quantity that has not undergone vector quantization and decoding. Therefore, the accuracy of pattern matching can be improved.
(4) According to the pattern matching method and apparatus of the present invention, in the normalization unit that normalizes the feature amount, the feature amount including VQ distortion after vector quantization and decoding thereof is the variance of the feature amount and the VQ distortion. Since normalization is performed based on the post-correction variance obtained as an addition value to the variance of, even if a single code vector is selected and the feature variance becomes zero, avoid division by zero during normalization be able to.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the best embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the main part of a distributed speech recognition apparatus to which the pattern matching method of the present invention is applied. A terminal 1 for vector-quantizing and transmitting an acoustic feature quantity of an input speech signal, , Receiving and decoding the vector quantized data from the terminal 1 to reproduce the acoustic feature amount, and returning the recognition result obtained by pattern matching between the acoustic feature amount and the acoustic model (probability model) to the terminal 1 The server 2 and the learning device 3 that constructs an acoustic model used for speech recognition by the speech recognition server 2 through prior processing are the main components.

  In the terminal 1, the acoustic feature amount extraction unit 12 extracts the acoustic feature amount from the audio signal input from the microphone 11 and temporarily stores it. The acoustic feature amount is a time-series feature vector obtained by analyzing the input speech at regular time intervals (for example, 10 ms: hereinafter referred to as a frame). In this embodiment, a multidimensional feature vector is generated for each frame.

  The vector quantization unit 13 quantizes the feature vector (vector quantization) using a vector quantization (VQ) codebook. More specifically, a code vector that is closest to the feature vector on the VQ codebook is selected and its index is extracted. The extracted index is transmitted to the speech recognition server 2.

  In the speech recognition server 2, the vector quantization (VQ) decoding unit 21 decodes the received index using the same VQ codebook as the VQ codebook of the terminal 1, and the feature vector is decoded by the decoded code vector. Approximate. The VQ distortion dispersion storage unit 27 stores a quantization vector dispersion (VQ distortion) calculated in advance with respect to the acoustic feature quantity. As will be described later in detail, the feature quantity normalization unit 22 calculates the acoustic feature quantity based on the variance of the VQ distortion stored in advance in the VQ distortion variance storage section 27 and the variance of the decoded acoustic feature quantity. Normalize.

  The dispersion stored in the VQ distortion dispersion storage unit 27 generates, for example, a large number of acoustic feature quantities at random, and these are collated with the VQ codebook by the vector quantization unit 13 to quantize the shortest distance for all feature vectors. The quantization vector can be obtained and can be obtained in advance as the mean square distance between the feature vector and the quantization vector. The mean square distance between the feature vector and the quantization vector can be calculated for each dimension of the feature vector.

  The recognition processing unit 23 sequentially takes in the normalized acoustic feature values, collates the acoustic model 25 with the acoustic feature values in accordance with the grammatical constraint conditions stored in the word dictionary grammar storage unit 24, and determines the acoustic logarithm. A recognition result is output based on the likelihood. The recognition result transmission unit 26 transmits the voice recognition result to the terminal. In the terminal 1, the recognition result is received by the recognition result receiving unit 14 and appropriately processed.

  In the learning device 3, a large amount of learning speech data is stored in the learning speech database 32, and a transcription text of each speech data is stored in the transcription text storage unit 31. The acoustic feature quantity extraction unit 33 extracts an acoustic feature quantity from the voice data stored in the learning voice database 32. This acoustic feature amount is normalized by the normalizing unit 34 and is stored as the acoustic model 25 in association with the transcription text in the acoustic model learning unit 35.

  FIG. 2 is a block diagram showing the configuration of the main part of the feature quantity normalization unit 22. The average value / variance calculation unit 51 calculates the average value y and the variance x1 of the acoustic feature amount (feature vector) decoded by the VQ decoding unit 21. The calculation result of the average value y is subtracted from the acoustic feature amount in the adding unit 52. The variance x1 is added by the adder 53 to the VQ distortion variance x2 stored in the VQ distortion variance storage unit 27 in advance, and the variance x considering the VQ distortion is obtained. In the normalization unit 54, the acoustic feature value is normalized using the variance x, and the acoustic feature value after MVN is calculated.

  FIG. 3 is a block diagram showing another configuration of the feature quantity normalization unit 22, and the same reference numerals as those described above represent the same or equivalent parts.

  The VQ distortion distribution storage unit 27 stores VQ distortion distribution for each code vector or code vector class (set). The VQ distortion variance selection unit 55 extracts the corresponding VQ distortion variance from the VQ distortion variance storage unit 27 based on the code vector of the acoustic feature quantity being decoded or the class of the code vector notified from the VQ decoding unit 21. Further, the average value of the extracted VQ distortion variance is calculated and output to the adder 53 for every frame of a single utterance or for every predetermined number of frames.

  FIG. 4 is a block diagram showing a configuration of still another embodiment of the distributed speech recognition apparatus to which the pattern matching of the present invention is applied. The same reference numerals as those described above represent the same or equivalent parts.

  In each of the above-described embodiments, it has been described that the variance of the VQ distortion added in the speech recognition server 2 is statistically obtained in advance, stored in the storage device 27, and read out. It is characterized in that a VQ distortion variance calculation unit 15 is provided.

  The VQ distortion variance calculation unit 15 calculates the distance between each feature vector and each quantization vector when the vector quantization unit 13 determines the shortest distance quantization vector on each code vector. To obtain the variance of the VQ distortion. This distribution is transmitted to the speech recognition server 2 and is used by the feature amount normalization unit 22 to normalize the corresponding frame. The VQ distortion variance calculation unit 15 may obtain the variance of all frames for each dimension of the feature vector, or may obtain the variance for each predetermined number of frames.

  In the above-described embodiment, the pattern matching method and the normalization method of the present invention have been applied to the speech recognition apparatus. However, the present invention is not limited to this, as shown in FIG. In addition, the present invention can be similarly applied to an image recognition apparatus that recognizes an image captured by the camera 16.

  In this case, the terminal 1a is provided with an image feature quantity extraction unit 12a instead of the acoustic feature quantity extraction unit 12, and the image recognition server 2a is provided with an object model storage unit 25a instead of the acoustic model storage unit 25. In the object model storage unit 25a, data obtained by normalizing feature amounts extracted from a large number of images registered in the learning image database 32a and a teacher stored in the object model teacher data storage unit 31a are stored. An object model (probability model) learned based on the data is registered. The recognition processing unit 23 collates the data normalized by the feature amount normalizing unit 22 with the object model to obtain a recognition result.

  Furthermore, as shown in FIG. 6, the pattern matching method and its normalization method of the present invention are based on the user's preference input from the preference data input unit 17, such as services and products that match the user's preference. The present invention can be similarly applied to a recommendation device that predicts a category.

  In this case, the terminal 1 b is provided with a user profile feature amount extraction unit 12 b instead of the acoustic feature amount extraction unit 12, and digitized preference data is input from the preference data input unit 17. In the recommendation server 2b, a user profile category model storage unit 25b is provided instead of the acoustic model storage unit 25. In the user profile category model storage unit 25b, data obtained by normalizing feature quantities extracted from a large amount of preference information registered in the learning user preference database 32b and the category teacher data storage unit 31b are stored. The user profile category model (probability model) learned based on the current teacher data is registered. The recognition processing unit 23 collates the data normalized by the feature amount normalizing unit 22 with the user profile category model to obtain a recommendation result.

  As described above, the pattern matching method and its normalization method of the present invention calculate the similarity between an unknown input pattern and many known standard patterns, and the standard pattern with the highest similarity is used as the pattern matching result. It can be applied to all pattern matching methods and their normalization methods.

1 is a block diagram of a first embodiment of a distributed speech recognition apparatus to which a pattern matching method of the present invention is applied. It is the block diagram which showed the structure of the acoustic feature-value normalization part. It is the block diagram which showed the other structure of the acoustic feature-value normalization part. It is a block diagram of other embodiment of the distributed speech recognition apparatus to which the pattern matching of this invention is applied. It is a block diagram of the image recognition apparatus to which the pattern matching of this invention is applied. It is a block diagram of a recommendation device to which pattern matching of the present invention is applied. It is the figure which showed an example of the vector quantization codebook for distributed speech recognition.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Terminal, 2 ... Speech recognition server, 3 ... Learning apparatus, 11 ... Microphone, 12 ... Acoustic feature-value extraction part, 13 ... Vector quantization part, 21 ... Vector quantization (VQ) decoding part, 22 ... Acoustic feature-value Normalization unit, 23 ... recognition processing unit, 24 ... word dictionary grammar storage unit, 25 ... acoustic model storage unit, 26 ... recognition result transmission unit, 27 ... VQ distortion distribution storage unit

Claims (12)

In a feature quantity normalization device that normalizes a decoded feature quantity after being extracted from an input signal and once vector quantized,
Means for calculating an average value and variance of the decoded feature values;
A post-correction variance calculation means for calculating a post-correction variance by adding a variance of vector quantization distortion to the variance of the decoded feature quantity;
And a normalizing unit that normalizes the decoded feature value using the average value and the variance after correction. Storage means for storing the variance of the vector quantization distortion calculated separately;
The feature quantity normalization apparatus according to claim 1, wherein the variance of the vector quantization distortion is read from the storage unit. The feature amount extracted from the input signal is vector-quantized into a code vector based on a matching result with a VQ codebook,
The storage means stores a variance for each code vector or class of code vectors,
Selecting means for selecting a distribution corresponding to the code vector or its class from the storage means;
The feature amount normalization apparatus according to claim 2, wherein the post-correction variance calculation unit adds a frame average value of the selected variance to the variance of the decoded feature amount. In a feature quantity normalization method for normalizing a decoded feature quantity after being extracted from an input signal and once vector quantized,
Calculating an average value and variance of the decoded feature values;
A procedure of calculating a corrected variance by adding a variance of vector quantization distortion to the variance of the decoded feature quantity;
And a step of normalizing the decoded feature value using the average value and the corrected variance. In a pattern matching device that is extracted from an input signal, once vector quantized, and then collated with a probability model to output a pattern matching result.
Decoding means for decoding vector quantized data of features extracted from the input signal;
Feature quantity normalizing means for normalizing the decoded feature quantity;
Recognizing processing means for collating the normalized feature quantity with a probability model and outputting a pattern matching result;
The feature amount normalizing means includes:
Means for calculating an average value and variance of the decoded feature values;
A post-correction fraction calculation unit for calculating a post-correction variance by adding a variance of vector quantization distortion to the variance of the decoded feature quantity;
And a normalizing unit that normalizes the decoded feature amount using the average value and the variance after correction. Storage means for storing the variance of the vector quantization distortion calculated separately;
The pattern matching apparatus according to claim 5, wherein the variance of the vector quantization distortion is read from the storage unit. The feature amount extracted from the input signal is vector-quantized into a code vector based on a matching result with a VQ codebook,
The storage means stores a variance for each code vector or class of code vectors,
Selecting means for selecting a distribution corresponding to the code vector or its class from the storage means;
The pattern matching apparatus according to claim 6, wherein the post-correction variance calculation unit adds a frame average value of the selected variance to the variance of the decoded feature amount. Means for receiving a variance of the vector quantization distortion;
The pattern matching apparatus according to claim 5, wherein the corrected variance calculation unit adds the received variance to the variance of the decoded feature quantity.   The pattern matching apparatus according to claim 5, wherein the input signal is a voice signal, and the pattern matching result is a voice recognition result.   9. The pattern matching apparatus according to claim 5, wherein the input signal is an image signal, and the pattern matching result is an image recognition result.   The pattern matching apparatus according to claim 5, wherein the input signal is user preference information, and the pattern matching result is a recommendation result. In a pattern matching method, which is extracted from an input signal, once vector quantized, and then collated with a probability model to output a pattern matching result.
Decoding vector quantization data of feature quantities extracted from an input signal and decoding the feature quantities;
Normalizing the decoded feature quantity;
A step of collating the normalized feature quantity with a probability model and outputting a pattern matching result,
The procedure for normalizing the feature amount is:
Calculating an average value and variance of the decoded feature values;
A procedure of calculating a corrected variance by adding a variance of vector quantization distortion to the variance of the decoded feature quantity;
And a step of normalizing the decoded feature amount using the average value and the corrected variance.