JP4860962B2 - Speech recognition apparatus, speech recognition method, and program - Google Patents

Description

  The present invention relates to a speech recognition apparatus, a speech recognition method, and a program for performing speech recognition in an environment where noise occurs.

  In general, in speech recognition, an acoustic analysis unit that converts a speech sample uttered by a speaker into a sequence of feature parameters, and a sequence of feature parameters obtained by the acoustic analysis unit is stored in a storage device such as a memory or a hard disk in advance. It is composed of two parts: a speech collating unit that collates with information related to the characteristic parameters of vocabulary words and recognizes speech with the highest similarity as a recognition result. Known acoustic analysis methods for converting a speech sample into a series of characteristic parameters include cepstrum analysis and linear prediction analysis. These analysis methods are described in detail in Non-Patent Document 1, for example. MFCC (Mel-Frequency Cepstrum Coefficient) has been proposed as a cepstrum obtained from the logarithmic power spectrum of a filter bank arranged on the mel scale, and is often used as a feature parameter in speech recognition. Hidden Markov Model (HMM) is used as a method for creating information on feature parameters of vocabulary words and a speech matching method between the created information and a sequence of feature parameters converted from input speech. Is generally used. In the method based on the HMM, speech units such as syllables, semi-syllables, phonemes, biphones, and triphones are modeled by the HMM. These models are generally called acoustic models. The method of creating the acoustic model is described in detail in Sec. 6.4 of Non-Patent Document 1. Further, a Viterbi algorithm described in Sec. 6.4 of Non-Patent Document 1 allows a person skilled in the art to easily configure a speech recognition device.

Hereinafter, the HMM and Viterbi algorithm will be described in detail. The HMM is defined by a set of six parameters M = (S, Y, A, B, π, F).
S: Finite set of states S = [s _i ]
Y: Set of output symbols
A: Set of state transition probabilities A = [a _ij ] a _ij is the transition probability from state s _i to state s _j

B: Set of output probabilities B = [b _j (x)] b _j (x) is the probability of outputting an output symbol x at the transition to state s _j

π: set of initial state probabilities π = [π _i ] π _i is the probability that the initial state is s _j

F: Set of final states At that time, the conventional Viterbi algorithm is defined as follows through forward probability f (i, t).
Step 0:

Step 1: Repeat step 1.1 for t = 1, 2,.
Step 1.1: Repeat steps 1.1.1 to 1.1.3 for all i.
Step 1.1.1:

  Step 1.1.2:

  Step 1.1.3:

Here, the symbol included in the right side of the expression (1-5) is an operator that connects the left state transition sequence and the right state to create a new state transition sequence.
Step 2:

FIG. 11 shows a minimum configuration of a Left-to-Right HMM acoustic model that is often used in speech recognition. The minimum configuration of the Left-to-Right HMM acoustic model consists of two states, state i-1 and state i. The probability of transition from state i-1 to state i is _ai-1i , and from state i to state i. The probability of self-looping is given by a _ii . Further, the state i has an output probability b _i (x). The forward probabilities f (i−1, t−1) and f (i, t−1) in state i−1 and state i at time t−1 have already been sequentially calculated by the Viterbi algorithm.
At this time, as shown in FIG. 12, according to the conventional Viterbi algorithm, the forward probability f (i, t) in the state i at time t is expressed by Equations (1-3) and (1-4). By

  Is required. In speech recognition using the HMM, the word sequence W corresponding to the mixture number M of the normal distribution of the HMM that maximizes the Viterbi score calculated by Expression (1-7) is used as the recognition result. Further, in the Viterbi algorithm, if log f (i, t) is used instead of f (i, t), multiplication can be replaced with addition, so that the calculation becomes easier and probability values of 0 or more and 1 or less are obtained. There is also an advantage that the problem of underflow associated with multiplication can be avoided.

Also, in the matching process using the Viterbi algorithm, in order to reduce the amount of calculation, a technique called branch hunting is also widely used, in which words whose Viterbi score is significantly smaller than others are thinned out even during the matching.
On the other hand, a method based on a Gaussian Mixture Model (GMM) is known as a means for modeling noise. The GMM can be regarded as a one-state HMM without state transition.

Conventionally, as a means for improving robustness against noise in a speech recognition system,
Conventional method 1) Method of synthesizing a noise-superimposed speech HMM acoustic model, which is a speech HMM acoustic model superimposed with noise, from a clean speech HMM acoustic model and a noise GMM acoustic model Conventional method 2) Input for each frame Method of subtracting noise spectrum estimate from spectrum or method of multiplying input spectrum by speech spectrum / (voice spectrum + noise spectrum) Conventional method 3) Noise superimposed speech HMM sound from speech data with noise superimposed on clean speech Methods for learning a model Conventional methods 4) A method combining conventional methods 2) and 3) is known. As the conventional method 1, the HMM synthesis method described in Non-Patent Document 2 and Non-Patent Document 3 is well known. As the conventional method 2, a spectral subtraction method described in Non-Patent Document 4 and a minimum mean square error (MMSE) method described in Non-Patent Document 5 are well known.

  However, since the number of acoustic models synthesized in the conventional method 1 is proportional to the number of types of noise, there is a problem that the required calculation amount and memory size become enormous when dealing with various types of noise in a real environment. there were. For example, in the case of constructing a Japanese speech recognition system, if 54 phonemes shown in FIG. 9 are modeled by an HMM, the speech recognition system uses 54 clean phoneme HMM models created from clean speech. Will have. If there are 33 types of noise and there are 33 corresponding GMM acoustic models of noise, 1882 noise superimposed phoneme HMM acoustic models, which is a total of 54 times 33, are created. become. In other words, the acoustic model size is 33 times that of the case of having one set of clean phoneme HMM acoustic models, so that the required calculation amount and memory size become enormous and difficult to realize.

  On the other hand, the conventional method 2 is effective for stationary noise, but if there is non-stationary noise such as a door closing sound, a cupping sound, or a curtain closing sound, a significant reduction in the speech recognition rate is inevitable. This is a fatal problem, and the problem that it is necessary to improve speech recognition performance in a real environment where non-stationary noise exists naturally has become apparent.

  Conventional method 3 exhibits better speech recognition performance than conventional method 1 and conventional method 2, but in order to learn a noise superimposed speech HMM acoustic model, speech data in which noise is superimposed on clean speech for each noise is used. There is a problem that the amount of calculation and memory required for creation are proportional to the number of types of noise. For example, when a Japanese speech recognition system is constructed, when 54 phonemes shown in FIG. 9 are modeled by an HMM, if there are 33 types of noise, only a clean phoneme HMM is created. Compared to the above, the voice data with noise superimposed is created 33 times, and the required calculation amount and memory size are enormous, making it difficult to realize.

  Conventional method 4 has the strongest robustness against noise compared to conventional method 1, conventional method 2, and conventional method 3, but is robust against the problems of conventional method 1, conventional method 2, and conventional method 3, that is, non-stationary noise. The problem is that the amount of computation and the amount of memory are not proportional to the number of types of noise.

  The reason why the speech recognition performance in an environment where non-stationary noise exists is significantly reduced is considered to be because the feature parameter value of the speech sample is greatly deformed by the superposition of non-stationary noise. For example, an example will be described in which the glass breaks in the middle of speaking the voice “radio” and the noise “gachan” is superimposed on the voice. FIG. 13 shows an example of a waveform in the case where the noise “Gachang” is superimposed on the sound “Radio”. Here, below the waveform, time divisions when “radio” is decomposed into phonemes shown in FIG. 9 are shown. If the noise “Gachang” occurs at the moment when the vowel “i” of “Di” of “Radio” is uttered, the noise “Gachang” is superimposed on the time interval indicated by the dashed arrow. At this time, in the midway result of recognition by the Viterbi algorithm at time T1, the correct word “radio” is ranked first as shown in FIG. However, in the result of the recognition by the Viterbi algorithm at time T2, if the spectrum pattern of the noise “Gachan” happens to be similar to the spectrum pattern of the speech “Casey”, as shown in FIG. The correct word “radio” is ranked fifth, and the incorrect word “Lacket” is ranked first, instead.

  As described above, due to noise superposition, in a frame in which the deformation of the speech spectrum is large, there is a phenomenon that the value of the similarity of the correct word is remarkably lowered or the value of the similarity of the incorrect word is remarkably increased. It tends to occur. At this time, the ranking of the similarity is changed between candidate words, and the similarity value of the incorrect word is higher than the similarity of the correct word. In the worst case, it is often used in combination with the Viterbi algorithm. The correct word may be spilled out of the candidate words by the known technique of branch hunting. This causes a significant reduction in recognition rate.

More specifically, in the case of a frame in which the influence of non-stationary noise is significant, the value of the output probability b _i (x _t ) used in the calculation of Expression (1-4) in the conventional Viterbi algorithm described above is non- The value is significantly smaller than when no stationary noise is superimposed. This causes a phenomenon that the correct word f (i, t) has a smaller value than that of the incorrect word. Or, if the spectrum of a frame with unsteady noise happens to be similar to the spectrum of a frame with an incorrect word, the incorrect word output probability b _i (x _t ) becomes a large value and the incorrect word There is also a phenomenon in which f (i, t) is larger than that of the correct word.
If this rank fluctuation caused by the presence of non-stationary noise can be suppressed, speech recognition robust to non-stationary noise can be realized. In order to solve such a problem, Patent Documents 1 to 3 exist as patent documents in which technical contents for performing speech recognition in response to non-stationary noise are described.

Lawrence Rabiner and Biing-Hwang Juang, "Fundamentals of Speech Recognition", Prentice Hall, 1993 F. Martin, K. Shikano and Y. Minami, "Recognition of Noisy Speech by Composition of Hidden Markov Models," Proc. Eurospeech, Berlin, Germany, pp.1031-1034, 1993. M. J. F. Gales and S. Young, "Cepstrum Parameter Compensation for HMM Recognition," Speech Communication, Vol.12, No.3, pp.231-239, 1993. S. Boll, "Suppression of Acoustic Noise in Speech Using Spectral Subtraction," IEEE Trans. ASSP, Vol.ASSP-27, No.2, pp.113-120, 1979. Y. Ephraim and D. Malah, "Speech Enhancement Using a Minimum Mean Square Error Short-Time Spectral Amplitude Estimator," IEEE Trans. Vol. ASSP-32, No. 6, pp. 1109-1121, 1984. JP 2002-91480 A JP 2003-177771 A JP 2003-280686 A

  Patent Document 1 discloses a technique similar to the HMM synthesis method belonging to the conventional method 1 described above. In Patent Document 1, a method for generating a noise-superimposed speech HMM acoustic model from a speech HMM acoustic model and a noise GMM acoustic model with a single normal distribution as described in Non-Patent Documents 2 and 3 above Then, it pays attention to the point which cannot cope with various non-stationary noises. And in this patent document 1, it aims at coping with nonstationary noise by extending the acoustic model of noise to GMM by mixed normal distribution, and improving the expressive capability with respect to various nonstationary noise. However, since the technique described in Patent Document 1 belongs to the conventional method 1, as the type of noise to be assumed increases, there is a problem that the data size for creating the acoustic model and the acoustic model size become enormous, and the acoustic model creation time, It cannot be avoided from the problem that the verification time by the Viterbi algorithm becomes enormous.

  The technique disclosed in Patent Document 2 is also similar to the HMM synthesis method belonging to the conventional method 1 described above. In Patent Document 2, it is noted that in Patent Document 1 a noise-superimposed speech HMM acoustic model must be synthesized under the restriction that the signal-to-noise ratio of the input speech is known. It is an object of the present invention to deal with non-stationary noise by generating a noise-superimposed speech HMM acoustic model having a multipath configuration for a plurality of signal-to-noise ratios from the GMM acoustic model. However, since the technique described in Patent Document 2 also belongs to the conventional method 1, as the type of noise to be assumed increases, the problem that the data size for creating the acoustic model and the acoustic model size become enormous, and the acoustic model creation time, It cannot be avoided from the problem that the verification time by the Viterbi algorithm becomes enormous.

  The technique disclosed in Patent Document 3 is also similar to the HMM synthesis method belonging to the conventional method 1 described above. In this Patent Document 3, since the type of noise is specified for each utterance in Non-Patent Document 2 and the noise superimposed speech and the noise superimposed speech HMM acoustic model are compared, Focusing on the fact that the noise generated irregularly cannot be dealt with, the utterance section is divided into appropriate sections such as frames, and one noise-superimposed voice HMM sound is selected from a plurality of noise-superimposed voice HMM acoustic models for each section. The object is to cope with suddenly generated noise or irregularly generated noise by selecting a model and performing matching processing using the Viterbi algorithm. However, since the technique described in Patent Document 3 also belongs to the conventional method 1, as the type of noise to be assumed increases, there is a problem that the data size for creating the acoustic model and the acoustic model size become enormous, and the acoustic model creation time, It cannot be avoided from the problem that the verification time by the Viterbi algorithm becomes enormous.

  The present invention has been made in view of such problems, and in an environment where noise occurs, a speech recognition device, a speech recognition method, and a speech recognition device that perform highly accurate speech recognition while suppressing processing amount and memory consumption. The purpose is to provide a program.

In order to solve the above-described problem, the invention described in claim 1 stores a speech HMM (Hidden Markov Model) acoustic model in a speech recognition device that performs speech recognition using a Viterbi algorithm on speech input to the device. Stored in the first storage means, a second storage means for storing a noise GMM (Gaussian Mixture Model) acoustic model , and every predetermined frame of the input speech. First calculation means for calculating an output probability of the speech HMM acoustic model for the speech feature parameter to be recognized for each state of the speech HMM acoustic model, and for the feature parameter for each predetermined frame , a second calculating means for calculating the output probability of the GMM acoustic model of the noise stored in the second storage means, said first calculating means for each said predetermined frame More calculated output probabilities, and, from the calculated output probabilities by said second calculating means, selection means for selecting the largest output probability, the output probability selected by said selecting means the output probability Provided is a speech recognition apparatus comprising: a Viterbi algorithm used as an output probability of an HMM acoustic model of speech in a selected frame; and a collating means for performing speech recognition of the input speech using the Viterbi algorithm To do.

According to this configuration, the speech recognition apparatus can store the speech HMM acoustic model and the noise GMM acoustic model separately in the first storage unit and the second storage unit without synthesizing, Speech recognition processing can be performed using a small number of acoustic models. The speech recognition apparatus, calculated output probabilities by the first calculating means, and the largest output probability selected from among the calculated output probabilities by the second calculating means, the speech HMM acoustic model Since the output probability is used in the Viterbi algorithm, it is possible to prevent a reduction in recognition accuracy due to the influence of noise. Therefore, the speech recognition apparatus can perform highly accurate speech recognition while suppressing processing amount and memory consumption in an environment where noise is generated.

According to a second aspect of the present invention, in the voice recognition device according to the first aspect, the selection unit includes:
First selecting means and the output probability calculation by the first calculating means and the first to be selected as the maximum of the maximum output probability of output probability noise of the second calculated output probabilities by the calculation means And a second selection means for selecting a larger output probability among the maximum output probabilities of noise selected by the selection means.
According to this configuration, since the speech recognition apparatus selects the output probability in two stages, it is possible to grasp the maximum output probability of noise. Based on the maximum output probability of noise, frequent or grasp the noise being generated, the output probability of the noise GMM acoustic models to grasp the when selected in place of the output probability of the speech HMM acoustic model This is effective when the maximum output probability of noise is used as information.

The invention according to claim 3 is the speech recognition apparatus according to claim 1 or 2, wherein the second storage means has one GMM acoustic model of noise created by mixing a plurality of types of noise. It is memorized.
According to this configuration, the speech recognition apparatus stores only one GMM acoustic model of noise, and uses only one GMM acoustic model of noise for calculation, so that the processing load and acoustic model for creating the acoustic model are reduced. The amount of memory for storing and the amount of calculation for speech recognition can be reduced.

According to a fourth aspect of the present invention, in the voice recognition device according to the first or second aspect, the second storage means is a noise generated by superimposing a voice with a predetermined signal-to-noise ratio on the noise. The GMM acoustic model is stored.
According to this configuration, the speech recognition apparatus can perform speech recognition based on a GMM acoustic model of noise that is close to the real environment and is created by superimposing speech on noise at a predetermined signal-to-noise ratio. Recognition accuracy can be increased.

According to a fifth aspect of the present invention, in the speech recognition device according to any one of the first, second, and fourth aspects, the second storage means is based on a mutual distance between GMM acoustic models of noise. The noise GMM acoustic model is classified, and for each classification, two or more noise GMM acoustic models re-created based on the noise data from which the noise GMM acoustic model was created are stored. It is characterized by.
According to this configuration, the speech recognition apparatus can classify a plurality of noises into a set of similar ones, and perform speech recognition using two or more GMM acoustic models of noise created for each classification. It is possible to perform highly accurate speech recognition while reducing the processing amount.

The invention according to claim 6, in the speech recognition method for performing speech recognition using the Viterbi algorithm, a predetermined frame of the speech to be recognized, and, for each state of the speech HMM acoustic model, and the recognition target A first calculation step of calculating an output probability of the speech HMM acoustic model with respect to a speech feature parameter; and a second calculation step of calculating an output probability of a noise GMM acoustic model with respect to the feature parameter for each predetermined frame. When the said first calculated output probabilities in the calculation step for each predetermined frame, and from among the calculated output probabilities in the second calculation step, a selection step of selecting the maximum output probabilities, the has been output probabilities selected by the selection step, voice HMM acoustic model in the frame of the output probability is selected Used as the output probability Viterbi algorithm provides a speech recognition method characterized by having a matching step of performing speech recognition of the speech to be the recognition target by using the Viterbi algorithm.

According to this method, calculated output probabilities in the first calculation step, and the maximum output probabilities selected from among the calculated output probabilities in the second calculation step, the output of the speech HMM acoustic model Since it is used for the Viterbi algorithm as a probability , the recognition accuracy does not deteriorate due to the influence of noise. For this reason, a highly accurate speech recognition method can be realized.

Invention of claim 7, the computer, each predetermined frame of the speech to be recognized, and, for each state of the speech HMM acoustic model, the speech HMM acoustic model for the feature parameters of the speech to be the recognition target A first calculation step of calculating an output probability of the noise, a second calculation step of calculating an output probability of a noise GMM acoustic model for the feature parameter for each predetermined frame , and the first calculation step for each predetermined frame calculated output probabilities in the calculation step, and, from the calculated output probabilities in the second calculation step, a selection step of selecting the maximum output probability, the output probability selected by said selecting step, the used in the Viterbi algorithm as an output probability of the speech HMM acoustic model in the frame of the output probability is selected, those It provides a program for executing a matching step of performing speech recognition of the speech to be the recognition target by using the Viterbi algorithm.

According to the present invention, the speech recognition apparatus can store the speech HMM acoustic model and the noise GMM acoustic model separately in the first storage unit and the second storage unit, and uses a small number of acoustic models. Voice recognition processing can be performed. The speech recognition apparatus, calculated output probabilities by the first calculating means, and the largest output probability selected from among the calculated output probabilities by the second calculating means, the speech HMM acoustic model Since the output probability is used in the Viterbi algorithm, it is possible to prevent a reduction in recognition accuracy due to the influence of noise. For this reason, the speech recognition apparatus can realize highly accurate speech recognition while suppressing processing amount and memory consumption.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings. The speech recognition apparatus 10 according to the embodiment of the present invention has a hardware configuration of a computer. That is, the speech recognition apparatus 10 includes a storage unit including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and a hard disk device (not shown), and an input / output interface. Various kinds of software are stored in the storage unit of the speech recognition apparatus 10. For example, an acoustic model of voice and noise is stored in the hard disk device. In addition, the hard disk stores a program for performing arithmetic processing according to the following Viterbi algorithm of the present invention.
The Viterbi algorithm of the present invention expresses the output probability distribution of the GMM acoustic model of non-stationary noise k (k = 1, 2,..., K) as c _k (x), _where K is the number of types of non-stationary noise. Then, it is defined as follows.
Step 0 ':

Step 1 ′: Step 1.1 ′ is repeated for t = 1, 2,.
Step 1.1 ′: Steps 1.1.1 ′ to 1.1.5 ′ are repeated for all i.
Step 1.1.1 ′:

  Step 1.1.2 ':

  Step 1.1.3 ':

  Step 1.1.4 ':

  Step 1.1.5 ':

  Step 2 ':

As described above, in the Viterbi algorithm of the present invention, in each frame, the value of the output probability b _i (x _t ) of each state of the HMM acoustic model of the candidate word is set as the maximum output probability of the GMM acoustic model of non-stationary noise. In comparison, if the value of b _i (x _t ) is larger, it is used as it is. If the maximum output probability of the nonstationary GMM acoustic model is larger, the output probability is used as the output probability b _i (x _t ). did. The reason why such an algorithm is used is to suppress the rank fluctuation caused by the presence of non-stationary noise. That is, in a frame in which the influence of non-stationary noise is significant, the output probability b _i of the HMM acoustic model of the correct word used in the calculation of Expression (1-4) in Step 1.1.2 of the conventional Viterbi algorithm described above. This is to prevent (x _t ) from becoming a remarkably small value.

  The Viterbi algorithm of the present invention has a noise GMM acoustic model likelihood calculation process with respect to a characteristic parameter, which is proportional to the number of non-stationary noise GMM acoustic models, as compared with the conventional Viterbi algorithm, and Expression (2-4) And the calculation process according to Formula (2-5) only increases.

FIG. 1 shows a minimum configuration of a Left-to-Right HMM acoustic model to which the present invention is applied. The minimum configuration of the Left-to-Right HMM acoustic model is composed of two states i-1 and i, and the probability of transition from state i-1 to state i is given by _ai- 1i. The probability of self-looping from i to state i is given by a _ii . In addition to the output probability b _i (x), the state i has the output probability c _i (x) of the noise GMM acoustic model by the number K (K> 0) of the noise GMM acoustic model. The forward probabilities f (i-1, t-1) and f (i, t-1) in state i-1 and state i at time t-1 are sequentially calculated by the Viterbi algorithm of the present invention described above. It is assumed that
At this time, as shown in FIG. 2, the forward probability f (i, t) in the state i at the time t is expressed by the equations (2-3), (2-4), (2) in the Viterbi algorithm of the present invention. 2-5) and (2-6)

Will be required.
The functional configuration of the speech recognition apparatus 10 shown in FIG. 3 is realized by the software and hardware described above included in the speech recognition apparatus 10. As shown in the figure, the speech recognition apparatus 10 includes a first storage unit 11, a second storage unit 12, a first calculation unit 13, a second calculation unit 14, a selection unit 15, And a collation unit 16.
The first storage unit 11 includes a voice HMM acoustic model stored in a ROM, RAM, or hard disk device. In the first storage unit 11, an HMM acoustic model created in advance is input from the outside via an input / output interface and stored.

  The second storage unit 12 includes a non-stationary noise GMM acoustic model stored in the hard disk device. In the second storage unit 12, a GMM acoustic model of noise created in advance is input from the outside via an input / output interface and stored. In the present embodiment, a GMM that is generally used as an acoustic model of non-stationary noise is used. As the noise GMM acoustic model used here, for example, one noise GMM acoustic model created by mixing a plurality of types of noise, or superimposing a voice with a predetermined signal-to-noise ratio smaller than 1 is superimposed on the noise. It may be a GMM acoustic model of the created noise. Further, as the other noise GMM acoustic models, the noise GMM acoustic models are classified based on the mutual distance between the noise GMM acoustic models, and the noise GMM acoustic model is created for each classification. A plurality of noise GMM acoustic models recreated based on the noise may be used. Here, the mutual distance is a value indicating a relative perspective relationship between GMM acoustic models of noise, and will be described in detail later. Which noise GMM acoustic model is used for speech recognition is determined based on the balance between the required amount of calculation processing and memory and the recognition accuracy.

The first calculation unit 13, the second calculation unit 14, the selection unit 15, and the collation unit 16 are realized by the CPU of the speech recognition apparatus 10 executing a program in which the Viterbi algorithm of the present invention is described. It is a function.
The first calculation unit 13 calculates an output probability using the HMM acoustic model of speech stored in the first storage unit 11 and the feature parameters of speech to be recognized. At this time, the first calculator 13 calculates an output probability for the feature parameter of the frame for each frame of the speech to be recognized and for each state of the HMM acoustic model of the speech.

The second calculation unit 14 calculates an output probability for each frame using the noise GMM acoustic model stored in the second storage unit 12 and the feature parameter of the speech to be recognized. At this time, when a plurality of noise GMM acoustic models are stored in the second storage unit 12, the second calculation unit 14 calculates an output probability for each noise GMM acoustic model.
The selection unit 15 has the maximum output probability among the output probability of the HMM acoustic model calculated by the first calculation unit 13 and the output probability of the GMM acoustic model of each noise calculated by the second calculation unit 14. Select. This selection process is performed for each frame and for each state of the speech HMM acoustic model.

  The function provided in the selection unit 15 will be described in more detail. In the present embodiment, the selection unit 15 includes a first selection unit 151 and a second selection unit 152. The 1st selection part 151 performs the arithmetic processing according to Formula (2-4) among the Viterbi algorithms of this invention mentioned above. Specifically, when a plurality of GMM acoustic models of noise are stored in the second storage unit 12, the first selection unit 151 includes a plurality of values calculated by the second calculation unit 14 for each frame. The output probabilities of the GMM acoustic model of the noise are compared with each other, and the maximum output probability is selected as the maximum output probability of the noise.

  The 2nd selection part 152 performs the arithmetic processing according to Formula (2-5) among Viterbi algorithms of this invention. Specifically, the second selection unit 152 selects the output probability calculated by the first calculation unit 13 and the first selection unit 151 for each frame and for each state of the speech HMM acoustic model. The maximum output probability of the generated noise is compared, and the larger output probability is selected.

The collation unit 16 performs collation processing by using the output probability for each frame and each state selected by the selection unit 15 in the conventional Viterbi algorithm, and outputs a speech recognition result. Specifically, after replacing b _i (x _t ) in Expression (1-4) of the conventional Viterbi algorithm with the output probability selected by the selection unit 15, the conventional Viterbi after Expression (1-5) is used. Perform operations according to the algorithm. If b _i (x _t ) and the output probability of the HMM acoustic model of the speech selected by the selection unit 15 are equal, it is not necessary to replace b _i (x _t ) with the output probability selected by the selection unit 15. Good.

  As described above, the function of the speech recognition apparatus 10 according to the present embodiment is as follows for each noise GMM acoustic model stored in the second storage unit 12 for each frame as compared with the conventional speech recognition apparatus. The second calculation unit 14, the first selection unit 151, and the second selection unit 152 that calculate the output probability for the feature parameter are added, and the other functional configurations are the same as those of the conventional speech recognition apparatus. It is.

Since the speech recognition apparatus 10 only needs to learn a GMM acoustic model of noise, the amount of processing and memory required for learning the acoustic model is much smaller than that of the conventional method. Further, since the speech recognition apparatus 10 only has to additionally hold a GMM acoustic model of noise at the time of recognition, the amount of memory required at the time of recognition is greatly reduced. Further, when the speech recognition apparatus 10 is brought into a new environment, the speech recognition apparatus 10 only needs to learn a GMM acoustic model of noise in the environment and additionally accumulate it in the second storage unit 12. It is easy.
Next, the conventional method 2 having the smallest data size for acoustic model creation, the acoustic model size, the acoustic model creation time and the matching time by the Viterbi algorithm, the conventional method 4 having the highest recognition performance, and the present invention. By comparing the experimental results with such an example, the effect of the example according to the present invention is demonstrated.

As an acoustic model of speech used for collation processing by the Viterbi algorithm,
A) HMM acoustic model of speech generated from learning speech data obtained by applying a known spectral subtraction method to clean human speech (acoustic model in conventional method 2)
B) The sound of placing a coffee cup on a coffee dish (hereinafter referred to as “cup sound” or “Cup”), the sound of walking on the floor of the floor with slippers (hereinafter referred to as “slipper sound” or “Surippa”) and SNR (Signal to Noise Ratio) −20 dB, which is a signal-to-noise ratio of human clean speech less than 1 with respect to the sound of opening and closing the shutter (hereinafter referred to as “shutter sound” or “Amado”) HMM acoustic model (acoustic model in the conventional method 4) created from the speech data for learning subjected to the spectral subtraction method and superposed with SNR and SNR-10 dB
Two kinds of were made. Here, the definition of SNR is as follows.
SNR = 10 * log10 (voice power / noise power)
Next, acoustic models in the first and second embodiments are created. The basis for preparation will be described below.
The mutual distance D (i, j) between the GMM model i and the GMM model j is defined by the following equation.

Here, μ (i, l, m), σ (i, l, m), and p (i, l, m) are the mean value and standard deviation of the m-th normal distribution of dimension l of GMM model i, respectively. Value, weight. M _i and M _j represent the number of normal distributions of the GMM models i and j, respectively. L represents the number of dimensions of the GMM model.
At this time, if the mutual distance between the GMM models is defined as in the above equation, the Sammon method (Jon W. Sammon, JR., "A Nonlinear Mapping for Data Structure Analysis") , "IEEE Trans. Computers, Vol. C-18, No. 5, May 1969.), a relative perspective relationship between a plurality of GMM models can be projected onto a two-dimensional plane. When the GMM has a single normal distribution, the above mutual distance formula is simplified as follows.

  At this time, HMM acoustic model of 25 phonemes of males and female by single normal distribution created from speech data of 25 Japanese phonemes (5 vowels, 2 semi-vowels, 18 consonants) HMM acoustic model of 25 phonemes, GMM acoustic model of 40 wild bird calls with a single normal distribution created from 40 wild bird call data, and cup sounds and slippers recorded in the house As shown in FIG. 4, a GMM acoustic model of 33 noises created from 33 types of non-stationary noise data including sound and shutter sound is expressed by a two-dimensional plane as shown in FIG. 4 using Equation (6) and the known Sammon method. Projected into a shape. Here, the HMM acoustic model of Japanese men's Japanese phonemes is ■, the HMM acoustic model of Japanese women's Japanese phonemes is □, and the GMM acoustic model of wild birds singing is ×, which is unsteady noise. Each GMM acoustic model is represented by ▲.

  As shown in the projection results of FIG. 6, the non-stationary noise and the speech are clearly separated with a bird cry between them, and the 33 types of non-stationary noise form a cluster. Therefore, even if a GMM acoustic model with a small number of noises is created using all 33 types of non-stationary noise, it is considered that the degree of separation from the HMM acoustic model of speech is high.

Therefore, in addition to the above A) and B), the following acoustic models C) and D) were created.
C) (Acoustic model in the first embodiment according to the present invention)
(1) HMM acoustic model of speech created from learning speech data obtained by applying spectral subtraction to clean human speech (2) 33 unsteady types including cup sound, slipper sound and shutter sound recorded in a house One GMM acoustic model of noise with a single normal distribution created using all noise data

D) (Acoustic model in the second embodiment according to the present invention)
(1) HMM acoustic model of speech created from learning speech data obtained by applying spectral subtraction to clean human speech (2) 33 unsteady types including cup sound, slipper sound and shutter sound recorded in a house One GMM acoustic model of noise with a single normal distribution created by using all of the noise data superimposed with SNR-20dB, which is a signal-to-noise ratio of less than 1 on human clean speech (3) In a house A single sample created using all of the 33 types of non-stationary noise data including cup sounds, slipper sounds, and shutter sound recorded, and superimposing human clean speech with a signal-to-noise ratio of less than 1 at SNR-10 dB. One GMM acoustic model of noise with a normal distribution

  Next, the cup sound, slipper sound, and shutter sound recorded in the house in the voice data of the place name 100 words by 15 males and 15 females who were not used for the HMM acoustic model creation of the above sound, SNR 20 dB and SNR 10 dB The two data superimposed on each other were used as evaluation data for speech recognition.

  FIG. 5 shows an HMM acoustic model of speech in the case of collation using the conventional Viterbi algorithm (conventional method 2) using the speech HMM acoustic model of A) in the word recognition task of 100 place names. When using the Viterbi algorithm according to the present invention using the conventional Viterbi algorithm (conventional method 4) and using the HMM acoustic model of speech and the GMM acoustic model of noise (first implementation) Comparison of recognition performance of evaluation data according to the Viterbi algorithm of the present invention using the HMM acoustic model of speech and the noise GMM acoustic model of D) above (second example) for each SNR (Example) SNR 20 dB and SNR 10 dB).

  From this result, in the case of a cup sound, both the first and second embodiments according to the present invention have a recognition performance that is superior to that of the conventional method 2. In addition, the first and second embodiments of the present invention had the same recognition performance as the conventional method 4, but as shown in the comparison table of FIG. There is an advantage that the size, the acoustic model size, the acoustic model creation time, and the matching time by the Viterbi algorithm are significantly less than the conventional method 4.

Further, in the case of slipper sound and shutter sound, both the first and second embodiments according to the present invention have significantly higher recognition performance than the conventional method 2 and the conventional method 4.
Therefore, it has been clarified that the Viterbi algorithm of the present invention is overwhelmingly superior to the presence of non-stationary noise such as cup sound, slipper sound, and shutter sound compared with the conventional Viterbi algorithm.

  Next, a noise GMM acoustic model created in the third embodiment will be described. A GMM acoustic model of noise with a single normal distribution is created for each of 33 types of non-stationary noise data including cup sound, slipper sound and shutter sound recorded in a house, and the equation (6) and the known Sammon method are used. Each of the 33 types of non-stationary noise data is projected onto a two-dimensional plane as shown in FIG. The noise GMM acoustic model located at the center of FIG. 7 is relatively close to other noise GMM acoustic models, and the noise GMM acoustic model located at the periphery of FIG. 7 is relatively similar to other noise GMM acoustic models. It ’s far away. Therefore, as shown in FIG. 7, the following acoustic model was created based on the noise GMM acoustic model divided into two regions inside and outside the circle and located in each region.

E) (Acoustic model in the third embodiment according to the present invention)
(1) HMM acoustic model of speech created from learning speech data obtained by performing spectral subtraction on human clean speech (2) GMM acoustic model of noise located in each region shown in FIG. The noise data (33 kinds of unsteady noise data including cup sound, slipper sound, and shutter sound recorded in the house) used in creating the sound is a signal-to-noise ratio smaller than 1 for human clean speech. Two GMM acoustic models of noise with a single normal distribution created from data superimposed at SNR-20 dB (3) When creating a GMM acoustic model of noise located in each of the two regions shown in FIG. SNR, which is a signal-to-noise ratio that is less than 1 for human clean speech in the noise data (33 types of non-stationary noise data including cup sounds, slipper sounds, and shutter sound recorded in a house) Created from superimposed data 10 dB, 2 pieces noise GMM acoustic model with a single normal distribution

  Next, SNR 20 dB and SNR 10 dB are recorded as the cup sound, slipper sound and shutter sound recorded in the house in the sound data of the place name 100 words of 15 males and 15 females who were not used for the HMM acoustic model creation of the above voice. The acoustic models superimposed on each were used as evaluation data.

  FIG. 8 shows an HMM acoustic model of speech in the case of collation with the conventional Viterbi algorithm using the HMM acoustic model of speech of A) in the word recognition task of 100 place names (conventional method 2) and B). When using the conventional Viterbi algorithm (conventional method 4) and using the HMM acoustic model of speech and the noise GMM acoustic model of D) to collate with the Viterbi algorithm of the present invention (second implementation) Comparison of recognition performance for evaluation data in the case of matching with the Viterbi algorithm of the present invention using the HMM acoustic model of speech and the noise GMM acoustic model of E) above (Example 3) for each SNR (SNR 20 dB and SNR 10 dB).

  From this result, in the case of a cup sound, the second and third embodiments according to the present invention have a significantly higher recognition performance than the conventional method 2. Further, when compared with the conventional method 4, the second embodiment and the third embodiment have similar recognition performance, but as shown in FIG. 6, the data size for creating the acoustic model, the acoustic model size There is an advantage that the acoustic model creation time and the matching time by the Viterbi algorithm are significantly less than the conventional method 4.

Further, in the case of slipper sound and shutter sound, both the second and third embodiments according to the present invention have a significantly higher recognition performance than the conventional method 2 and the conventional method 4.
Accordingly, it has been clarified that the Viterbi algorithm of the present invention is overwhelmingly superior to the conventional Viterbi algorithm with respect to the presence of non-stationary noise such as cup sounds, slipper sounds, and shutter sound.

  Note that the region division of 33 types of non-stationary noise data projected in a two-dimensional plane is not only divided into the inner side and the outer side as shown in FIG. 8, but the outer region is further divided as shown in FIG. Needless to say, it may be divided into two regions, and the number of divisions may be increased as appropriate. As the number of divisions increases, a noise GMM acoustic model based on a set of noises close to each other can be created, and speech recognition accuracy can be improved.

As described above, when the Viterbi algorithm of the present invention is used, when the influence of non-stationary noise is significant, it is possible to “pass through” without causing the order fluctuation that frequently occurs in the conventional Viterbi algorithm, and the accuracy of speech recognition is improved. It can be prevented from lowering.
Furthermore, an advantage of the present invention is that it only needs to have a GMM acoustic model of noise for each group of non-stationary noise. For example, in the case of constructing a Japanese speech recognition system by modeling 54 phonemes shown in FIG. 9 with HMM, in addition to 54 clean phoneme HMM acoustic models created from clean speech, If there are 33 types and the number of corresponding GMM acoustic models of noise is 33, a total of 87 noise superimposed phoneme HMM acoustic models may be created. Furthermore, if 33 types of noise are grouped into two types, a total of 56 noise superimposed phoneme HMM acoustic models may be created. For this reason, it is possible to reduce the required memory amount and calculation amount.

  FIG. 6 shows a comparison table between the present invention and the conventional method. Here, it is assumed that a Japanese speech recognition system based on the 54 phonemes shown in FIG. 9 is constructed. There are 33 types of noise. The data size for acoustic model creation, the acoustic model creation time, and the collation time by the Viterbi algorithm are set to 1 in the case of the conventional method 2, and are expressed as a magnification with respect to it. The acoustic model size is represented by the number of HMM acoustic models to be created. In parentheses, the magnification and the number of the HMM acoustic models when 33 types of noise are grouped into two types are shown.

  As shown in this comparison table, the present invention has a problem that the conventional method 1, the conventional method 3 and the conventional method 4 have, that is, the data size for creating the acoustic model and the acoustic model size become enormous with the number of types of noise. There is no malfunction. Similarly, the present invention has no inconvenience that the conventional method 1, the conventional method 3, and the conventional method 4 have an enormous time for making an acoustic model and a matching time by the Viterbi algorithm with the number of types of noise.

In addition, the present invention has an effect that robustness against non-stationary noise is high as compared with the conventional method 1, the conventional method 2, the conventional method 3, and the conventional method 4.
In addition, compared with Patent Documents 1, 2, and 3, the present invention has a problem that the data size for creating an acoustic model and the acoustic model size become enormous even if the number of types of noise assumed increases, and the acoustic model creation time. And a problem that the verification time by the Viterbi algorithm becomes enormous.
In the above description, the embodiment in which the GMM acoustic model of noise with a single normal distribution is applied to the present invention has been described. However, the GMM acoustic of noise with a mixed normal distribution having higher accuracy than the noise GMM acoustic model with a single normal distribution has been described. Needless to say, the model may be applied to the present invention.

  As described above, the speech recognition apparatus 10 can suppress candidate rank variation due to the presence of noise by using the Viterbi algorithm of the present invention, and realizes speech recognition that is robust against non-stationary noise. be able to. Furthermore, since speech recognition can be realized using a small number of GMM acoustic models, it is advantageous in terms of calculation amount and memory amount. As described above, the speech recognition apparatus 10 can realize highly accurate speech recognition while suppressing the amount of processing and the amount of memory in an environment where noise is generated.

  The present invention can be used for efficient and highly accurate speech recognition particularly in an environment where non-stationary noise occurs.

It is a figure for demonstrating the Left-to-Right HMM acoustic model which concerns on embodiment of this invention. It is a figure for demonstrating the Viterbi algorithm which concerns on the same embodiment. It is a figure which shows the structure of the speech recognition apparatus which concerns on the same embodiment. It is a figure which shows a mode that the HMM acoustic model of the human voice, the GMM acoustic model of the bird cry, and the GMM acoustic model of the noise were two-dimensionally projected. It is a figure which shows the performance comparison with the conventional method and the Example which concerns on this invention. It is a comparison table between the conventional method and the present invention. It is a figure which shows a mode when the GMM acoustic model of noise is mapped to a two-dimensional plane. It is a figure which shows the performance comparison with the conventional method and the Example which concerns on this invention. It is a figure for demonstrating the kind of HMM acoustic model of a phoneme. It is a figure which shows the example of the area | region division of the GMM acoustic model of the noise mapped on the two-dimensional plane. It is a figure for demonstrating the conventional Left-to-Right HMM acoustic model. It is a figure for demonstrating the conventional Viterbi algorithm. It is a figure which shows the example of the waveform of non-stationary noise superimposition audio | voice. It is a figure which shows the order fluctuation | variation at the time of non-stationary noise superimposition voice collation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Speech recognition apparatus 11 1st memory | storage part 12 2nd memory | storage part 13 1st calculation part 14 2nd calculation part 15 Selection part 151 1st selection part 152 2nd selection part 16 Collation part

Claims (7)

In a speech recognition device that performs speech recognition using a Viterbi algorithm for speech input to the device,
First storage means for storing an HMM (Hidden Markov Model) acoustic model of speech;
A second storage means for storing a noise GMM (Gaussian Mixture Model) acoustic model;
Output of the speech HMM acoustic model for the speech feature parameter to be recognized for each predetermined frame of the speech input and for each state of the speech HMM acoustic model stored in the first storage means A first calculating means for calculating a probability ;
Second calculation means for calculating an output probability of a GMM acoustic model of noise stored in the second storage means for the feature parameter for each predetermined frame ;
Output probability calculated by said first calculating means for each said predetermined frame, and from among the calculated output probabilities by said second calculating means, selection means for selecting the largest output probability,
Used in the Viterbi algorithm output probabilities selected by the selection means as the output probability of the speech HMM acoustic model in the frame in which the output probability is selected, the matching performing voice recognition of the voice to be the input by using the Viterbi algorithm And a voice recognition device. The selection means includes
First selection means for selecting the maximum output probability among the output probabilities calculated by the second calculation means as the maximum output probability of noise;
A second selection unit that selects a larger output probability among the output probability calculated by the first calculation unit and the maximum output probability of the noise selected by the first selection unit; The speech recognition apparatus according to claim 1, wherein The speech recognition apparatus according to claim 1, wherein the second storage unit stores one GMM acoustic model of noise created by mixing a plurality of types of noise. The speech according to claim 1 or 2, wherein the second storage means stores a GMM acoustic model of noise created by superimposing speech on noise at a predetermined signal-to-noise ratio. Recognition device. In the second storage means,
Classifying the GMM acoustic model of the noise based on the mutual distance between the GMM acoustic models of the noise;
The two or more GMM acoustic models of noise that are recreated based on the noise data that is the basis for creating the GMM acoustic model of the noise are stored for each classification. 5. The speech recognition device according to any one of 4 above. In a speech recognition method that performs speech recognition using the Viterbi algorithm,
Each predetermined frame of speech to be recognized, and, for each state of the speech HMM acoustic model, a first calculation step of calculating the output probability of the speech HMM acoustic model for the feature parameters of the speech to be the recognition target ,
A second calculation step of calculating an output probability of a noise GMM acoustic model for the feature parameter for each predetermined frame ;
Wherein said first calculated output probabilities in the calculation step for each predetermined frame, and from among the calculated output probabilities in the second calculation step, a selection step of selecting the maximum output probability,
The output probability selected in the selection step is used in the Viterbi algorithm as the output probability of the HMM acoustic model of speech in the frame in which the output probability is selected , and speech recognition of the speech to be recognized is performed using the Viterbi algorithm. A voice recognition method comprising: a collating step to perform . On the computer,
Each predetermined frame of speech to be recognized, and, for each state of the speech HMM acoustic model, a first calculation step of calculating the output probability of the speech HMM acoustic model for the feature parameters of the speech to be the recognition target ,
A second calculation step of calculating an output probability of a noise GMM acoustic model for the feature parameter for each predetermined frame ;
Wherein said first calculated output probabilities in the calculation step for each predetermined frame, and from among the calculated output probabilities in the second calculation step, a selection step of selecting the maximum output probability,
The output probability selected in the selection step is used in the Viterbi algorithm as the output probability of the HMM acoustic model of speech in the frame in which the output probability is selected , and speech recognition of the speech to be recognized is performed using the Viterbi algorithm. program for executing a matching step of performing.

Images