JP3029654B2 - Voice recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Purpose of the Invention] (Industrial application field) The present invention relates to a speech recognition apparatus capable of effectively improving the recognition performance of word speech and continuous speech in practical use.

(Prior Art) Speech recognition is one of the important techniques for realizing a man-machine interface by a natural method, and various researches and developments have been made conventionally.

However, when a human inputs a voice to a machine (speech recognition device), if he / she utters freely like a natural conversation between a normal human and a human, "e ~", "n ~", " Speaking that is not directly related to the user's intention, such as interjections or tongue tapping, is performed. Such an utterance component other than voice representing linguistic information intended by the user is defined as an unnecessary word. It is known that if this unnecessary word is added before or after a word or sentence voice to be input, the recognition rate in current speech recognition is significantly reduced. Since this is an utterance that is not a recognition target, there are few examples of performance evaluation at the laboratory level, and even a discrete word speech recognizer is one of the serious problems when it is put to practical use. In other words, before and after the word that the user tries to input, "good stagnation" and unnecessary non-verbal voices that are unnecessary due to inexperience with the system are often added.
It becomes difficult to detect the start point and end point (word boundary) of speech. The same applies to phrase units, phrases, or continuous utterances.

The word spotting method has been proposed as one of the effective means for dealing with such a problem. In this method, after analyzing the input voice, the presence of a word is always checked at all times without detecting a word boundary as in the conventional method. For this reason, a huge amount of input voice patterns and pattern matching between the recognition dictionary are required as compared with the conventional method, but the power of the computer is increased, and it becomes possible to realize even inexpensive LSIs, workstations, and personal computers. Was.

As a method for performing speech recognition in a noise environment stably and with high accuracy, a noise immune learning method has been developed in which learning is performed by artificially adding noise to the above-described word spotting method. This method is different from the conventional word spotting method in that continuity of pattern matching (word spotting) is performed not only at the time of recognition but also at the time of learning. It requires a huge amount of computation, but even without real-time processing, it controls the noise environment and gradually lowers the signal-to-noise ratio (S / N) to simulate superimposed noise. By using this pattern, it has become possible to perform speech recognition that is resistant to noise.

However, the unnecessary words targeted by the present invention, that is, careless utterances, non-verbal voices, fuzzy words, tongue tapping, in other words, speech information that is unnecessary for the sentence actually required are not effective. Even if the spotting method is used, voice input cannot be performed easily.

(Problems to be Solved by the Invention) As described above, in the conventional speech recognition method, a user does not intend to input an utterance, that is, an unnecessary word added before or after a word or sentence voice to be input. Measures have not been taken sufficiently. This hinders the practical use of voice-based dialogue, which is the most natural means of human communication.

The present invention has been made in view of the above, and it is an object of the present invention to provide an utterance component (interjection, tongue tapping, Iodomi, It is an object of the present invention to provide a robust means for improving recognition performance degradation due to non-verbal speech.
Thus, the burden on the user is reduced, and voice input can be easily performed even when the user speaks more naturally. For this reason, it is possible to smoothly and efficiently perform the dialog between the human and the computer.

[Configuration of the invention]

(Means for Solving the Problems) In order to achieve the above object, the present invention artificially generates a voice when an unnecessary word (inadvertent utterance) is added, and uses the generated voice as learning voice data. In addition, learning for improving word spotting performance is performed.

At this time, there are various variations of the unnecessary word, and there are various words depending on individuals, cultures, ethnicities, and regions. That is, in the conventional speech recognition, the speech content can be described as a speech language, but the scope of the speech targeted in the present invention is very wide, and it is diversified, and the duration and the unnecessary words should be recognized. It is also necessary to consider the interval between utterances and the like when generating the learning speech data to which the above-mentioned unnecessary words are added.

Therefore, the speech recognition apparatus used in the present invention is intended to reduce erroneous recognition due to addition of an unnecessary word and erection (insertion, addition) of an erroneous word on the premise of the word spotting method. In this method, a huge amount of learning data is generated by using a pure speech database and an unnecessary word database, and a word spotting process is performed using the synthesized data by utilizing a simulation capability and a computer power of a computer. Thus, learning data is extracted, and a recognition dictionary for word spotting is generated.

(Operation) According to the present invention, the speech recognition device operates stably and obtains robustness even in the case of a user's utterance to which unnecessary words have been added, which could not be resisted in the past, so that the user can naturally, smoothly and comfortably. It is possible to interact with the computer without disturbing thinking. Therefore, the burden on the user is reduced compared to the conventional speech recognition application, and the application to a dialog system, a data entry system, and the like is expanded.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings. First, prior to the description of the system, unnecessary words will be briefly described.

FIG. 1 shows a voice pattern to which an unnecessary word is added. In addition to “Ah ~ Sapporo” and “Sapporo” intended by the user, an unnecessary word “Ah ~” is added. Moreover, the time interval between "Ano" and "Sapporo" is shorter than the time length of the prompting sound of "Sapporo", and not only does the beginning and end of the voice become unclear, but also "Ano" and "Sapporo"
"Asa" combining the "sa" part of "" is regarded as a single word, which may cause erroneous recognition.
That is, various unnecessary words that may be added before and after have a great effect on speech recognition.

FIG. 2 is a schematic configuration diagram of a speech recognition device according to the present invention. This device consists of a speech analysis unit that performs speech analysis, a recognition unit that includes a word spotting unit that performs word spotting processing, and a learning unit that learns a dictionary for word spotting to operate stably for unnecessary words. Have been. The operation of the present apparatus is performed in either the recognition mode or the learning mode.

As described above, the processing of unnecessary words uttered unintentionally by the user when the speech recognition device is put into practical use is central to the present invention, but the types of unnecessary words and the utterance method are diverse, and there are many variations. is there. In addition, it is difficult to express cases that do not express any linguistic content, interjections, etc., using phoneme sequences used in general speech recognition methods or as limited categories. There was a problem that it could not be handled. The present invention focuses on the simulation function of the computer, artificially simulates the situation where unnecessary words are added before and after the intended speech, performs learning processing, and performs recognition using word spotting. A recognition dictionary that operates strongly and stably for unnecessary words is automatically generated.

Next, a speech recognition system will be described. First, as shown in FIG. 2, the system includes a speech recognition device including a recognition unit and a learning speech recognition device including a learning unit necessary for learning. Therefore, learning will be described first.

As shown in FIG. 2, a large number of word voices to be recognized in the course of recognition are collected in advance as learning voice data and held as a voice database 23, and unnecessary word voices are separately collected. Prepare in advance. The system is in a learning mode, and the mode switch 19 is connected to a lower terminal.

Then, the audio data synthesizing unit 25 synthesizes the audio data in which the unnecessary words to be provided for the learning process are added along with one or both of the words before and after the recognition target word. Points to keep in mind at this time are:
Either control the utterance of the recognition target word and the unnecessary word so that they do not overlap, or if the unnecessary word may be added only before the intended word, synthesize only the words that meet the conditions. To do it. "that~",
“Eto” and the like are unnecessary words added only before, and conversely, words added only after are similarly added only to the rear part of the recognition target word to synthesize the learning speech data.

Further, it is effective to synthesize the learning data by controlling the interval between the unnecessary word and the recognition target word by using the knowledge obtained by observing the dialogue by voice in advance. For example, if "ano-" is added when trying to enter the city name "Sapporo", "ano-" is added before "Sapporo" as shown in FIG. Is changed from -1.5 seconds to -0.2 seconds to synthesize the learning voice data. That is, if the utterance "Sapporo" is _Si and "Ano" is _Ni , the synthesized data
S _i ′ is as follows.

S _i ′ = S _i + αN _ip Here, a gain constant for setting an utterance level when α is superimposed, and p is a variable for defining a time axis. In the example described above, the range from -1.5 seconds to -0.2 seconds is appropriately selected to synthesize the learning voice data.

When performing the above-described processing, it is necessary to have knowledge for synthesizing voice data to which unnecessary words are artificially added. However, simple processing as described above may be used, but performing detailed simulation is also effective. It is a target.

Next, an embodiment of the learning process using the voice data obtained by the synthesis will be described.

A feature of the present invention is that the learning processing unit can use information on the positional relationship between the word voice and the unnecessary word voice on the time axis in order to perform processing for artificially adding unnecessary words. This is significantly different from the learning process for speech recognition using the speech data collected in step (1). In a speech recognition device based on the word spotting method with noise immunity, learning was performed by controlling the level of word speech and the level of noise, that is, the signal-to-noise ratio (S / N). Is characterized in that

The speech data obtained by the above-described synthesis processing is subjected to word spotting processing in the word spotting section 22 during learning. The method at this time is basically based on a learning type word spotting method. That is, even in the learning mode, the recognition processing is performed on the synthesized voice data using the word spotting method.

Various methods can be applied to the word spotting pattern matching at this time. Here, an example of a composite similarity that is strong against pattern deformation will be described.

In the recognition in the learning mode, as shown in FIG. 6, the composite similarity value S _ij exceeds a certain threshold with respect to the starting end i and the ending candidate point j in the starting end candidate section in the word to be recognized.
If the maximum value MAXS _ij at a certain interval, and outputs the word as a recognition result.

That is, the learning feature vector extraction unit in FIG.
At 26, when the word recognition result for the input word A is correct in the learning mode as shown in FIG. 7, the maximum similarity and the maximum similarity of the input word match, and the starting point t _s indicating the maximum similarity is shown. ^A word feature vector corresponding to ^A and terminal t _e ^A Is used as a learning word feature vector in the cutout learning process.

On the other hand, when the word recognition result is incorrect, the maximum similarity and the maximum similarity of the input word A are different, and the word feature vector corresponding to the start t _s ^A and the end t _e ^A that gives the maximum similarity for the input word A With cut Day as a word feature vector for learning, t _s ^B and word feature vector corresponding to the end t _e ^B became maximum similarity of words led to erroneous recognition B (word indicating the maximum similarity) Use for learning. This is shown in FIG.

Based on the learning feature vectors extracted in this way, the recognition dictionary creation unit 27 creates a recognition dictionary based on the composite similarity method. The recognition ability is improved, and the rejection ability for unnecessary words is improved. The key is learning to increase the stability and operate stably. The dictionary created by the recognition dictionary creating unit 27 is stored in the recognition dictionary 28, and a dictionary that is strong against unnecessary words is constructed.

A speech recognition device using the recognition dictionary 28 constructed in the learning mode as described above will be described.

The device in the present embodiment is the same as that shown in FIG.
Voice input unit 20, voice analysis unit 21, word spotting unit 22
And the recognition dictionary 28. The voice input unit 20 converts a voice signal input via a microphone or the like into a digital signal and provides an output to the voice analysis unit 21.

The audio input unit 20 includes, for example, a low-pass filter that removes a frequency component of 5.4 KHz or more and an A / D converter that converts a digital signal with a sampling frequency of 12 KHz and a quantization bit number of 16 bits. In the next voice analysis unit 21, the time window length is 24 ms for the digitized input voice signal.
Discrete Fourier Transform using FET Hamming Window
(Fast Fourier Transform) algorithm is used.
Then, the power spectrum obtained by the FET is smoothed and logarithmic processing is performed to obtain a 16-dimensional frequency spectrum in the frequency direction. This process sets the analysis frame period to 8m
s., which represents the frequency spectrum every 8 ms.
The dimensional feature parameters are obtained.

The voice analysis method at this time is not limited to the FET analysis described above, and other analysis processing methods such as LPC analysis, cepstrum analysis, and filter analysis can be used.

The word spotting unit 22 uses the above-described speech feature parameters to make the start and end non-fixed, and performs pattern matching with the recognition dictionary continuously on the time axis. In this case, the frequency direction using the characteristic parameters of the 16-dimensional, the word feature vector used in pattern matching assuming starting t _s and the end t _e of the word, on the time axis obtained by equally dividing the two end points
We use 12 voice feature parameters. That is, a 16 × 12 = 192 dimensional vector with 16 dimensions in the frequency direction and 12 dimensions in the time axis direction Is used as a word feature vector, and pattern matching with the recognition dictionary 28 is performed continuously every frame period (for example, 8 ms.).

The above-mentioned word feature vector used here is a time-frequency spectrum, and it has already been obtained that the word spotting based on the compound similarity method based on this is stable with respect to noise. In the word spotting section 22, a plurality of start points satisfying a certain voice section condition are provisionally set based on the provisionally set end point, and a plurality of word feature vectors are determined from the provisional voice section determined between these start end points. Then, the extracted data is compared with a dictionary to determine a similarity value. This process is performed while shifting in the time axis direction for each frame cycle.

Then, the similarity values calculated for each feature vector are compared, and the recognition target category that has obtained the maximum similarity is:
The voice section (start / end information) is obtained as a recognition result for the input voice. According to the above speech recognition processing by word spotting, since the start and end detection is not performed in advance, the problem caused by the conventional detection error of the start and end is a problem.
Can be reduced. For this purpose, an operation for performing the similarity calculation many times without defining the start and end is required,
The effect obtained as a performance improvement is great.

Next, an example will be described in which a recognition dictionary that can actively deal with unnecessary words by a learning process is created and the performance for unnecessary words is dramatically improved.

In the present embodiment, in the learning process in the learning mode,
As described above, the unnecessary speech data is added, the learning speech data is synthesized, the word spotting process is performed, and the training speech feature vector is extracted in the training patent vector extraction unit 26. Is automatically extracted.

In the case of creating a recognition dictionary based on the composite similarity method in the recognition dictionary creation unit 26, learning is performed via a covariance matrix K.

Where the covariance matrix of the category Is defined as follows:

Here, the category name, t represents transposition.

Eigenvalue λ _k ⁽⁾ obtained by KL expansion (principal component analysis) of the covariance matrix for each category, eigenvector Is obtained and used for pattern matching.

Now, the learning voice data automatically extracted from the learning voice data with unnecessary words added before and after Using By this process, the covariance matrix K is sequentially updated, and the learning dictionary is enriched by the learning process. Where α
Represents a coefficient for updating.

When updating the covariance matrix in the recognition dictionary creation unit 27,
Recognition result information described above (correct / false determination result and start / end information)
It is effective to use the learning word feature vector obtained for the input voice of category A. Is used to perform a learning process as shown in FIG. By this processing, the reject function can be enhanced. That is,
When the category of the input voice is A and it is mistakenly recognized as B due to the unnecessary word It is effective to update the recognition dictionary by performing a learning process by using. In particular, the synthesis of artificial learning speech patterns
It is particularly important to set the condition close to the actual use environment.

For example, unnecessary words can be considerably “habit” depending on the speaker.
It is effective to tune for a specific speaker because there is (mouth).

It is also effective to use it together with superimposition of noise, and it can be performed with the configuration as shown in FIG. In FIG. 2, a noise database is added. In this case, when synthesizing voice data, learning is performed by simulating both addition of unnecessary words and noise superposition.

In other words, for noise superposition, speech and noise data are superimposed on the time axis, and the S / N is gradually reduced to generate synthesized speech data. It is arranged before and after to synthesize the learning speech data.

The same applies to the learning process.
Speech recognition performance that is strong against noise and stable against unnecessary words can be obtained.

Note that the present invention is not limited to the above-described embodiment. For example, in the example, the input speech is a discretely uttered word, but it can be applied to the recognition of continuous words, phrases and speech by continuous utterance. Although the embodiment has been described, other statistical methods such as Mahalanobis distance, subspace method, neural network, and HMM (Hidden Marcov Model) can be applied.

Further, the number of dimensions of the speech feature vector used for the recognition processing is not particularly limited.

The point of the present invention is to artificially simulate a user's careless, unintentional utterance when using an actual speech recognition device as a man-machine interface, and use learning speech data to which the unnecessary words are added. In the recognition processing of the word spotting method, which is a continuous pattern matching with non-fixed start and end in the present invention.

Various modifications can be made without departing from the scope of the present invention.

〔The invention's effect〕

As described above, according to the present invention, when speech recognition by word spotting is applied, in order to eliminate unnecessary words and obtain a stable operation, learning of a speech recognition dictionary is performed by This is performed using the learning voice data artificially synthesized by adding the voice data before and after.
Therefore, there are extremely many variations
Creation of tongue tapping, exhalation sounds, interjections, etc. can be artificially simulated, so even if the recognition target word changes, use the advances in computer power that will continue to evolve in the future to create a highly accurate recognition dictionary Becomes possible.

In addition, by using it together with background noise learning by the speech recognition device, a practical and robust speech recognition device can be realized, and a man-machine interface that makes use of the naturalness of speech input becomes possible. The effect is obtained.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a voice pattern to which an unnecessary word is added.
FIG. 2 is a diagram showing a configuration according to the present invention, FIG. 3 is a diagram for explaining a simulation of adding unnecessary words for learning speech data synthesis, FIG. 4 is a diagram showing a procedure for creating a recognition dictionary, FIG. FIG. 6 is a diagram showing a configuration for performing learning on noise-resistant and unnecessary-words according to the present invention. FIG. 6 is a diagram for explaining recognition in a learning mode. FIG. 7 is a feature when a word recognition result is a correct answer. FIG. 8 is a diagram for explaining a method of extracting a vector, and FIG. 8 is a diagram for explaining a method of extracting a feature vector when a word recognition result is incorrect.

Continuation of front page (56) References JP-A-1-182898 (JP, A) JP-A-1-158498 (JP, A) JP-A-62-65093 (JP, A) JP-A-59-119400 (JP) , A) JP-A-58-23097 (JP, A) IEICE Technical Report [Voice], Vol. 89, No. 90, SP89-19, “Speech Recognition of Unspecified Speakers in Noisy Environment by Learning Type Word Spotting Method”, p51-58, (published June 23, 1989) IEICE Technical Report [ Voice], Vol. 91, No. 96, SP91-22, “Detection of words from continuous speech including unnecessary words”, pp. 33-39, (published June 21, 1991) Proceedings of the Acoustical Society of Japan Fall Meeting 2-8 -8 “After-unnecessary effects on speech recognition due to word spotting in a noisy environment”, p. 61-62 (published on September 20, 1990) Proceedings of the Acoustical Society of Japan, Spring Meeting, 1991, 2-5-5 "A Study on Word Spotting from Dialogue Speech", p. 63-64 (issued on March 27, 1991) (58) Fields surveyed (Int. Cl. ⁷ , DB name) G10L 15/06 G10L 15/20 G10L 15/10 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. A feature parameter of an input voice is obtained,
In a voice recognition device that performs similarity matching between the characteristic parameter and data of a dictionary stored in advance for recognition and outputs a recognition result for the voice based on the matching result, data to be recognized and the data A speech recognition apparatus characterized in that a word spotting process is performed using synthesized data obtained by adding unnecessary word data to the data to extract learning data from the data and use the data for learning a recognition dictionary.