KR100981540B1 - Speech recognition method of processing silence model in a continous speech recognition system - Google Patents

Abstract

The present invention relates to a speech recognition method through a silent model processing in a continuous speech recognition system, and to classify a silent model for continuous speech recognition. The present invention aims to provide a speech recognition method that can improve the recognition rate and implement a simple language model by creating a data structure with the phonemes of words belonging to.

To this end, the present invention, the speech recognition method in a continuous speech recognition system, a first step of generating a language model by defining a silent model; A second step of constructing a data structure to be used in a recognition process by composing an intermediate silence in the language model as starting phonemes of word groups in every class; A third step of recognizing a word corresponding to a start class by using the data structure; A fourth step of tracing back only the value ending with the last phoneme of the word corresponding to the end class in the recognition result and dividing it into phoneme units; And a fifth step of deleting the intermediate silence from the phoneme unit division information and verifying using the semitone phone model, and outputting assuming that only a column of the verified word is recognized as a sentence.

Silence, Language Model, Speech Recognition, Trace-back, Viterbi

Description

Speech recognition method of processing silence model in a continous speech recognition system             

1 is an exemplary configuration of a general voice recognition system.

2A and 2B are diagrams illustrating an embodiment of HMM topology of a silent model and types of silent models that may appear in a continuous speech recognition grammar used in the present invention.

3A and 3B are temporary explanatory diagrams showing a recognition class data structure generated when using a statistical language model used in the present invention.

Figure 4 is a flow diagram of an embodiment of a speech recognition method through silence model processing according to the present invention.

* Explanation of symbols on the main parts of the drawing

11: endpoint detector 12: feature extractor

13: Viterbi Explorer 14: Pronunciation Dictionary

15: Phoneme Model Database 16: Speech Verifier                 

17: semitone phone model database

The present invention provides a silent model processing that can improve speech recognition rate by assuming that the first phoneme of each word is recognized in the recognition process instead of the language model processing method used in the continuous speech recognition system. It relates to a voice recognition method through.

In general, a well-known speech recognition method uses a Hidden Markov Model (HMM). Here, the Viterbi search is performed through the speech recognition process, which compares the difference between the HMM constructed by pre-training the candidate words to be recognized and the features of the currently input speech to determine the most similar candidate word. It's a process.

For better understanding, a general speech recognition system will be described with reference to FIG. 1.

First, when a voice is input, the endpoint detector 11 searches for a voice section excluding a silent section before and after the voice. Thereafter, the feature extractor 12 extracts a feature of the speech from the speech signal of the speech section found above.                         

Next, in the Viterbi searcher 13, the words most similar to the likelihood are selected by using the speech feature values for the words registered in the pronunciation dictionary 14 composed of the phoneme model database 15.

Subsequently, the speech verifier 16 divides the feature interval into phoneme units using the word selected by the Viterbi searcher 13, and then obtains a Likelihood Ratio Confidence Score using the semi-phoneme model. .

The speech verification method is a method of determining which confidence value (Confidence Score or Confidence Measure) value is used to determine whether to accept (Accept) or reject (Reject) the recognition result. Here, reliability is a measure of how reliable the result is for the speech recognition result. If the reliability value is high, the recognition result should be accepted as reliable, and if it is low, it is difficult to trust the result. The recognition result should be rejected.

Finally, if the word is rejected, the speech verification process 16 performs the speech verification process as described above for the next candidate word.

On the other hand, in the case of recognizing a sentence, the above utterance verification process is applied in the same manner, only the grammar is added, and the sentence unit is verified.

The reliability is different from the Viterbi search result. That is, the Viterbi search result number represents a simple similarity to a word or phoneme, while the reliability is a relative value of the probability that the word is spoken from other phonemes or words for the recognized phoneme or word. Means.                         

To determine the reliability, a phone model and an anti-phone model are required.

The phoneme model is an HMM created by training extracted phonemes by extracting phonemes actually spoken from a voice. The phoneme model is a model used in a speech recognition system based on a general HMM.

The semitone phone model, on the other hand, refers to an HMM that is trained using phonemes that are very similar to actual phonemes (these are called cohort sets).

As such, in the speech recognition system, a phoneme model and a semiphoneme model exist for each phoneme used.

For example, there is a "ㅏ" phoneme model for the phoneme "ㅏ" and a semi-phoneme model for "ㅏ".

For example, the model of "ㅏ" phoneme is made by extracting only the "ㅏ" phoneme from the speech database and training it according to HMM's training method. And in order to construct a half-phoneme model for "ㅏ", we need to find a similar phoneme set for "ㅏ". This can be obtained from the phoneme recognition result. By performing the phoneme recognition process, it is possible to determine which phonemes other than "ㅏ" are misrecognized as "ㅏ" and collect them to determine a similar phoneme set for "ㅏ". In other words, if the phonemes such as "ㅓ, ㅓ, ㅕ" are misidentified as "ㅏ", they can be called similar phoneme sets, and when they are collected and subjected to HMM training, a semi-phoneme model for "ㅏ" phonemes is generated.

If a phoneme model and a semiphoneme model are generated for all phonemes in this manner, the reliability of the input voice is calculated as follows.

First, the phoneme model is searched to find the most similar phoneme.

The similarity is calculated for the semitone phone model.

The final reliability can be obtained by calculating the difference between the similarity between the phoneme model and the similarity between the semi-phoneme model and adjusting a range of the reliability value by applying a predetermined specific function thereto.

However, the continuous speech recognition system using the HMM should be able to recognize that the words are spoken continuously. At this time, even if the user pronounces the words in succession or pronounces the words and words while resting a little, it should be recognized. To this end, it is assumed that silence can come between words, and the recognition process is performed using the silence model.

The method of processing the silent model is mainly implemented in a finite-state model method or a statistical language model (bigram, trigram, etc.) by assuming silence as a word in the language model. The use of bigrams in the statistical language model method, which is widely used among these methods, defines silence as a word, so instead of linking between word (A) and word (B), "word (A), silence, word (B) "is bound to be. Since the definition of a bigram can only explain between two words, P (word B / word A) should be obtained using two probability values, P (mute / word A) and P (word B / mute). By the way, when a word is made up of two or more words, the silence may enter between several words such as the words A, B, C, ..., so that "P (word B / word A) ≠ P (mute / word A) X P (word B / mute) ". The reason is that after silence, word B, word C, ... can come. In addition, mute can be looped (looping) a number of times, expressing this can not express the relationship between the words A, B according to the information of the bigram (bigram).

Therefore, in the current technical field, a language model for continuous speech recognition is constructed instead of adopting a language model method by assuming a silent model as a word in a continuous speech recognition system that recognizes two or more words consecutively. Continuous speech recognition using the constructed language model is required.

The present invention has been proposed to solve the above problems, and distinguishes the silent model for continuous speech recognition, and in particular, the beginning of words belonging to each class without considering the silent between the words as a word in the language model. The purpose of this study is to provide a speech recognition method that can improve the recognition rate and implement a simple language model by making a data structure with phonemes.

According to an aspect of the present invention, there is provided a speech recognition method in a continuous speech recognition system, comprising: a first step of defining a silence model to generate a language model; A second step of constructing a data structure to be used in a recognition process by composing an intermediate silence in the language model as starting phonemes of word groups in every class; A third step of recognizing a word corresponding to a start class by using the data structure; A fourth step of tracing back only the value ending with the last phoneme of the word corresponding to the end class in the recognition result and dividing it into phoneme units; And a fifth step of deleting the intermediate silence from the phoneme unit division information and verifying using the semi-phoneme model, and outputting assuming that only a column of the verified word is recognized as a sentence.

delete

The present invention assumes that there is a silent phoneme at the beginning of every word instead of adopting a language model method by assuming a silent model as a word in a continuous speech recognition system that recognizes two or more words consecutively. When a word becomes a recognition object, it is processed to start the first phoneme or silence of every word. In particular, when this method is adopted, the performance is greatly improved when silent numbers are entered at intervals of three words or four words when pronouncing continuous numeric recognition such as a phone number and social security number.

The above-mentioned objects, features and advantages will become more apparent from the following detailed description in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

When constructing a language model for continuous speech recognition, the silence model is divided into three types (here, assuming that q1 and q2 are the same) as shown in FIG. 2A. An HMM topology example of this silent model is shown in FIG. 2B.

In FIG. 2A, q1 (start mute) and q2 (end mute) model the silence at the beginning and end of the continuous voice, and q3 (middle silence) model the silence between words.

In FIG. 2B, q1 and q2 are treated as language models, and q3 is defined as optional phonemes that exist before every word. In addition, since q1 and q2 are mute at the beginning and end of the sentence, it is defined as mute having three states, and q3 is defined as mute having one state.

The process of creating a language model for silence in Figures 2a and 2b is as follows. That is, it is assumed that q1 and q2 are one word, and q3 defines the starting phoneme of the next word group and obtains a probability value corresponding to a bigram or a trigram.

3A and 3B illustrate the recognition process using the probability values obtained in FIGS. 2A and 2B.

First, a data structure is created as shown in FIG. 3B. In the language model of FIG. 3A, class1 and class2 representing word groups are defined, and q1 and q2 are also defined as independent classes composed of one word.

3B shows a data structure composed of units of each class. That is, if class1 is composed of words F1, F2, ..., the phoneme q3 is added before every word as optional before every word corresponding to this class1. In this case, F1 and F2 may be represented by a tree structure or a linear structure to express phonemes. And the relationship between class and class is represented by probability value. In addition, q1, q2 and q3 add a self-looping arc to enable looping.

4 is a flowchart illustrating an example of a speech recognition method through silent model processing according to the present invention, and illustrates a procedure of performing a recognition operation from language model information.

Looking at the overall process, in the continuous speech recognition system, the silent model is divided into three types (q1, q2, and q3), q1 and q2 are composed of independent words to form a language model, and q3 is a group of words in each class. By constructing the beginning phone of, we make a data structure to be used in the recognition process. By using this data structure, the word corresponding to the starting class is recognized by Viterbi, and only the word corresponding to the last class is obtained from the recognition result. The phoneme is divided into phonemes.

Specifically, it is as follows.

First, we create a language model by defining three silent models. At this time, the silence at the beginning and the end of the sentence is defined as q1 and q2, and since it generally takes a duration, it is composed of three states and expresses self-looping with the language model. Defined as q3 and composed of 1 state, but not considered as a language model, only the relationship between words is represented by statistical probability distribution.

Subsequently, using the class language model to create a data structure containing the silence model in accordance with the recognition process (401 ~ 403). In other words, create a data structure that can be an efficient recognition process from class word grammar, but make the data structure self-looping to the q3 phoneme before each word in every class other than the class consisting of q1. Make.

That is, the class and word information is read from the language model information (401), the class to which q3 is added from the class word grammar is selected (402), and a data structure is formed such that the q3 phoneme is self-looping (403). .

Next, Viterbi is recognized by using the obtained data structure. At this time, the initial class is obtained before the Viterbi process starts (404), and the Viterbi process is performed only for words corresponding to this class. Perform 405. That is, after setting only the word list corresponding to the initial class as an initial value for Viterbi search (404), a general Viterbi algorithm is performed (405).

Subsequently, only words corresponding to the last (end) class are obtained from the recognition result (406), and phoneme units are divided (407). That is, only the values ending with the last phoneme of the word corresponding to the termination class are traced back to the phoneme unit. In this case, the trace-backed sentence obtains partition information in word units and phoneme units. In particular, since the q3 phoneme is a silent model, the phoneme is deleted from the phoneme split information (408) and then verified (409) using the semitone phone model. That is, the verification process is performed by using the phoneme segmentation information and the semi-phoneme model in the word from which the q3 phoneme is removed.

Looking at the phoneme trace back process in more detail as follows.

Speech recognition finds a phoneme for that time each time the time axis increases (the number of frames increases). Thus, the end point of speech recognition becomes the last frame of the input speech. The phoneme displays only the last phoneme of the last word. Therefore, the phoneme sequence or word sequence from the first frame of the input speech to the present is obtained before the speech recognition result is known. This process is called "trace-back". This method stores a phoneme (or word) string that has been recognized up to the frame where the search result of each frame is stored.

The reason for the trace-back is that the result of recognizing the first frame from the last frame recognition result is traced back. This automatically calculates the phoneme sequence that has been recognized in units of frames from the first frame to the last frame of the input voice. The present invention verifies that q3 is looping in the phoneme string and that q1, q2 and q3 are not included in the verification step.

Finally, it is assumed that only a column of the word whose verification is confirmed is recognized as a sentence and output (410).

The method of the present invention as described above may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible in the art without departing from the technical spirit of the present invention. It will be clear to those of ordinary knowledge.

As described above, the present invention models three types of silences in the utterance section, and in particular, defines the silences in the middle of the word as HMM parameters in units of one state, and instead of using the language model method. Bigrams, trigrams are defined by creating a data structure that is defined as a phoneme in a class and an optional silence that can precede all words in the class in the course of the recognition of this class. trigram) does not define the intermediate silent (q3) as an independent word, which can maintain the constraints of the information continuously, resulting in improved recognition rate and simplifying the language model of the silent.

Claims (7)

In the speech recognition method in a continuous speech recognition system, A first step of defining a silent model to generate a language model; A second step of constructing a data structure to be used in a recognition process by composing an intermediate silence in the language model as starting phonemes of word groups in every class; A third step of recognizing a word corresponding to a start class by using the data structure; A fourth step of tracing back only the value ending with the last phoneme of the word corresponding to the end class in the recognition result and dividing it into phoneme units; And A fifth step of deleting the middle silence from the phoneme division information and verifying using the semi-phoneme model, and assuming that only the columns of the verified words are recognized as sentences Speech recognition method through silent model processing in a continuous speech recognition system comprising a. The method of claim 1, In the first step, The silence model is divided into three types (q1, q2, and q3), and the silence at the beginning and the end of the sentence is defined as q1 and q2, and because it generally takes a duration, it is composed of three states and self-looping with the language model. (self-looping) is expressed, and the silence between words is defined as q3 and composed of 1 state, but without considering the language model, only the relation between words is represented by the statistical probability distribution. Speech recognition method through silent model processing in system. The method of claim 2, The second step, Create a data structure to be an efficient recognition process from the class word grammar from the first step, where q3 phonemes are self-looping before each word in every class other than the class consisting of q1. Speech recognition method through a silent model processing in a continuous speech recognition system, characterized in that to create a data structure. 4. The method according to any one of claims 1 to 3, The third step, Perform the Viterbi process using the result of the second step, but obtain the initial (start) class before the Viterbi process starts, and perform the Viterbi process only for words corresponding to the initial (start) class Speech recognition method through silent model processing in a continuous speech recognition system, characterized in that. The method of claim 4, wherein The fourth step, A continuous sequence that finds and traces back phonemes ending with a word corresponding to an end (last) class in Viterbi result, obtains each word and phoneme segmentation information, and removes q3 phonemes from the segmentation information. Speech recognition method through silent model processing in speech recognition system. The method of claim 4, wherein The verification process of the fifth step, q3 Speech recognition method using a silence model processing in a continuous speech recognition system characterized in that the verification (verification) using the phoneme segmentation information and the anti-phone model in the phoneme removed phoneme. delete

Images