KR100832556B1 - Speech Recognition Methods for the Robust Distant-talking Speech Recognition System - Google Patents

Abstract

The present invention provides a speech recognition method for a robust remote speech recognition system for improving the performance of the remote speech recognition function in the pattern matching part or the performance of the non-registered word rejection function in the post-processing part in the speech recognition system. In the pattern matching part of the speech recognition system, the recognition rate (ie, far speech recognition performance) of the reliable speech recognition system is improved by applying the left and right phoneme connection technique and the noise synthesis and adaptive algorithm for the speech database construction. In addition, by using the new half-phoneme model or the new language model in the post-processing part, the rejection function for the non-registered words is reduced, thereby reducing the probability of misrecognition and improving the rejection probability for the non-registered words. To calculate the semitone phone likelihood ratio To reduce it.

In the post-processing part of the speech recognition system, the present invention enables speech verification using a new half-phone model and a sequential unregistered word rejection function applying a language model for a small continuous speech recognition system. It can greatly contribute to the commercialization of the continuous speech recognition system by greatly improving the recognition rate of unregistered words while maintaining the recognition rate.

Description

Speech Recognition Methods for Robust Distant-talking Speech Recognition System

1 is a block diagram illustrating a general speech recognition system.

FIG. 2 is a diagram showing a Hidden Markov Model (HMM) of an audio signal in a pattern matching part as shown in FIG.

FIG. 3 is a diagram illustrating a signal separation network for obtaining signals separated from mixed signals in the pattern recognition part of FIG. 1 as an example; FIG.

FIG. 4 illustrates an example of a bigram network capable of all transitions for two words in language modeling in the post-processing part of FIG. 1.

FIG. 5 is a diagram illustrating a recognition network for explaining language modeling in a post-processing portion in FIG. 1 as an example; FIG.

6 is a flowchart illustrating a speech recognition method for a robust far-field speech recognition system according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating, for example, a pronunciation string generating step in FIG. 6; FIG.

8 is a flowchart illustrating a speech recognition method for a robust far-field speech recognition system according to this embodiment of the present invention.

FIG. 9 illustrates a signal synthesis network for obtaining a signal obtained by mixing a voice signal and various noise signals in the pattern recognition part of FIG. 8; FIG.

FIG. 10 is a diagram illustrating an example of a mixed signal in which an actual original signal is mixed with a noise signal in FIG. 9; FIG.

FIG. 11 is a block diagram illustrating an example model re-estimation system in FIG. 8; FIG.

12 is a flowchart illustrating a speech recognition method for a robust far-field speech recognition system according to an embodiment of the present invention.

FIG. 13 is a flow chart illustrating the triphone model clustering step in FIG. 12. FIG.

FIG. 14 is a view for explaining an example of constructing a voice database optimized in FIG. 12; FIG.

15 is a flowchart illustrating a speech recognition method for a robust far-field speech recognition system according to an embodiment of the present invention.

FIG. 16 is a view for explaining a state transition probability in FIG. 15. FIG.

17 is a flowchart illustrating a speech recognition method for a robust far-field speech recognition system according to an exemplary embodiment of the present invention.

FIG. 18 is a diagram illustrating a recognition network for explaining language modeling in a post-processing portion in FIG. 17 as an example; FIG.

FIG. 19 is a diagram for explaining a recognition network configuration in FIG. 17; FIG.

Explanation of symbols on the main parts of the drawings

11: preprocessing part 12: pattern matching part

13: post-processing part

The present invention relates to a speech recognition method for a robust far-field speech recognition system. In particular, the present invention relates to improving the performance of the far-field speech recognition function in the pattern matching part or to the non-registered word rejection function in the post-processing part. The present invention relates to a speech recognition method for a robust far-field speech recognition system to improve performance.

In general, the means by which humans can express their thoughts to others take many forms, of which voice is the most basic means of communication used by humans, compared to other methods in terms of convenience and economy. Has excellent properties.

Speech processing by humans can be divided into two aspects, speech production and speech perception. The speech generation is a series of processes for delivering a speaker's intention, and the speech recognition refers to a process of recognizing speech contents from a voice spoken by the other speaker. Studies related to these two aspects of speech have been conducted separately, and have been conducted under various academic backgrounds such as linguistics, phonetics, phonology, physiology and anatomy.

Such results have led to rapid advances in technology such as signal processing technology, large storage capacity, and high-speed computer processing technology. As a result, research that utilizes these results in practical terms, rather than just experimental results, is actively conducted. In terms of computational theory, various studies related to speech processing have been conducted.

Approaches for performing speech recognition in terms of speech recognition of speech processing are largely acoustic-phonetic method, statistical pattern recognition method, artificial intelligence method and It can be classified into four types, neural network method.

First, the audible speech display method is an approach focused on the preprocessing part of performing speech recognition. Since the speech is a signal and the language information is included in the signal, the signals are divided into a series of phonemes or words. It is composed of heat '. So, after capturing a signal, the same characteristic is divided into segments of the same signal using the characteristics of the speech from that signal, and how the segments match the language called the phoneme. This method finds a phoneme sequence from the corresponding signal, finds a string of words from the phoneme sequence, and finds out which sentence is the last input voice.

Secondly, the statistical pattern recognition method may be classified into a template-based approach and a model-based approach. The template-based approach, Dynamic Time Warping (DTW), It is an approach that uses only some of the statistical methods, and the model's basic approach is to model certain units using statistics of some segmented schedule unit, and the most widely used HID (Hidden Markov Model). ) Is the approach.

Third, the artificial intelligence method is an approach that performs speech recognition with an emphasis on the upper level stages of the knowledge processing process, how to recognize speech in the human brain.

Fourth, the neural network method is an approach that mimics biological computational power by using a large number of interconnected artificial neurons.

The speech recognition system using various approaches as described above includes Isolated Word Speech Recognition for recognizing isolated words according to a speech form, and Continuous Speech Recognition for recognizing speech in successive words. Speech Recognition). In this case, in the case of continuous speech, in the case of a word having a relatively small vocabulary, it is mainly connected word recognition recognized by acoustic characteristics, and a relatively large vocabulary is used for linguistic knowledge. It can be classified into sentence speech recognition that recognizes content. In addition, depending on the speaker's dependency, it can be divided into Speaker Dependent speech recognition that can be recognized only by a specific speaker and Speaker Independent speech recognition that can recognize all speakers. Can be.

The speech recognition system through various approaches and classifications as described above is used for accurate analysis of speech features, speaker's personality, type of speech, number of vocabulary, language complexity, environmental factors, and perception in order to implement a practical practical system. There is a need for research to overcome various problems such as the unit of, and many studies have been conducted in various countries around the world to develop a system capable of recognizing natural speech, which is the final goal of speech recognition technology.

Recently, in the speech recognition system, the statistical pattern recognition method as described above has become mainstream, and the pattern recognition is largely shown in FIG. 1, with a preprocessing portion 11, a pattern matching portion 12, and a post-processing. It can be divided into three parts of the part (13).

In other words, the speech recognition system using the statistical pattern recognition method described above, as shown in FIG. 1, extracts from the preprocessing portion 11 which extracts and parameterizes the language information of the speech and parameterizes it, and extracts from the preprocessing portion 11. Improve the recognition rate using a pattern matching part 12 for performing a learning process by analyzing and modeling the pattern of the feature parameter, or by performing a recognition process for matching with a predetermined model. It is divided into a post-processing portion 13 that performs a function such as a rejection function for rejecting unregistered words, an adaptation technique for adapting to environmental changes, and the like.

The preprocessing portion 11 includes a voice feature analyzer 11-1, which extracts / segments an endpoint, analyzes a vocal model, analyzes an auditory model, removes noise, and receives a voice input through a voice input device such as a microphone. Channel distortion compensation and the like.

The pattern matching part 12 includes a pattern recognizing part 12-1 and a database 12-2 for storing an acoustic model (ie, a phoneme or a word), and the information processed by the preprocessing part 11. Clustering, dynamic matching, vector quantization (VQ), neural network (NN), probability model (GMM / HMM) are processed.

The post-processing portion 13 includes a language processor 13-1 and a database 13-2 for storing language models (ie, vocabulary, grammar, and subject), so that the language model and adaptation (MAP, MLLR) are provided. , Rejection (OOV (Out of Vocabulary)).

In addition, the model-based HMM approach performed in the pattern matching part 12 may simultaneously model the spectral change and the time change of the speech signal, and use finite state and state transitions for this purpose.

The usefulness of the HMM is not to accurately model the speech generation process, but rather to estimate the parameters using the given data and find the most suitable model for the newly input speech. In other words, there is an efficient algorithm for training and recognition.

'는 음성 벡터 혹은 관측을 나타내고, '

'와 '

'는 각각 상태 천이 확률과 심볼 관측 확률 분포함수를 의미한다.At this time, various types of HMMs are used to model the voice, but a Left-to-Right (LTR) model having a simple structure as shown in FIG. 2 is used. As shown in Fig. 2, states 2, 3, and 4 are outputs, and states 1 and 5 have no outputs, and only serve to connect the model. '

'Represents a negative vector or observation, and'

'Wow '

'Means state transition probability and symbol observation probability distribution function, respectively.

)은 제1상태(i)에 있던 모델이 제2상태(j)로 상태를 변화시킬 조건부 확률로서 아래의 수학식 1과 같다. 여기서, '

'는 임의의 시간(t)에서의 모델의 상태를 나타낸다.The transition probability (

) Is a conditional probability that the model in the first state (i) changes state to the second state (j), as shown in Equation 1 below. here, '

'Represents the state of the model at any time t.

)은 상수로 가정하여 시간에 따라 변하지 않으며, 전체 상태의 개수를 'N'이라 했을 때에, 모든 초기 상태들(i = 1, ..., N-1)은 아래의 수학식 2와 같은 조건을 만족해야 한다.And the transition probability (

) Does not change over time assuming a constant, and when the total number of states is 'N', all initial states (i = 1, ..., N-1) are the same as in Equation 2 below. Must be satisfied.

)는 제2상태(j)에 의하여 발생될 관측들의 분포를 나타내는데, 이것은 제2상태(j)가 관측(

)을 생성할 확률(Likelihood; 연속 출력 분포인 경우)을 의미한다. 연속 분포일 경우에 아래의 수학식 3과 같은 조건을 만족하여야 한다.The symbol observation probability distribution function (

) Represents the distribution of observations that will be generated by the second state j, which means that the second state j

) Is the probability of generating (Likelihood). In the case of continuous distribution, the following condition must be satisfied.

, for j=2, ..., N-1

, for j = 2, ..., N-1

On the other hand, in the case of a far-field speech recognition system that controls an arbitrary terminal by voice, the user should not only be easy and accurate in use, but also should work well in a real situation. An important factor is robustness.

In most cases, the performance of far speech recognition is remarkable because the environment in which the actual speech recognition system is to be used is different from the learning environment. In particular, in the case of a remote voice input having a long distance between a microphone and a speaker, a sudden change in the actual surrounding environment occurs due to ambient noise, microphone, channel distortion, etc. in addition to the input signal, resulting in a sudden degradation of the system.

In other words, in the process of inputting voice from a long distance, noise other than the input signal may exist, which reduces the recognition rate. Therefore, noise must be removed. This situation is different from that of the learning data, so the recognition rate is drastically reduced. This effect is called the Lombard effect.

Therefore, for long distance speech recognition that works well in a real environment, environmental compensation is required to exclude the influence of the changed environment. In other words, for a long distance speech recognition system with high reliability, an environmental compensation technique that excludes the influence of the changed environment or the speaker is essential.

Therefore, in the pattern matching part 12 in the speech recognition system having the above-described configuration, the feature extracted from the preprocessing part 11 is applied by applying an independent component analysis (ICA) method as blind source separation (BSS). A learning process of analyzing and modeling a parameter pattern is performed or a recognition process of matching with a predetermined model is performed.

In this case, the ICA technique is a statistical method of separating the mixed signals obtained from two or more sensors into independent components. First, a model in which signals are mixed to explain the signal separation operation by the ICA is described. As follows.

)를 가정한다. 즉, 벡터(

)는 N개의 독립적인 스칼라 값을 갖는 원신호들이며, 해당 N개의 원신호들은 N개의 센서열로 입력되며, 이를 '

'라고 가정하면, 이때의 입력된 신호는 원신호들이 필터링되어 혼합된 것으로서 아래의 수학식 4와 같이 표현된다.Vector of order N with the mutually independent mean value '0' between each order at any time (t)

Assume). That is, vector (

) Are original signals with N independent scalar values, and the corresponding N original signals are input to N sensor strings.

', The input signal at this time is a mixture of the original signal is filtered and expressed as shown in Equation 4 below.

번째 원신호와

번째 센서 사이의 M차의 필터(

)로서 모델링된다. 상기 BSS는 환경에 대한 사전 지식 없이 '

'로부터 원신호(

)를 추출하는 문제이다. 상기 수학식 4는 컨벌루션 혼합을 나타내며, 'M=1' 인 경우가 아래의 수학식 5와 같이 즉시 혼합(Instantaneous Mixture)인 경우이다.And, in the equation (4)

The first original signal

Filter of order M between the first sensor

Modeled as). The BSS is a '

From "

) Is the problem of extracting. Equation 4 represents a convolutional mixture, and 'M = 1' is a case of Instantaneous Mixture as shown in Equation 5 below.

)를 추정하기 위해서는, 상기 수학식 4에서의 M차 필터들(

)에 의해 컨벌루션 혼합된 신호들로부터 디컨벌루션 과정을 수행해야 한다.And, from the mixed signals the original signal (

In order to estimate), M-order filters (Equation 4)

The deconvolution process must be performed from convolutionally mixed signals.

The signal separation scheme using the ICA technique was proposed as an information-theoretic approach using mutual information by Bell and Sejnowski in 1995, and in 1996 Torkkola time We have developed an algorithm of time-domain feedback structure.

와

)로부터 분리된 신호들을 얻기 위한 신호 분리 네트워크를 예로 나타낸 도면이다.3 illustrates the mixed signals (eg,

Wow

Shows an example of a signal separation network for obtaining signals separated from

'와 '

' 간의 상호 정보를 최소화하도록 하기 위해서, 학습 과정을 통해 분리 행렬(

)을 구하도록 한다. 여기서, '

'는 비선형 시그모이드 함수이며, 학습 규칙은 아래의 수학식 6과 같은 확률 변화 상승 법칙(Stochastic Gradient Ascent Rule)을 사용하도록 한다. 또한, 해당 '

'는 학습률이다.In the pattern matching portion 12,

'Wow '

In order to minimize the mutual information between ', the learning process

). here, '

'Is a nonlinear sigmoid function, and the learning rule uses a Stochastic Gradient Ascent Rule as shown in Equation 6 below. Also, that '

'Is the learning rate.

By the way, in the pattern matching portion 12 in the speech recognition system having the above-described configuration, the recognition process is important, but above all, the learning process is very important. It is no exaggeration to say that the learning process in the pattern recognition unit 12-1 of the pattern matching part 12 determines the speech recognition rate according to the rich pattern, i.e., how much learning data. If a man or woman of various ages has a voice produced in various ways in a single PLU (Phone Like Unit) and trains using the learning data, the recognition performance will be excellent. However, since it is impossible to acquire the desired amount of learning data each time the recognition target is changed, in this case, there is a disadvantage in that the speech recognition rate is naturally reduced.

On the other hand, the speech recognition system having the above-described configuration can be divided into a system of isolated word recognition and a system of continuous speech recognition according to the recognition target unit. The isolated word recognition system is a system in which an input voice is one of predetermined words and recognizes a word sound having a clear start and end. The continuous speech recognition system recognizes an input voice itself, which is a continuous sentence.

However, the speech recognition function is performed under the assumption that both systems will input only certain predetermined target words. Therefore, if a user speaks a word other than the target word by mistake or knowingly, Since the recognition result is represented as one of the words to be recognized having a probability, a problem of misrecognition occurs.

Therefore, in the speech processing unit 13-1 of the post-processing portion 13 in the speech recognition system having the above-described configuration, when a voice other than the word to be recognized is input, it is not mistaken as another word and the input voice is wrong. Voice recognition rejection function (ie, non-registered word rejection function) is determined to be performed. Here, the non-registered word rejection function is classified into a keyword spotting method and a speech verification method according to the method.

The key word detection method is a method of rejecting an unregistered word using a phoneme filler model. Most keyword detection methods are based on linked word recognition algorithms using keyword models and filler models. Here, filler models are used to represent speech sections (ie, non-keywords) that do not correspond to keywords and sections of non-voice (ie, silence) or background noise.

In addition, the key word detection method may exhibit good performance mainly when the distinction between the key word model and the filler model is relatively clear. Thus, mainly the key word becomes the word to be recognized, and the filler model is the other words, and the performance depends on how well the filler model models other things without being similar to the key word model.

However, the key word detection method has a disadvantage in that performance is reduced in the case of a variable vocabulary speech recognition system in which a word to be recognized is changed from time to time. In other words, after completing the training with the key word and non-key word already established, and performing the speech recognition rejection function, the newly added or changed word should have a higher similarity to the key word than the similarity to the non-key word already modeled. Only non-core words are modeled for all words other than the key words, so they will have a more general distribution in the pattern space, so that newly added or changed words are also included in the general pattern. There is a disadvantage that it is more likely to be classified as.

The speech verification method may perform a verification operation of determining whether to accept or reject a recognition result of a word or a phoneme unit. Conventional technology has rejected unregistered words using a speech verification method using an anti-phone model. If a well-trained phone model is used, the semi-phone model can be used without special training. Suggested to make. In addition, using the reliability of phoneme units, the reliability of word units that can be used in the variable vocabulary word recognition system was constructed. Here, the 'reliability' is a measure of how reliable the result is for the speech recognition result, which is different from the Viterbi search result value of the HMM model. In this case, the Viterbi search result number indicates a simple similarity to a word or phoneme, but the reliability is a relative value of the probability that the word is uttered from other phonemes or words for the phoneme or word that is a recognized result. Say

By the way, the semi-phoneme model refers to a set of similar phonemes excluding self-phones. In general, the more similar phonemes, the more semitones are modeled. However, if the size of the similar phoneme is too large, the amount of training data is too large. have.

In the existing technology, the semi-phoneme model contrary to the phoneme model is predetermined and registered in the memory. In other words, if you have a well-trained phoneme model, you can create a semi-phoneme model without any special training to create a semi-phoneme model, and the semi-phoneme set also includes all the phonemes except self-phone and silence. It was done. Specifically, the weights and averages of Best Gaussian, 2nd Best Gaussian, 3rd Best Gaussian, and 4th Best Gaussian , Take the dispersion. Thus, the semiphoneme model of each phoneme has three states, each state having about 148 branches. The transition probability takes the average of all the phonemes except its own phone.

However, the semitone phone model also has a disadvantage in that it does not solve the mismatch condition resulting from environmental changes such as noise environment.

The reliability of the above-described word unit is as follows.

Since the Viterbi search is performed using the variable vocabulary word recognizer, it is basically recognized in word units, but the recognized words are internally recognized in phoneme units. Accordingly, the reliability of the phoneme unit is averaged in order to obtain the reliability by comparing the recognized phoneme units with respective semitone models.

)에 상응하는 발화 검증 모델 을 사용하는 신뢰도를 선택한다. 여기서, 각 패턴 1에 대해서 음소 모델을 '

'라 표시하고, 반음소 모델을 '

'라 표시한다. 이에 따라서, 음소 단위들을 평균한 단어 단위의 신뢰도는 아래의 수학식 7과 같다.First of all, if you have a pattern other than yourself

Choose the confidence level using the speech verification model corresponding to Here, for each pattern 1, the phoneme model is

'And the semiphone model

' Accordingly, the reliability of the word unit averaging the phoneme units is expressed by Equation 7 below.

) 이하라면 거절시키게 된다. 여기서, '

'는 음의 값을 가지는 상수이며, 가변 어휘 단어 인식기에서 인식된 결과인 등록어(

)는 '

' 음소들로 구성되어 있다.A threshold value predetermined by the reliability of the word unit (

) Or less. here, '

'Is a negative valued constant, and the registered word (

) Means "

'Consists of phonemes.

As described above, a technique for rejecting an unregistered word using the speech verification method using the semi-phoneme model is well known as being suitable as an unregistered word rejection technique of a variable lexical continuous speech recognition system in which a recognition object is changed or added. In addition to increasing the computational complexity of the continuous speech recognition system, there is a disadvantage in that it is not possible to solve the inconsistency condition caused by the change of the speech environment.

In addition, due to the increasing performance of computers, the increase in the amount of computation is not greatly affected, but when used in an embedded environment such as a DSP or a PDA, it causes a large loss in memory and recognition speed. .

Next, the language modeling by the language processor 13-1 in the post-processing part 13 will be described.

)에 대한 언어 모델 값은 아래와 같은 수학식 8로 계산된다.In a language model network represented by a finite state network (FSN), each node of the network defines a state of the language model, and an arc represents a transition to a new state according to the input word. Word string

The language model value for) is calculated by Equation 8 below.

'는 언어 모델의 히스토리(History)로 과거 단어열을 의미하며, 해당 '

'는 과거 단어들을 조건으로 현재 단어(

)가 나타날 확률을 의미한다.Where, "

"Is a history of language models, meaning past word strings,

"Means that the current word (

) Is the probability of appearing.

)을 일정한 동치 관계(Equivalence Relation)를 사용하여 나눈다.When creating the language model, in order to compensate for the lack of data in learning, in general, the past word string (

) Is divided using Equivalence Relation.

,

)가 동일한 경우에 동치 관계가 성립한다. 그리고, 이에 따라 각 동치 부류는 아래와 같은 수학식 9와 같은 확률 값을 가진다.For example, the word Trigram is the last two words of the history (

,

Equivalence is established when) are the same. Accordingly, each equivalent class has a probability value as shown in Equation 9 below.

In addition, various language models may be designed according to a method of partitioning a history that the language model may have. By partitioning the history based on the recent N word strings, you can obtain a N-gram language model, and also, if you divide the words into grammatical / linguistic classes to which each word belongs, instead of a class engram ( Class N-gram) can be obtained.

In order to transparently use various language models in the recognition process, a unified expression method is required, and the language model network is an expression method configured for this purpose.

)와 입력 단어(

)를 사용하여 '

'의 쌍으로 정의된다.The language model network is generated using a language model context inherent in each model, wherein the language model context (LM Context) is a history (

) And the input word (

)use with '

Is defined as a pair of '.

In particular, since a word is a continuous input, each time a new word input is entered it transitions from a specific context to a new context. This context transition is common to all language models. The context of the language model is finite and the transition between contexts is determined by the input word and thus can be expressed as an FSN.

At this time, the state of the FSN corresponds to each context of the language model, and the arc stores language model values calculated from input words and history.

)와 단어(

)의 쌍 '

'으로 표현되며, 아크는 입력 단어(

)와 특정 컨텍스트로의 전이 확률(즉, 컨텍스트의 히스토리에 조건을 둔 바이그램 확률 값)로 이루어진다. 또한, 초기 상태에는 그림자가 추가되며, 네트워크의 어떤 상태도 종료 상태가 될 수 있다.For example, as shown in Fig. 4, when representing the Bigram Network in which all transitions for the two words (a, b) are possible, each state is a history (

) And words (

) '

", And the arc represents the input word (

) And the probability of transition to a particular context (i.e., a bigram probability value based on the context's history). In addition, a shadow is added to the initial state, and any state of the network may be an end state.

)이 가중됨으로써, 인식률을 향상시킬 수 있게 된다.Most of the existing language models as described above have been studied for improving the recognition performance of a large capacity continuous speech recognition system. For example, in the recognition network as shown in FIG. 5, the key words "naraeya" are more likely to be connected to words such as "room, living room, kitchen, ... boiler, air conditioner", followed by "shine," If you model the probability that words such as turn off, turn on, turn off, turn up the temperature, turn down the temperature, etc. are connected, the language model (10)

), The recognition rate can be improved.

In fact, in a large-capacity continuous speech recognition system consisting of more than 1000 words, such a language model can greatly contribute to the improvement of the recognition rate when constructing a filler model well.

However, as described above, when the recognition target word is added or changed, although it is an unregistered sentence, it does not fall into the filler model but has a disadvantage in that the probability of misrecognition becomes very high.

In order to solve the above disadvantages, the present invention is to improve the performance of the remote speech recognition function in the pattern matching portion in the speech recognition system, or to improve the performance of the non-registered word rejection function in the post-processing portion. It is an object of the present invention to provide a speech recognition method for a robust remote speech recognition system.

In addition, the present invention improves the recognition rate (ie, far speech recognition performance) of the highly reliable far speech recognition system by applying the left and right phoneme connection technique and the noise synthesis and adaptive algorithm for constructing the speech database in the pattern matching part of the speech recognition system. It has a purpose.

Alternatively, the present invention is to improve the performance of the non-registered word rejection function by applying a reliability-based rejection technique using a new half-phone model in the post-processing portion of the speech recognition system.

In addition, an object of the present invention is to reduce the amount of calculation of the phoneme to half phoneme likelihood ratio for measuring reliability by a new reliability measurement method using a new halftone model in the post-processing portion of a speech recognition system.

In addition, the present invention is to perform a rejection function for the unregistered words by using a new language model in the post-processing portion of the speech recognition system, thereby reducing the probability of misrecognition to improve the rejection probability for the unregistered words. have.

In addition, the present invention allows the post-processing portion of the speech recognition system to perform sequential verification using a new half-phone model and a sequential unregistered rejection function applying a language model for a small continuous speech recognition system. While maintaining the recognition rate, it greatly improves the recognition rate of unregistered words, greatly contributing to the commercialization of continuous speech recognition system, and can be applied to embedded speech recognition systems such as DSP or PDA, which have not been applied due to the increase of calculation amount. There is this.

A voice recognition method for a robust long-range speech recognition system according to an embodiment of the present invention for achieving the above object, in the speech recognition system for recognizing the voice of the words continuously input through the voice input device, Generating the input voice as a pronunciation string through a phoneme unit pronunciation string generator, converting the pronunciation string into a triphone of a recognition unit, and recognizing the converted triphone through a corresponding triphone transformation; Combining the end phoneme of the word with the first phoneme of the word following the word when recognizing the triphone to generate a triphone sequence that does not appear during the voice recognition; And a step of recognizing a voice for the continuous input word using the recognized triphone.

Alternatively, a voice recognition method for a robust long-range speech recognition system according to this embodiment of the present invention is a voice recognition system for recognizing a voice of a word that is continuously input through a voice input device. Acquiring the generated noise signals through a plurality of voice input devices to implement a noise signal database; Combining the obtained noise signals with a clean original audio signal to generate mixed signals; Learning the generated mixed signals to acquire learning data collected in a plurality of noise environments to implement learning data databases; A step of converting the acquired training data into raw data in a plurality of noise environments in a model re-estimation system and learning the HID model; And a step of recognizing a speech for the continuous input word using the trained HMM model.

Herein, the learning process of the HMM model may include: generating a model parameter from an model parameter database and initializing it as an initial model parameter; After reading the training data from the training data database, using the read training data to partition the state sequence for the original model parameter or the modified model parameter; Determining a probability by using the read training data and the state sequence divided information; Re-estimating new model parameters using the read training data and the determined probabilities; Determining whether the reestimated model parameter has converged to the HMM model, and upgrading the model parameter database.

In addition, when the reestimated model parameter is not converged to the HMM model, the learning process with the HMM model includes processing the state sequence segmentation step by processing the reestimated model parameter as the modified model parameter. Characterized in that it further comprises the step of performing repeatedly.

Alternatively, the voice recognition method for the robust long-range speech recognition system according to the third embodiment of the present invention, in the speech recognition system that recognizes the voice of the words that are continuously input through the voice input device, stores the clean original audio signal Adding the original audio database to an adaptive database storing necessary voice data according to an application field; Clustering the plurality of triphone models such that a specific number of similar phoneme clusters are generated to adapt the plurality of triphone models using the adaptation data stored in the adaptation database; Estimating a set of linear transformations for the mean value of the Gaussian mixed Hidden Markov Model (HMM) model through Maximum Likelihood Linear Regression (MLLR) to average compensate the corresponding HMM model; Learning the clustered triphone models and performing MAP adaptation of the adaptation data on the average compensated HMM model according to the triphone-specific state investigated by the training; And a step of recognizing speech for the continuous input word using the adaptation model learned by the MAP adaptation.

The triphone model clustering process may include: setting an initial center value for the average of all the triphones, and then moving the center values of each cluster to all input vectors and dividing them into two; Measuring the Euclidean distance with each divided cluster for all input vectors and grouping them into a member vector of the cluster having the smallest distance; Updating a center point of the cluster through the member vectors of the clusters, and then determining whether an update value of the corresponding error is less than or equal to a preset threshold; Determining whether the update value of the error is divided into a predetermined number of clusters when the update value of the error is less than or equal to a predetermined threshold value; And terminating the triphone model clustering process when the cluster is divided into the predetermined number of clusters.

In addition, the triphone model clustering process may include moving and dividing the center value of each cluster into two when the update value of the error is not equal to or less than a predetermined threshold value or is not divided into the predetermined number of clusters. It characterized by further comprising the step to perform repeatedly.

Alternatively, the speech recognition method for the robust long-range speech recognition system according to an embodiment of the present invention, by using the HMM (Hidden Markov Model) model for modeling the speech in the language processing unit in the post-processing part to measure the reliability In the speech recognition system performing the non-registered word rejection function by measuring the reliability, the state transition probability of the HMM model is set to '1-state' as the state transition of the HMM model to have a semitone characteristic. Replacing with a value of state transition probability; Taking a corresponding frame observation probability for the Gaussian model of the segmented frame in reverse order to make use of the semitone features; And calculating the likelihood ratio of a phoneme to a half phoneme by using the replaced state transition probability and the symbol observation probability of taking the inverse frame.

Alternatively, a speech recognition method for a robust long-range speech recognition system according to a fifth embodiment of the present invention may be applied to a speech recognition system that performs a non-registered word rejection function by applying a rejection technique using a language model in a language processing unit within a post-processing portion. A process of pre-modeling that the words and the connection probabilities between the words are the same so that all recognition objects follow each node; Selecting a sentence to be recognized in a state in which words and words are all connected, and then dividing the corresponding sentence to be recognized into words to be recognized; Constructing a recognition network using the divided recognition target word, and then performing recognition of an input voice through the corresponding recognition network; Checking whether a final recognized word string output through the recognition network matches the recognized sentence; And if the final recognized word string does not match the recognition target sentence, performing a non-registered word rejection process.

The present invention provides a novel learning and recognition technique for the remote speech recognition system in the pattern matching portion of the speech recognition system, and also improves the existing semiphoneme for the performance improvement of the unregistered word rejection function performed by the language processing unit in the post-processing portion. Using the speech verification method using the model, we provide a new rejection algorithm that compensates for the disadvantages of the non-register rejection techniques. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

A voice recognition method for a robust far-field speech recognition system according to an embodiment of the present invention will be described with reference to the flowchart of FIG. 6.

The speech recognition method for a robust long-range speech recognition system according to an embodiment of the present invention improves the recognition rate of a system by applying a left and right phoneme connection technique to a pattern matching part in a continuous word recognition system.

In the continuous word recognition system, when an input voice (for example, “Nara-ya-chan”) is applied through a voice input device such as a microphone (step S61), in the pattern matching part, a phoneme unit phoneme string is used. The pronunciation sequence through the generator is expressed as 'N AA LE JA Silence AA NQ PP AA NX Silence P UW LQ KH JX' (step S62).

In other words, in the speech recognition, texts in actual pronunciation form are used without using texts in the form of letters. In this case, the texts can be registered in real time with a pronunciation string used as learning data or a speech dictionary used in recognition. A system capable of generating a pronunciation string may be automatically generated through the phoneme unit pronunciation string generator. 7 is a diagram illustrating an automatic pronunciation string generation operation, for example.

Thereafter, the pronunciation string generated in step 62 (S62) is referred to as' N + AA N-AA + L AA-L + E L-E + JA E-JA JA-AA + NQ AA + NQ AA-NQ + PP. . In operation S63, the operation is converted into a tri-phone which is a recognition unit such as '.' At this time, in the triphone conversion step S43, the silence in the 62nd step S62 is converted into 'JA-AA + NQ'.

Then, the triphone is trained through the triphone conversion step S43 (step S64), and a speech recognition process for continuous words is performed using the learned triphone (step S65).

However, the end phoneme of the first word and the first phoneme of the second word are linked to the actual continuous speech. That is, in the case of learning the triphone, the voice has a characteristic of forming the first phoneme from the silent section. However, when continuous speech is recorded in the situation of the recognition process, the voice is different from the voice. Since the recognition rate of the continuous word recognition system is greatly reduced.

Thus, during the triphone learning in the 64th step (S64), when the program is forcibly combined with the end phoneme of an arbitrary word and the first phoneme (ie, left and right phonemes) of the word following the word, By generating triphone sequences that do not appear, the recognition rate of the continuous word recognition system can be improved.

The above-described left and right phoneme connection technique can considerably improve the performance of the continuous word recognition system by considering the consonant phenomenon that occurs when a human voice is naturally spoken at the recognition stage.

In fact, when the connected word recognition experiment using the language model is performed, the recognition rate of the system using the language model is improved by 8 (%) or more than the basic system, but the probability of misrecognition of the non-registered words is increased. In the case of using the left-right phoneme connection technique as described above, the recognition rate of unregistered words is not changed, while improving the recognition rate of 7.8 (%) than the basic system.

This proves to be an excellent technique for improving the recognition rate of small-capacity speech recognition systems with relatively different filler models while lowering the probability of false recognition of unregistered words.

A speech recognition method for a robust far-field speech recognition system according to this embodiment of the present invention will be described with reference to the flowchart of FIG. 8.

A speech recognition method for a robust long-range speech recognition system according to an embodiment of the present invention improves the recognition rate of a system by applying a noise mixing technique using an ICA in a pattern matching part in a continuous word recognition system. . That is, this embodiment of the present invention relates to a noise synthesis technique capable of acquiring learning data in a variety of environments by applying an ICA technique widely used as a blind source separation (BSS) technique.

In speech recognition, voices spoken through various microphones in various ambient noise environments are regarded as independent component elements to separate voice and environmental noise.

Thus, in reverse of the speech recognition operation, noise signals N ₁ , N ₂ ,..., N _k generated in various kinds of noise environments are obtained through various kinds of microphones (step S81), that is, Noise signals with various signal-to-noise ratios (SNRs) are acquired to implement a noise signal database. At this time, the original audio signal database storing the clean original audio signal S ₁ is also implemented.

Then, as shown in Figure 9, by combining the noise signals (N ₁ , N ₂ , ..., N _k ) obtained in the 81 _st step (S81) to a clean original audio signal (S ₁ ) Generate mixed signals M ₁ , M ₂ ,..., M _k (step S82).

FIG. 9 is a diagram illustrating a signal synthesis network for obtaining a signal obtained by mixing the original speech signal and various noise signals in the pattern recognition part. At this time, when the actual original audio signal is mixed with the noise signal, the mixed signal as shown in FIG. 10 appears.

'는 1보다 작은 랜덤 변수(Random Variable)이다.The noise synthesis is determined by Equation 11 below. here, '

'Is a random variable that is less than one.

Thus, to learn the mixed signals (M ₁ , M ₂ , ..., M _k ) generated in the 82 (S82) (step S83), a variety of environments for one phone like unit (PLU) It is possible to acquire the learning data collected in step (S84). That is, it enables to implement training data databases for mixed signals having various signal-to-noise ratios.

Accordingly, the training data acquired by the noise synthesis technique as described above (ie, the voice raw data) is obtained by the EM by the model re-estimation system as shown in FIG. 11. The Baum-welch reestimation operation, which is well known as the Expectation-Maximization technique, converts the limited voice low data into low data of various noise environments (step S85) to create an environment close to the actual recognition environment. The program is forcibly made by the program, thereby enabling learning with a more robust HMM (Hidden Markov Model) model (step S86).

'를 극대화시키기 위해서 모델 매개 변수(

)를 조정함으로써 이루어진다.As shown in FIG. 11, the model re-estimation system includes a model parameter database 111, a training data database 112, a model initializer 113, a state sequence divider 114, Probability determination unit 15, the model re-estimation unit 116, and a model converging unit 117, it is made to perform the learning process to the HMM model in step 86 (S86). That is, the learning process to the HMM model in step 86 (S86), according to the Boul-well algorithm

Model parameters (

By adjusting).

The model parameter database 111 stores model parameters applied from the model converging unit 117, and the training data database 112 stores the acquired training data as described above.

)로서 상기 상태열 분할부(114)에 인가한다.The model initialization unit 113 reads out the model parameters from the model parameter database 111 and initializes the read model parameters by initial model parameters (

) Is applied to the state string divider 114.

)에 대해 상태열 분할 동작을 수행하여 해당 상태열 분할된 정보를 상기 확률 결정부(115)에 인가한 후에, 상기 학습 데이터 데이터베이스(112)로부터 학습 데이터를 판독하고 해당 판독된 학습 데이터를 이용하여 상기 모델 수렴부(117)로부터 인가되는 수정된 모델 매개 변수(

)에 대해 상태열 분할 동작을 수행하여 해당 상태열 분할된 정보를 상기 확률 결정부(115)에 인가한 다.The state sequence divider 114 reads the training data from the training data database 112 and uses the first training parameters applied from the model initializer 113 to read the training data from the training data database 112.

After applying the state sequence division operation to the probability determination unit 115 by performing the state sequence division operation on the), the training data is read from the training data database 112 and using the read training data A modified model parameter applied from the model convergence unit 117 (

) Is applied to the probability determiner 115.

)을 결정하는데, 이때 아래의 수학식 12와 같이 확률(

)을 구하여 해당 구한 확률(

)을 상기 모델 재추정부(116)에 인가한다.The probability determination unit 15 reads the training data from the training data database 112 and uses the read data and the information applied from the state string dividing unit 114 to determine the probability (

), Where the probability (

) To get the probability (

) Is applied to the model recalculation unit 116.

)을 이용하여 모델 파라미터를 재추정하는데, 이때 아래의 수학식 12와 같이 새로운 파라미터(

)를 재추정하여 해당 새로운 파라미터(

)를 상기 모델 수렴부(117)에 인가한다.The model re-estimation unit 116 reads the training data from the training data database 112, and the probability (determined by the readout data and the probability determination unit 15)

) To re-estimate model parameters, where new parameters (

) To reestimate the corresponding new parameter (

) Is applied to the model convergence unit 117.

)가 '

'일 때에 모델 파라미터로 근사화된 것으로 판단하고 이를 상기 모델 파라미터 데이터베이스(111)에 업그레이드시켜 주게 된다. 반면에, 상기 모델 수렴부(117)는 상기 모델 추정부(116)에서 재추정한 모델 파라미터가 강인한 HMM 모델로 수렴되지 않은 경우에 상기 모델 재추정부(116)로부터 인가되는 새로운 파라미터(

)를 다시 상기 수정된 모델 매개 변수(

)로서 상기 상태열 분할부(114)로 인가한다.The model converging unit 117 determines whether the model parameter re-estimated by the model estimating unit 116 has converged to a robust HMM model, and at this time, a new parameter applied from the model re-estimation unit 116 (

) Is "

'Is determined to be approximated as a model parameter and upgraded to the model parameter database 111. On the other hand, the model converging unit 117 is a new parameter applied from the model re-estimation unit 116 when the model parameter re-estimated by the model estimating unit 116 does not converge to the robust HMM model.

Back to the modified model parameters (

) Is applied to the state string divider 114.

Thereafter, using the HMM model learned through the Boul-well re-estimation operation using the training data of various noise environments, the speech recognition process can be accurately performed even in various noise environments (step S87).

At this time, it can be seen that the trained HMM model shows a very good recognition rate in experiments using test data in various noise environments.

That is, the speech recognition method for the robust long-range speech recognition system according to the present embodiment to which the noise synthesis method using the ICA as described above is applied, compensates for the limitation of the learning data that is limited in the prior art and various environments. It can be applied to, can greatly contribute to the commercialization of speech recognition technology.

A speech recognition method for a robust remote speech recognition system according to an embodiment of the present invention will be described with reference to the flowchart of FIG. 12 as follows.

The speech recognition method for the robust long-range speech recognition system according to the third embodiment of the present invention uses an adaptive modeling technique using a maximum a posteriori (MAP) and maximum likelihood linear regression (MLLR) technique in the pattern matching part of the continuous word recognition system. It is applied to improve the recognition rate of the system. That is, the speech recognition method for the robust remote speech recognition system according to the third embodiment of the present invention uses a MLLR-based model compensation technique to adapt a speech model suitable for a new environment by using a small amount of data. We will use the adaptive modeling technique to create the optimal model gradually by using the MAP technique while compensating the environment quickly and effectively.

In speech recognition systems, adaptive techniques are usefully used to solve speaker or environment variability problems. To solve speaker variability problems, MAP and MLLR methods are widely used for speaker adaptation. To change the characteristics of the test speaker similarly. In particular, it applies to Gaussian mean values in the HMM state. In addition, the MAP method is used when the adaptive data is large enough, and the MLLR method is used when the adaptive data is small.

)가 어떤 분포를 갖는 랜덤 변수라 가정하는데, 이때 만약에 해당 파라미터(

)가 하이퍼-파라미터(Hyper-parameter)(

)을 갖는 선확률 밀도 함수(

)와 유사도(

)를 갖는 관측 열로부터 추정된다면, 상기 MAP 적응 기 법은 다음과 같이 파라미터(

)의 포스티리어 모드(Posterior Mode)로 정의된다.The MAP adaptation technique is a technique for incorporating prior knowledge information included in training data into a linear density function and combining the adaptive data with the optimal data in an optimal manner. In that MAP, the parameters (

Is assumed to be a random variable with some distribution,

) Is a hyper-parameter (

Linear probability density function with

) And similarity (

If estimated from the observation column with), the MAP adaptation technique

) Is defined as the Postmode Mode.

)는 관측 열이 주어지기 전에 관심이 있는 파라미터에 대한 통계적 특성을 포함하여 파라미터가 어떤 제약된 값을 갖도록 한다. 상기 HMM과 같이 상태와 혼합 성분이 내재된 은닉 과정을 포함하는 경우에, 일반적으로 MAP 추정은 매우 어렵다. 그러나, HMM 파라미터의 선밀도 함수가 완전 데이터 밀도의 컨저게이트 패밀리(Conjugate Family)에 속한다면, EM 알고리즘에 의해 MAP 추정을 쉽게 수행할 수 있다.The linear density function (

) Gives the parameter some constraint on its value, including the statistical characteristics of the parameter of interest before the observation column is given. In the case of including the concealment process inherent in state and mixed components, such as the HMM, MAP estimation is generally very difficult. However, if the linear density function of the HMM parameter belongs to the conjugate family of full data density, the MAP estimation can be easily performed by the EM algorithm.

The MAP estimation more robustly estimates the parameters for less adaptive data than ML (Maximum Likelihood). The MAP has the advantage of converging to the ML estimate as the amount of adaptive data increases.

However, the MAP adapts only to the observed parameters, so that the adaptation is slow in the case of a large capacity recognizer having millions of parameters, but the MAP estimation is most effective in the speech recognition system when the parameters to be estimated are limited. to be.

Accordingly, the speech recognition method for the robust long-range speech recognition system according to the third embodiment of the present invention uses a modified form of the following adaptive expression, that is, a specific number (ie k) of tens of thousands of triphones. After performing pseudo-phoneme clustering with clusters of, model compensation is performed using MLLR, and then MAP adaptation is performed according to triphone-specific states investigated in the learning process.

First, centroid splitting algorithms cluster tens of thousands of triphones into similar phonemes into a specific number (i.e., k) clusters, adapting to store the necessary voice data for a small number of applications. In order to adapt tens of thousands of triphone models using a small amount of adaptation data read from the database, tens of thousands of triphone models are clustered such that k similar phoneme clusters are generated as shown in the flowchart of FIG. 13 (step S121).

In other words, as shown in the flowchart of FIG. 13, the parameter is first initialized, and one center value (ie, initial center value) for the mean of all the triphones is set as shown in Equation 13 below. (Step S131).

시켜 아래의 수학식 14와 같이 두 개로 분할한다(단계 S132).And splitting each cluster, moving the center value of each cluster

And divides it into two as shown in Equation 14 below (step S132).

Thereafter, all input vectors are reassigned, and the Euclidean distance with each divided cluster for all the input vectors is measured and grouped into the member vectors of the cluster having the smallest distance as shown in Equation 15 below (step S133). .

Then, the centroid update is performed, and the center point of the cluster is updated as shown in Equation 16 below through the member vector of each cluster (step S134).

Accordingly, the first termination is performed, and it is determined whether the update value of the error is equal to or less than the preset threshold (step S135). In this case, if the update value of the corresponding error is less than or equal to the preset threshold, step 136 (S136) is performed. Otherwise, the process returns to step 132 (S132).

Accordingly, when the second termination is performed, it is determined whether the partition is divided into a predetermined number of clusters (step S136). At this time, if the partition is divided into a predetermined number of clusters, the operation is terminated. Go back and do it again.

)이 사용된다.After performing the 121 th step S121, the HMM model is averagely compensated through the MLLR technique (step S122). Here, the MLLR is a model adaptation technique for predicting a set of linear transformations with respect to an average value of a Gaussian mixed HMM system. The effect of this transformation is to shift the state of the model, thereby transforming the state of the HMM system to suit the adaptive data. In order to predict the newly adapted mean value, a transformation matrix such as

) Is used.

'는 '

' 변환 행렬이고 'n'은 특징 파라미터 벡터 차수를 나타낸다. 해당 '

'은 확장 평균 벡터(Extended Mean Vector)로서, '

'이며, '

'는 바이어스 오프셋(Bias Offset)을 나타낸다.Where, "

'Is'

Is a transformation matrix and 'n' represents a feature parameter vector order. Applicable '

'Is an extended mean vector,

","

'Represents a bias offset.

)은, '

'로 분해된다. 여기서, 'b'는 바이어스 벡터를 나타내며, 'A'는 '

' 변환 행렬을 나타낸다.Therefore, the transformation matrix (

) Means "

Is decomposed into '. Here, 'b' represents a bias vector, and 'A'represents'

'Represents the transformation matrix.

Accordingly, in step 121, the clustered triphone models are trained, and adaptive data of the average compensated HMM model is calculated as shown in Equation 18 below according to the triphone-specific state investigated by the corresponding training. MAP adaptation is performed (step S123).

'은 적응 데이터에 대한 보움-웰취 재추정에 의해 추정된 적응 모델을 나타내며, '

'는 기존 학습 데이터의 개수를 나타내며, '

'은 적응 데이터의 개수를 나타낸다.here, '

'Represents the adaptive model estimated by the round-well re-estimation of the adaptation data,

'Represents the number of existing training data,

'Represents the number of adaptive data.

Thereafter, in step 123 (S123), a speech recognition process is performed using an adaptation model learned by MAP adaptation of the adaptation data (step S124).

As described above, the adaptive modeling technique using the MAP and MLLR techniques can be drastically reduced by only one to two weeks, compared to the existing system which took 6 months or more to replace the voice database depending on the application field. In other words, the voice data required for a small amount of application is recorded in the existing database for the areas that could not be applied because the tuning period of the voice database was too long for commercialization, and then the adaptive modeling method using the MAP and MLLR techniques described above is executed. In this way, an optimized voice database can be constructed in a very short period of time.

For example, as shown in FIG. 14, an adaptive database 92 storing the necessary voice data according to a small amount of application is added to a clean voice database (i.e., the original voice database) 91 as described above. The adaptive modeling technique using the MAP and MLLR techniques can be executed to construct the robot control database 93, the remote database 94 for the home network, and the database 95 for the high speed car.

On the other hand, the embodiment of the present invention as described above, this embodiment and the third embodiment compared to the prior art, for the evaluation of the distance speech recognition performance, composed of 30 men and women each for 100 sentences for home networks and robot control When 100 sentences are spoken, the home network and robot control sentences recorded from 0.3 (m), 1 (m), 2 (m), 3 (m) and 5 (m) away from the microphone are constructed. It can be seen that the voice recognition rate is as shown in the following tables (ie, Tables 1 to 5).

Table 1 below shows a speech recognition rate for the speech recognition system of the prior art.

100 sentences for home networks 100 sentences for robot control 0.3m 96.7 92.3 1m 90.3 91.2 2 m 85.5 82.9 3m 75.7 73.6 5 m 56.6 60.7

Table 2 below shows a speech recognition rate for the speech recognition system to which the left and right phoneme connection scheme according to an embodiment of the present invention is applied.

100 sentences for home networks 100 sentences for robot control 0.3m 96.5 93.1 1m 91.4 92.6 2 m 86.3 85.9 3m 82.5 80.4 5 m 66.9 71.8

Table 3 below shows a speech recognition rate for the speech recognition system to which the noise synthesis method using the ICA according to this embodiment of the present invention is applied.

100 sentences for home networks 100 sentences for robot control 0.3m 96.8 93.3 1m 94.0 93.0 2 m 88.9 83.4 3m 83.9 79.5 5 m 65.1 74.2

Table 4 below shows a speech recognition rate for the speech recognition system to which the adaptive modeling method using the MAP and MLLR techniques according to the third embodiment of the present invention.

100 sentences for home networks 100 sentences for robot control 0.3m 98.9 96.7 1m 95.6 95.1 2 m 89.7 90.2 3m 88.4 83.3 5 m 78.3 79.6

Table 5 below shows a speech recognition rate for the speech recognition system to which one embodiment, this embodiment, and three embodiments of the present invention are applied.

100 sentences for home networks 100 sentences for robot control 0.3m 99.0 97.8 1m 96.7 97.1 2 m 94.7 92.1 3m 92.3 89.6 5 m 82.3 83.5

As shown in the above tables, an optimal speech database using left and right phoneme connection technique according to an embodiment of the present invention, a noise synthesis technique according to this embodiment of the present invention, and an adaptive modeling technique according to the third embodiment of the present invention As a construction, it can be seen that it is very useful for constructing a robust far-field speech recognition system by coping with the change of the speech pattern resulting from the far-field speech.

A speech recognition method for a robust remote speech recognition system according to an embodiment of the present invention will be described with reference to the flowchart of FIG. 15 as follows.

In the speech recognition method for a robust long-range speech recognition system according to an embodiment of the present invention, the language processing unit in the post-processing portion of the speech recognition system applies a new reliability measurement method using a new halftone model to perform an unregistered word rejection function. In this case, the following new calculation technique is used to reduce the amount of computation that is increased when calculating the phoneme to half-phoneme likelihood ratio (LLR) for measuring reliability in Equation 19 below.

)은 새로운 반음소 모델이다. 기존의 반음소 모델을 이용한 신뢰도 계산이 자신 외의 모든 음소의 평균을 내거나, 유사 음소 집합을 미리 메모리에 저장하고 계산하는 것에 비해, 새로운 반음소 모델(

)은 자신의 음소 모델만을 사용하여 음소 대 반음소 우도비(LLR)를 다음과 같이 계산한다.Where the semitone model (

) Is the new halftone model. Reliability calculations using existing half-phone models mean that all other phonemes are averaged or similar phoneme sets are stored in memory beforehand and calculated.

) Calculates the phoneme-to-tone phone likelihood ratio (LLR) using only its phoneme model:

)을 '

'로 대치하여 해당 HMM 모델의 상태 천이를 자동적으로 자신의 상태 천이와 반대로 설정함으로써(단계 S151) 반음소의 특성을 지닐 수 있도록 해 준다.First, as shown in FIG. 16, the state transition probability of the HMM model (

)

By replacing the state transition of the corresponding HMM model with its own state transition automatically (step S151), so that it has the characteristics of the semitone.

)의 가우시안 모델에 대한 심볼 관측 확률도, 아래의 수학식 20과 같이 프레임을 역으로 취함으로써(단계 S152), 음소 모델이 가지는 특성에 반하는 반음소의 특성을 살릴 수 있도록 해 준다.And segmented frames (

The symbol observation probability for the Gaussian model of Fig. 9) is also reversed by taking the frame as shown in Equation 20 below (step S152), thereby making it possible to make use of the semi-phoneme characteristics contrary to the characteristics of the phoneme model.

'의 반음소 모델은 '

'로 대치되는데, 즉 새로 운 반음소 모델로 대치되어진다.At this time, '

Halftone model of '

', That is, the new halftone model.

Thus, the likelihood ratio of the phoneme to the half phoneme is represented by Equation 19 using the state transition probability replaced in step 151 and the symbol observation probability of taking an inverse frame in step 152 (S152). And reliability is measured (step S153).

Accordingly, when the reliability is measured as in the step 153 (S153), the likelihood ratio of the phoneme to the half phoneme can be adaptively measured even when the recognition target is changed or the quality of the voice is distorted by noise. Not only that, but it also reduces the amount of computation.

After that, a new word rejection function using the new halftone model as described above is applied to perform the non-registered word rejection function (step S154). In this case, as a result of the experiment, it can be seen that the novel half-phone model-based reliability calculation method of the present invention is an excellent speech verification technique that is robust against noise in continuous word recognition as well as isolated words, and has a small calculation amount.

A speech recognition method for a robust remote speech recognition system according to an exemplary embodiment of the present invention will be described with reference to the flowchart of FIG. 17 as follows.

In fact, in commercialization systems such as home networks and robot control, only less than 300 words are sufficient. Accordingly, in the speech recognition method for the robust long-range speech recognition system according to the fifth embodiment of the present invention, the language processing unit in the post-processing portion of the small-scale speech recognition system applies a rejection technique using a language model to perform an unregistered word rejection function. As a result, the probability of misrecognition is lowered, thereby improving the probability of rejection of unregistered words.

In other words, if the existing language modeling is focused on improving the recognition rate, the speech recognition method for the robust long-range speech recognition system according to the fifth embodiment of the present invention may be maintained while the recognition rate is the same as that of the existing speech recognition system. The focus was on rejecting unregistered words. At this time, if the network rejection scheme using the language model is configured as shown in FIG.

First, each node is set in advance so that the connection probability between words is the same so that all recognition objects can follow (step S171). In this case, the final recognition word string through the backpropagation path may be recognized only when it matches the predetermined recognition sentence.

)이 가중된다.In the recognition network as shown in Fig. 18, the first key word "Naraeya" and the second key word "room, living room, kitchen, ... boiler, air conditioner" and the like, "light up, turn off, turn on, turn off" If the model is modeled as having the same probability that the third core words are connected to each other among the third core words, such as "raise the temperature, lower the temperature", the language model (

) Is weighted.

After that, the sentence to be recognized is selected in such a state that the words and the words are all connected to each other, such as, "Naraya-bang-bang", "Narae-yabang-bang", "Narae-ja-bang," (Step S172).

Then, the recognition target sentence selected in the step 172 (S172), such as "naraeya", "home", "light up", "light off", "living room", "gas valve", "close", etc. Each is divided into words (step S173).

Thus, a recognition network as shown in FIG. 19 is constructed using the recognition target word divided in the first step S73 (step S174).

Accordingly, after the recognition of the voice input through the voice input device is performed through the recognition network configured in the step 174 (step S174) (step S175), the final recognition word string output through the corresponding recognition network is the above-described code 172. It is checked whether it matches the sentence to be recognized selected in step S172 (step S176), and if it matches at this time, the recognition processing is performed (step S177), while if it does not match, the rejection processing is performed (step S178).

The speech recognition method for the robust long-range speech recognition system according to the fifth embodiment of the present invention as described above is based on the characteristics of the large-capacity speech recognition system having a relatively low recognition rate using the characteristics of the speech recognition system having a higher recognition rate as the recognition scale is smaller. The language modeling technique used to improve the recognition rate is applied to the rejection function of the small-capacity speech recognition system.

For example, if a sentence consisting of three consecutive words is to be recognized, in theory, it has a probability of rejecting an unregistered word of 1/300 * 300 * 300 = 99.99999 (%). In fact, when 300 sentences of three consecutive nodes are recognized on a network of 300 words, it is well understood that a rejection probability for an unregistered word is higher than 99 (%).

A speech recognition method for a robust long-range speech recognition system according to six embodiments of the present invention is as follows.

A speech recognition method for a robust long-range speech recognition system according to a sixth embodiment of the present invention is a hybrid rejection technique. Language modeling according to a fifth embodiment of the present invention allows to perform an out-of-vocabulary (OOV) rejection function sequentially.

For example, in order to train the Triphone model used in the simulation, a total of eight types (ie, ETRI's PBW445 database, POW3848 database, the Korean Institute of Engineering PBW452 database, four-digit digits, single digits, high frequency) 2000 vocabulary database, voice database of the newly collected home network control command database according to the third embodiment of the present invention, intelligent robot database), and the database for evaluation is a variety of control collected according to six embodiments of the present invention Use a sentence database. In addition, the number of triphones, a context-dependent model of the speech database, participated in the training was 10,349 including the Silence model, and the speech database was sampled at 16 (k) and linearized at 16 (Bit). Have the format of PCM.

At this time, the speech database participating in the recognition experiment is shown in Table 6 below. Also, comparing the OOV probability of the speech verification method according to the prior art and the hybrid rejection method according to the six embodiments of the present invention, same.

Database type Vocabulary Repetitions per vocabulary PDA Control Database 80 200 times Database for Home Network Control 300 200 times Database for Intelligent Robot Control 150 200 times

Conventional Speech Verification Technique Speech Verification Technique of the Invention Rejection technique using language model of the present invention Hybrid Rejection Techniques of the Invention Recognition rate Unregistered Word Rejection Rate Recognition rate Unregistered Word Rejection Rate Recognition rate Unregistered Word Rejection Rate Recognition rate Unregistered Word Rejection Rate For PDA control 93.6 94.1 93.6 97.8 93.6 98.9 94.5 99.5 For home networks 92.2 85.5 92.1 92.5 92.5 99.7 93.2 100 For robot control 91.3 92.8 90.5 95.6 91.3 96.6 90.8 98.3

As shown in Table 7, the speech recognition method for the robust long-range speech recognition system according to six embodiments of the present invention shows a high performance unregistered word rejection rate of about 99 (%) without a change in recognition rate.

The speech recognition method for the robust long-range speech recognition system according to the sixth embodiment of the present invention, while maintaining the recognition rate of the small-scale continuous speech recognition system consisting of less than 300 words, greatly improves the non-registered word recognition rate commercialization of the continuous speech recognition system Can contribute significantly. In addition, by performing a non-registered word rejection function without a large change in the amount of calculation, it is possible to apply to an embedded speech recognition system such as DSP or PDA, which has not been applied due to the increase of the amount of calculation.

As described above, in the speech recognition system according to the present invention, the recognition rate of the long-range speech recognition system (i.e., the far-field speech recognition system) is applied by applying the left and right phoneme connection technique and the noise synthesis and adaptive algorithm for constructing the speech database in the pattern matching part. Performance, and by using the new half-phoneme model or the new language model in the post-processing part to perform the rejection function for unregistered words, the probability of misrecognition is lowered, thereby improving the rejection probability for unregistered words. Reduce the computation of the phoneme-to-tone phoneme likelihood ratio to measure.

In addition, according to the present invention, in the post-processing part of the speech recognition system, the speech verification using the new half-phone model and the sequential unregistered word rejection function applying the language model for the small-scale continuous speech recognition system are performed. It greatly improves the recognition rate of unregistered words while maintaining the recognition rate, and contributes to the commercialization of the continuous speech recognition system, and it can be applied to embedded speech recognition systems such as DSP or PDA, which have not been applied due to the increase of the calculation amount.

Claims (7)

delete In the speech recognition method for recognizing the voice of the words continuously input through the voice input device, Acquiring a noise signal generated in a plurality of noise environments through a plurality of voice input devices to implement a noise signal database; Combining the obtained noise signals with a clean original audio signal to generate mixed signals; Learning the generated mixed signals to acquire learning data collected in a plurality of noise environments to implement learning data databases; A step of converting the acquired training data into raw data in a plurality of noise environments in a model re-estimation system and learning the HID model; And a step of recognizing a speech for the continuous input word using the trained HMM model. In the speech recognition method for recognizing the voice of the words continuously input through the voice input device, Adding an original audio database storing clean original audio signals to an adaptive database storing necessary voice data according to an application field; Clustering the plurality of triphone models such that a specific number of similar phoneme clusters are generated to adapt the plurality of triphone models using the adaptation data stored in the adaptation database; Estimating a set of linear transformations for the mean value of the Gaussian mixed Hidden Markov Model (HMM) model through Maximum Likelihood Linear Regression (MLLR) to average compensate the corresponding HMM model; Learning the clustered triphone models and performing MAP adaptation of the adaptation data on the average compensated HMM model according to the triphone-specific state investigated by the training; And a speech recognition method for the continuous input word using the adaptation model learned by the MAP adaptation. The method of claim 3, The triphone model clustering process may include: setting an initial center value for the average of all the triphones, and then moving the center value of each cluster to all input vectors and dividing the cluster into two; Measuring the Euclidean distance with each divided cluster for all input vectors and grouping them into a member vector of the cluster having the smallest distance; Updating a center point of the cluster through the member vectors of the clusters, and then determining whether an update value of the corresponding error is less than or equal to a preset threshold; Determining whether the update value of the error is divided into a predetermined number of clusters when the update value of the error is less than or equal to a predetermined threshold value; And terminating the triphone model clustering process when the cluster is divided into the predetermined number of clusters. The method of claim 4, wherein In the triphone model clustering process, when the error update value is not equal to or less than a predetermined threshold value or is not divided into a predetermined number of clusters, the step of moving the center value of each cluster and dividing it into two is repeated. A voice recognition method for a robust long-range speech recognition system, characterized in that it further comprises the step of performing. In the speech recognition method of measuring the reliability by using a Hidden Markov Model (HMM) model for modeling the speech in the language processing unit in the post-processing portion, and performing the non-registered word rejection function by the corresponding reliability measurement, Replacing the state transition probability of the HMM model with a value of '1-state transition probability' so that the state transition of the HMM model is set opposite to the state transition of the HMM model so as to have semitone characteristics; Taking a corresponding frame observation probability for the Gaussian model of the segmented frame in reverse order to make use of the semitone features; And calculating the likelihood ratio of a phoneme to a half phoneme using the replaced state transition probability and the symbol observation probability of the inverse frame, and measuring the reliability. Way. In the speech recognition method of performing a non-registered word rejection function by applying a rejection technique using a language model in the language processing unit in the post-processing portion, Pre-modeling the words and the connection probabilities between words so that all recognition objects follow each node; Selecting a sentence to be recognized in a state in which words and words are all connected, and then dividing the corresponding sentence to be recognized into words to be recognized; Constructing a recognition network using the divided recognition target word, and then performing recognition of an input voice through the corresponding recognition network; Checking whether a final recognized word string output through the recognition network matches the recognized sentence; And performing a non-registered word rejection process when the final recognized word string does not match the sentence to be recognized.

Images