KR20100056859A - Voice recognition apparatus and method - Google Patents

Abstract

PURPOSE: A voice recognizing device and a method thereof are provided to compensate a sound model for remaining noise in a mask-based multichannel sound source separating method for voice recognition, thereby improving voice recognition performance. CONSTITUTION: A voice mask predicting unit(20) predicts a voice mask from a multichannel voice signal. A noise component remover(30) removes a noise component using the predicted voice mask. A voice combiner(40) combines voices using the multichannel voice signal from which the noise component is removed. A sound model generator(200) predicts an SNR(Signal to Noise Ratio) and a noise model using the voice mask. The sound model generator generates a sound model adapted to noise.

Description

Voice recognition apparatus and method

The present invention relates to a speech recognition apparatus and method for operating in a noisy environment, and more particularly, to a speech recognition apparatus and method for removing noise components left after being processed by a multi-channel sound source separation technique.

Voice is the most basic and natural means of communication. Human hearing systems can select and perceive only the sounds you want in a variety of noisy environments. To this end, the human auditory system uses the inter-aural time difference (ITD) and the inter-aural level difference (ILD) of signals coming through both auditory nerves to determine the direction of the sound source that produces the desired sound. After finding, the desired sound is separated from the sounds from other sources.

Similarly, the CASA (computational auditory scene analysis), a mask-based multichannel sound source separation technique, uses ITD and ILD between signals input to two microphones. After calculating the time-frequency mask from the ITD and ILD, the desired speech signal can be separated by applying the mask to the noisy speech signal.

FIG. 1 shows an example of a synthesized signal of noise canceled speech using a Gaussian kernel-based mask. For a method of Gaussian kernel-based mask estimation, see DL Wang and GJ Brown, Computational Auditory Scene Analysis: Principle, Algorithms and Applications , IEEEPress, Wiley-Interscience, 2006 and N.Roman, DLWang, and GJBrown. , "Speech segregation based on sound localization," J. Acoust. Soc. Amer ., Vol. 114, no. 4, pp. 2236-2252, July 2003.

(A) of FIG. 1 shows a speech signal without noise, and (b) of FIG. 1 shows a speech signal with noise (0 dB SNR), and noise is removed through the Gaussian kernel-based mask of FIG. Voice signal.

As shown in FIG. 1, when the signal (c) from which the noise is removed is compared with the noise-free speech signal (a), it can be seen that the noise signal remains. When using the signal with the residual noise, it is necessary to compensate for the residual noise since it may cause a deterioration of speech recognition performance.

As described above, since the ideal voice mask cannot be obtained in estimating the voice mask in a real environment, residual noise is generated, which causes a limitation in speech recognition performance.

In other words, when the voice signal is removed through the multi-channel sound source separation technology, the noise is not completely removed, which leads to a limitation in improving the speech recognition performance. This is particularly limited in low signal-to-noise ratio environments.

Therefore, there is an urgent need for a technique that can further improve speech recognition performance through a process of estimating an acoustic model adapted to noise by using noise masks obtained by conventional methods efficiently.

The present invention has been proposed to solve the above problems and to meet the above requirements, and an object thereof is to provide a speech recognition apparatus and method for improving speech recognition performance in a noisy environment.

According to an aspect of the present invention, there is provided a speech recognition apparatus, including: a speech mask estimator configured to estimate a speech mask from a multichannel speech signal; A noise component removing unit for removing noise components using the voice mask estimated by the voice mask estimating unit; A speech synthesizer for synthesizing speech using the multi-channel speech signal from which the noise component is removed from the noise component remover; An acoustic model generator for estimating a noise model and a signal-to-noise ratio using the voice mask and generating an acoustic model adapted to noise using the noise model and the signal-to-noise ratio; A feature extractor which extracts a voice feature from the voice signal output from the voice synthesizer; And a decoding unit for obtaining a speech recognition result using an acoustic model adapted to the speech feature and noise obtained by the feature extractor.

Here, the voice mask estimator comprises: a gamma-tone filter for separating the voice signal received from the outside into various frequency bands; An inter-channel time difference estimator estimating a time difference between microphone channels from the signal separated by the gamma-tone filtering unit; An inter-channel level difference estimator estimating a level difference between microphone channels from the signal separated by the gamma-tone filtering unit; And a voice mask calculator for obtaining a voice mask using the time difference between the microphone channels and the level difference between the microphone channels.

The acoustic model generator may include a noise mask calculator configured to obtain a noise mask from the voice mask estimated by the voice mask estimator; A mask-based noise model estimator for obtaining a noise model using the noise mask and a gamma-tone filtered signal; A mask-based signal-to-noise ratio estimator for obtaining a signal-to-noise ratio using the speech mask and the noise mask; And an acoustic model adaptor for obtaining an acoustic model adapted to noise using the estimated noise model and the estimated signal-to-noise ratio.

The mask-based noise model estimator may include a noise mask calculator configured to obtain a noise mask using the voice mask; A speech removing unit for removing a speech component by multiplying the noise mask with the gamma-tone filtered signal; A noise synthesizer configured to synthesize a noise signal from the signal from which the speech component is removed; A feature extractor which extracts a feature of the noise signal; And a noise model calculator for obtaining a noise model using the characteristics of the noise signal.

Here, the mask-based signal-to-noise ratio estimator includes: a voice mask averaging calculator for obtaining an average value on a frequency channel of the voice mask; A noise mask averaging unit for obtaining an average value on the frequency channel of the noise mask; A noise frame detector for detecting a noise frame using the average value of the voice mask; And a signal-to-noise ratio calculation unit for obtaining a speech mask mean value on a frequency and a noise mask mean value on a frequency for the noise frame, and obtaining a signal-to-noise ratio as a ratio between these average values.

In addition, the speech recognition method according to an embodiment of the present invention comprises the steps of estimating the speech mask from the multi-channel speech signal; Removing noise components using the estimated voice mask; Synthesizing speech using the multi-channel speech signal from which the noise component is removed; Estimating a noise model and a signal-to-noise ratio using the voice mask, and generating an acoustic model adapted to noise using the noise model and the signal-to-noise ratio; Extracting a speech feature from the synthesized speech signal; And obtaining a speech recognition result using the acoustic model adapted to the speech feature and noise.

Here, the voice mask estimating step may include: separating a voice signal received from the outside into several frequency bands using gamma-tone filtering; Estimating time difference between microphone channels from the separated signal; Estimating a level difference between microphone channels from the separated signal; And obtaining a voice mask using the time difference between the microphone channels and the level difference between the microphone channels.

Here, the generating of the acoustic model may include obtaining a noise mask from the estimated speech mask; Estimating a noise model using the noise mask and a gamma-tone filtered signal; Estimating a signal-to-noise ratio using the speech mask and the noise mask; And obtaining an acoustic model adapted to noise by using the estimated noise model and the estimated signal-to-noise ratio.

The estimating of the noise model may include obtaining a noise mask using the voice mask; Removing speech components by multiplying the noise mask with the gamma-tone filtered signal; Synthesizing a noise signal from the signal from which the speech component has been removed; Extracting features of the noise signal; And obtaining a noise model using the characteristics of the noise signal.

The estimating of the signal-to-noise ratio may include obtaining an average value on a frequency channel of the voice mask; Obtaining an average value on a frequency channel of the noise mask; Detecting a noise frame using the average value of the voice mask; And obtaining a mean value of speech mask on frequency and a mean value of noise mask on frequency for the noise frame, and obtaining a signal-to-noise ratio as a ratio between these mean values.

According to the present invention, there is an effect of improving the speech recognition performance by compensating the acoustic model for the noise left in the mask-based multi-channel sound source separation technique for speech recognition.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram of an embodiment of a speech recognition apparatus according to the present invention.

The speech recognition apparatus according to the present invention includes an A / D converter 10, a voice mask estimator 20, a noise component remover 30, a speech synthesizer 40, an acoustic model generator 200, and a feature extractor. 310 and the Viterbi decoding unit 320. The portion indicated by 100 is a multi-channel sound source separation module, and the portion indicated by 300 is a speech decoding unit.

The A / D converter 10 converts a multichannel audio signal, for example, a right channel audio signal and a left channel audio signal, respectively, input from external multichannel microphones (microphone 1 and microphone 2) into digital signals, respectively.

The voice mask estimator 20 is connected to an output terminal of the A / D converter 10 to receive a digital multi-channel voice signal from the A / D converter 10 and estimate a voice mask from the multi-channel voice signal. 3 shows a detailed configuration diagram of the voice mask estimator 20.

Referring to FIG. 3, the voice mask estimator 20 includes a gamma-tone filter 21, a signal difference estimator 22 between channels, a level calculator 23, a level difference estimator 24 between channels, and a voice. And a mask calculator 25.

The gamma-tone filter 21 separates the voice signal received from the outside into several frequency bands. For example, in the present embodiment, the gamma-tone filter 21 separates signals input from two microphones for each frequency band. The gamma-tone filter 21 may be implemented in the form of a filter bank.

The time difference estimator 22 estimates the time difference between microphone channels from the signal separated by the gamma-tone filter 21. The level calculator 23 takes an absolute value of the signal sample value separated by the gamma-tone filter 21 and adds the absolute value to obtain a signal level value.

The level difference estimator 24 estimates the level difference between microphone channels by obtaining a difference between signal level values output from the level calculator 23.

로 표현된다. 여 기서

는 주파수 채널 인덱스이고,

는 프레임 인덱스이다. The voice mask calculator 25 calculates a voice mask using the time difference between the microphone channels and the level difference between the microphone channels. For example, the voice mask calculator 25 calculates a voice mask based on the Gaussian kernel. In this case, the voice mask has a value between 0 and 1. Since the voice mask

It is expressed as here

Is the frequency channel index,

Is the frame index.

Referring back to FIG. 2, the noise component remover 30 removes a noise component from a voice signal having a noise component by using the voice mask estimated by the voice mask estimator 20. The speech synthesizer 40 synthesizes speech using the signal from which the noise component output from the noise component remover 30 is removed. In this case, the speech synthesis unit 40 preferably uses the method disclosed in the paper M. Weintraub, A Theory and Computational Model of Monaural Auditory Sound Separation , Ph.D.Thesis, Stanford University, 1985.

As described above, the speech recognition apparatus according to the present invention primarily removes noise components by using a multi-channel sound source separation technique. However, the speech signal output from the speech synthesis section 40 has a residual noise component, as shown in Fig. 1C.

In order to remove the residual noise component, the present invention efficiently estimates the noise model and the signal-to-noise ratio using the speech mask used in the multi-channel sound source separation, and uses the noise model and the signal-to-noise ratio to reduce the noise. The speech recognition performance is improved by estimating the adaptive acoustic model. The present invention includes an acoustic model generator 200 to estimate the acoustic model adapted to the noise.

The acoustic model generator 200 generates an acoustic model for compensating for the residual noise component of the speech signal. In detail, the acoustic model generator 200 multiplies the SNR weights of a clean-trained model and a noise model learned in a clean environment with a noise model adapted to the noise environment. estimate the corrupted model.

To this end, the acoustic model generator 200 includes a component for mask-based noise model estimation (MBNME) and a component for mask-based SNR estimation (MBSE). The detailed configuration of such an acoustic model generator 200 will be described below.

4 illustrates a configuration of the acoustic model generator 200 of FIG. 2.

Referring to FIG. 4, the acoustic model generator 200 may include a mask-based noise model estimator 210, a mask-based SNR (signal-to-noise ratio) estimator 220, an acoustic model adaptor 230, and no sound. Noise acoustic model 240 and acoustic model 250 adapted to noise.

The mask-based noise model estimator 210 calculates a noise mask using a voice mask and obtains a noise model using the noise mask and a gamma-tone filtered signal. In more detail, the mask-based noise model estimator 210 synthesizes a noise signal using noise mask information obtained from the voice mask obtained by the voice mask estimator 20, and estimates a noise model from the noise signal.

FIG. 5 is a diagram illustrating an embodiment of the mask-based noise model estimator 210 of FIG. 4. The mask-based noise model estimator 210 estimates a noise model using voice mask information provided from the voice mask estimator 20 and an arbitrary microphone signal filtered by the gamma tone-filter 21.

,를 값 1로부터 도 3에서 구한 음성마스크를 차감하여 [수학식 1]과 같이 구한다. Referring to FIG. 5, the mask-based noise model estimator 210 includes a noise mask calculator 211, a speech remover 212, a noise synthesizer 213, a feature extractor 214, and a noise model calculator 215. ) In order to estimate the noise model, the noise mask calculator 211 first obtains a noise mask using a voice mask. Specifically, the noise mask calculator 211 includes a noise mask,

, Is obtained as shown in Equation 1 by subtracting the voice mask obtained in FIG.

를 감마-톤 필터링된 마이크로폰1의 신호에 곱함으로써 음성성분을 제거한다. 이 경우, 다채널 신호들 중 어떠한 채널 신호라도 사용될 수 있다. 즉, 본 실시예에서는 마이크로폰1의 신호가 사용되었지만, 마이크로폰2의 신호가 사용될 수도 있으며, 다른 다채널 신호도 사용될 수 있다. Next, the voice remover 212 is the noise mask.

Multiply the signal of the gamma-tone filtered microphone 1 to remove the speech component. In this case, any of the multichannel signals may be used. That is, although the signal of microphone 1 is used in this embodiment, the signal of microphone 2 may be used, and other multichannel signals may also be used.

,k=1,…, K,를 추출한다. 이 때, MFCC(Mel-Frequency Cepstral Coefficient)의 차수, K, 는 시스템 및 응용예에 따라 다르게 결정될 수 있다. 잡음모델 연산부(215)는 [수학식 2]와 같이 추출된 MFCC로부터 평균값

과 분산값

을 구하여 잡음모델로서 사용한다.The noise synthesizer 213 synthesizes a noise signal using the signal from which the speech component is removed and outputs the noise signal to the feature extractor 214. The feature extractor 214 may use a MFCC (Mel-Frequency Cepstral Coefficient),

, k = 1,… Extract K, At this time, the order, K, of the Mel-Frequency Cepstral Coefficient (MFCC) may be determined differently according to a system and an application example. The noise model calculator 215 may calculate an average value from the MFCC extracted as shown in [Equation 2].

And variance

We obtain and use as noise model.

은 전체 프레임 수를 나타낸다.here

Represents the total number of frames.

Referring back to FIG. 4, the mask-based SNR estimator 220 obtains a signal-to-noise ratio using the voice mask and the noise mask. The mask-based SNR estimator 220 estimates the SNR based on a ratio of the average value between the speech mask obtained by the speech mask estimator 20 and the noise mask obtained from the mask-based noise model estimator 210.

FIG. 6 is a diagram illustrating an embodiment of the mask-based SNR estimator 220 of FIG. 4. That is, the signal-to-noise ratio is estimated using the speech mask calculated by the speech mask estimator 20 of FIG. 2 and the noise mask calculated by the mask-based noise model estimator 220 of FIG. 5.

Referring to FIG. 6, the mask-based SNR estimator 220 includes a voice mask average calculator 221, a noise mask average calculator 222, a noise frame detector 223, and a signal-to-noise ratio calculator 224. It gets warm.

,로부터 주파수 채널상에서 음성 마스크의 평균값,

,을 [수학식 3]을 이용하여 구한다. In the voice mask averaging unit 221, the voice mask estimated by the voice mask estimator 20 of FIG. 2,

The average value of the voice mask on the frequency channel from

And are calculated using Equation 3.

는 감마-톤 필터링부의 채널수이다. 여기에서 채널수는 32가 될 수 있으며, 그 적용예 또는 구현에 따라 달라질 수 있다. 이와 유사하게, 잡음마스크 평균 연산부(221)에서는 도 4의 마스크-기반 잡음모델 추정부(210)에서 계산된 잡음마스크,

,로부터 주파수 채널상에서 잡음 마스크의 평균값,

,을 [수학식 4]를 이용하여 구한다.here,

Is the number of channels in the gamma-tone filtering unit. Here, the number of channels may be 32, and may vary depending on the application or implementation thereof. Similarly, the noise mask averaging unit 221 calculates the noise mask calculated by the mask-based noise model estimator 210 of FIG.

The average value of the noise mask on the frequency channel from

And are obtained using Equation 4.

,을 이용하여 잡음 프레임을 검출한다. 이를 위해, 잡음 프레임 검출부(223)는 [수학식 5]와 같이 가장 먼저 초기 20프레임으로부터 음성 마스크의 주파수-시간상 평균값,

, 과 분산값,

,을 구한다.The noise frame detector 223 may calculate a voice mask average value obtained by the voice mask averaging unit 221 to obtain an accurate signal-to-noise ratio.

Detect noise frame using,. To this end, the noise frame detection unit 223 is the frequency-time average value of the speech mask from the first 20 frames, as shown in Equation 5,

, And variance,

Find,

은 상기 초기 프레임들의 개수이며, 본 실시예에서 20이다. 이러 한 프레임의 개수는 그 적용예 또는 구현에 따라 달라질 수 있으며, 본 발명에 이에 한정되지 않는다. 그 다음으로 잡음 프레임 검출부(223)는 초기 20프레임의 약 90% (즉, 18프레임 정도)가 잡음 프레임으로 선택되는 상수값,

를 설정하여 문턱값,

,을 [수학식 6]과 같이 구한다.here

Is the number of the initial frames, 20 in this embodiment. The number of such frames may vary depending on the application or implementation thereof, and is not limited thereto. Next, the noise frame detection unit 223 has a constant value in which about 90% of the initial 20 frames (that is, about 18 frames) is selected as the noise frame,

To set the threshold,

Find, as shown in [Equation 6].

,을 구한다.Next, the noise frame detection unit 223 aggregates a frame having a voice mask frequency-time average value that does not exceed the obtained threshold value through Equation 7,

Find,

Then, the signal-to-noise calculation unit 224 obtains the voice mask frequency average value g _S and the noise mask frequency average value g _N on the noise frame set from Equation 8, respectively, from the voice mask frequency average value and the noise mask frequency average value. .

는 잡음 프레임 집합,

,에 속하는 프레임 개수이다. 최종적으 로 신호-대-잡음비 즉 SNR, g,는 [수학식 9]와 같이 구한다.here

Is a set of noise frames,

Number of frames belonging to,. Finally, the signal-to-noise ratio, ie, SNR, g, is obtained as shown in [Equation 9].

,

)과 마스크-기반 신호-대-잡음비 추정부(220)에서 추정된 신호-대-잡음비(g)를 이용하여 미리 잡음이 없는 음성들로 학습해놓은 무잡음 음향모델(

,

)(240)를 잡음에 적응시킨 새로운 모델(

,

)(250)을 [수학식 10]과 같이 추정한다.Referring back to FIG. 4, the acoustic model adaptor 230 may extract the noise model estimated by the mask-based noise model estimator 210 and the signal-to-noise ratio estimated by the mask-based SNR estimator 220. The acoustic model 250 adapted to the noise is obtained. In other words, the acoustic model adaptation unit 230 estimates a noise model estimated by the mask-based noise model estimating unit 210.

,

) And a noise-free acoustic model trained on speechless noises using the signal-to-noise ratio (g) estimated by the mask-based signal-to-noise ratio estimator 220 (

,

) Is a new model that adapts 240 to noise.

,

) 250 is estimated as shown in [Equation 10].

This is described in the papers M. Gales and S. J. Young, "Robust continuous speech recognition using parallel model combination," IEEE Trans. Speech and Audio Proc., Vol. 4, no. 5, pp. 352-359, Sept. See 1996.

Referring back to FIG. 2, the voice decoding unit 300 extracts a voice feature, for example, Mel Frequency Cepstral Coefficient (MFCC) from the voice signal output from the voice synthesizer 40, and then from the acoustic model generator 200. Viterbi decoding is performed using the output acoustic model and the extracted speech feature to output a speech recognition result.

In detail, the voice decoding unit 300 includes a feature extractor 310 and a Viterbi decoder 320. The feature extractor 310 extracts a speech feature, for example, MFCC, from the speech signal from which the noise is first removed from the speech synthesizer 40. The Viterbi decoding unit 320 recognizes a word or sentence having the highest probability value through pattern matching with the acoustic model adapted to the noise obtained by the acoustic model adaptor 230 and the MFCC extracted by the feature extractor 310. Get the result. Viterbi decoding is a general method for obtaining speech recognition results. The pattern matching of acoustic models and MFCC is performed to select the acoustic model having the greatest probability, and the phoneme, word or sentence string corresponding to the acoustic model is selected. Refers to the process that results from recognition.

7 is a flowchart of a method for improving speech recognition performance in a speech recognition apparatus according to the present invention.

Referring to FIG. 7, the speech recognition apparatus determines whether a multi-channel speech signal is input in step 410. In detail, the speech recognition apparatus determines whether a speech signal is input from each of the plurality of microphones. When the multi-channel voice signal is input, the voice recognition apparatus estimates a voice mask from the multi-channel voice signal in step 420.

In detail, in operation 410, the apparatus for recognizing speech divides the speech signal into several frequency bands by performing gamma-tone filtering on the speech signal received from the outside. The speech recognition apparatus estimates a time difference between microphone channels and a level difference between microphone channels from the separated signals, and estimates a voice mask using the time difference between microphone channels and a level difference between microphone channels.

Subsequently, the speech recognition apparatus estimates the speech mask and then, in step 430, removes noise components from the multichannel speech signal using the speech mask and synthesizes speech using another channel speech signal from which the noise components are removed. After synthesizing the speech, the speech recognition apparatus extracts a speech feature, for example, a mel-frequency cepstral coefficient (MFCC), from the speech synthesized in step 440.

The speech recognition apparatus then estimates the noise model and signal-to-noise ratio using the speech mask in step 450 and generates an acoustic model adapted to noise.

 To estimate the noise model, the speech recognition apparatus obtains a noise mask using a speech mask and removes speech components by multiplying the noise mask by a gamma-tone filtered signal. The speech recognition apparatus synthesizes a noise signal from a signal from which speech components are removed and extracts a feature, ie, MFCC, from the synthesized noise signal. The speech recognition apparatus then estimates a noise model from this feature.

In addition, in order to estimate the signal-to-noise ratio, the speech recognition apparatus may estimate the SNR through the ratio of the average value between the speech mask and the noise mask. To this end, the speech recognition apparatus obtains an average value on the frequency channel of the voice mask and obtains an average value on the frequency channel of the noise mask. The speech recognition apparatus detects a noise frame by using a speech mask mean value. That is, the speech recognition apparatus calculates a threshold value that becomes a noise frame based on the frequency-time average value of the voice mask, and determines frames having a voice mask frequency-time average value that does not exceed the threshold value as the noise frame. The speech recognition apparatus obtains a speech mask average value and a noise mask average value for these noise frames, and then obtains a signal-to-noise ratio as a ratio between these average values.

As such, the speech recognition apparatus estimates the noise model and the signal-to-noise ratio, and then uses the estimated noise model and the signal-to-noise ratio, and uses a noisy acoustic model that has previously been learned with no-noise speech. Estimate acoustic model adapted to noise.

In operation 470, the speech recognition apparatus may obtain a speech recognition result by performing Viterbi decoding using an acoustic model and a speech feature adapted to noise.

FIG. 8 shows the female reading sound as noise, and the word speech DB (S. Kim, S. Oh, H.-Y. Jung, H.-B. Jeong, and J.-S. Kim, "Common speech database collection , "Proceedings of the Acoustical Society of Korea, Vol. 21, No. 1, pp. 21-24, July 2002), shows the recognition performance of the present invention in terms of word misrecognition (%). 18,240 words were used for the acoustic model training and 570 words were used for the recognition test. The head transfer function was placed at 0 ° and noise at 10 °, 20 ° and 40 °. Applied. The speech recognition system is based on a triphone hidden Markov model, and each triphone is represented by a left-to-right model with three states. Each state has four Gaussian mixtures, and the number of words used is 2,250 words. The performance compared in FIG. 5 is based on baseline performance using MFCC without noise processing (Baseline), performance of mask-based multi-channel sound source separation technology (MMSS), performance of the present invention (MMSS) Acoustic model combination (AMC). The performance of mask-based multi-channel sound separation technology shows that word recognition is relatively high at low signal-to-noise ratios, compared to very low word-to-noise ratios at high signal-to-noise ratios. This is determined by the influence of the residual noise remaining after the mask-based multi-channel sound source separation process, and the performance of the present invention compensating for this shows that the word misrecognition rate is significantly lowered even at a low signal-to-noise ratio. As a result, the word misrecognition rate was reduced by 52.14% compared to the mask-based multi-channel sound separation processing technology.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

FIG. 1 is a diagram illustrating an example of a synthesized signal of speech with noise removed using a Gaussian kernel-based mask.

2 is a diagram illustrating a configuration of an embodiment of a speech recognition apparatus according to the present invention.

FIG. 3 is a block diagram illustrating an embodiment of a voice mask estimator of FIG. 2.

4 is a block diagram illustrating an embodiment of an acoustic model generator of FIG. 2.

5 is a block diagram illustrating an embodiment of the mask-based noise model estimator of FIG. 4.

FIG. 6 illustrates a block diagram of an embodiment of the mask-based SNR estimator of FIG. 4.

7 is a flowchart illustrating a speech recognition method according to an embodiment of the present invention.

FIG. 8 is a diagram illustrating the recognition performance of the present invention as a word false recognition rate (%), which is performed by using a female voice as a noise and recognized using a word speech DB.

<Explanation of symbols for main parts of the drawings>

10; A / D converter

20; Voice Mask Estimator

30; Noise Component Rejection Unit

40; Speech synthesis

200; Acoustic model generator

310; Feature Extraction Unit

320; Viterbi decoding unit

Claims (10)

In the speech recognition device, A voice mask estimator for estimating a voice mask from a multi-channel voice signal; A noise component removing unit for removing noise components using the voice mask estimated by the voice mask estimating unit; A speech synthesizer for synthesizing speech using the multi-channel speech signal from which the noise component is removed from the noise component remover; An acoustic model generator for estimating a noise model and a signal-to-noise ratio using the voice mask and generating an acoustic model adapted to noise using the noise model and the signal-to-noise ratio; A feature extractor which extracts a voice feature from the voice signal output from the voice synthesizer; And And a decoding unit for obtaining a speech recognition result using an acoustic model adapted to the speech feature and noise obtained by the feature extractor. The voice mask estimator of claim 1, wherein the voice mask estimator A gamma-tone filter which separates an audio signal input from an external device into various frequency bands; An inter-channel time difference estimator estimating a time difference between microphone channels from the signal separated by the gamma-tone filtering unit; An inter-channel level difference estimator estimating a level difference between microphone channels from the signal separated by the gamma-tone filtering unit; And And a voice mask calculator configured to obtain a voice mask by using the time difference between the microphone channels and the level difference between the microphone channels. The method of claim 2, wherein the acoustic model generating unit A noise mask calculator for obtaining a noise mask from the voice mask estimated by the voice mask estimator; A mask-based noise model estimator for obtaining a noise model using the noise mask and a gamma-tone filtered signal; A mask-based signal-to-noise ratio estimator for obtaining a signal-to-noise ratio using the speech mask and the noise mask; And And an acoustic model adaptor for obtaining an acoustic model adapted to noise using the estimated noise model and the estimated signal-to-noise ratio. The method of claim 3, wherein the mask-based noise model estimator A noise mask calculator which obtains a noise mask using the voice mask; A speech removing unit for removing a speech component by multiplying the noise mask with the gamma-tone filtered signal; A noise synthesizer configured to synthesize a noise signal from the signal from which the speech component is removed; A feature extractor which extracts a feature of the noise signal; And And a noise model calculator for obtaining a noise model using the features of the noise signal. 5. The apparatus of claim 4, wherein the mask-based signal-to-noise ratio estimator A voice mask averaging unit for obtaining an average value on a frequency channel of the voice mask; A noise mask averaging unit for obtaining an average value on the frequency channel of the noise mask; A noise frame detector for detecting a noise frame using the average value of the voice mask; And And a signal-to-noise ratio calculation unit for obtaining a mean value of speech mask on frequency and a mean value of noise mask on frequency for the noise frame, and obtaining a signal-to-noise ratio as a ratio between these average values. In the speech recognition method, Estimating a speech mask from the multichannel speech signal; Removing noise components using the estimated voice mask; Synthesizing speech using the multi-channel speech signal from which the noise component is removed; Estimating a noise model and a signal-to-noise ratio using the voice mask, and generating an acoustic model adapted to noise using the noise model and the signal-to-noise ratio; Extracting a speech feature from the synthesized speech signal; And Obtaining a speech recognition result using the acoustic model adapted to the speech feature and noise. The method of claim 6, wherein the voice mask estimating step, Separating a voice signal received from the outside into several frequency bands using gamma-tone filtering; Estimating time difference between microphone channels from the separated signal; Estimating a level difference between microphone channels from the separated signal; And And obtaining a voice mask using the time difference between the microphone channels and the level difference between the microphone channels. The method of claim 6, wherein generating the acoustic model Obtaining a noise mask from the estimated speech mask; Estimating a noise model using the noise mask and a gamma-tone filtered signal; Estimating a signal-to-noise ratio using the speech mask and the noise mask; And Obtaining an acoustic model adapted to noise using the estimated noise model and the estimated signal-to-noise ratio. The method of claim 8, wherein estimating the noise model Obtaining a noise mask using the voice mask; Removing speech components by multiplying the noise mask with the gamma-tone filtered signal; Synthesizing a noise signal from the signal from which the speech component has been removed; Extracting features of the noise signal; And And obtaining a noise model using the features of the noise signal. 10. The method of claim 9, wherein estimating the signal-to-noise ratio Obtaining an average value on a frequency channel of the voice mask; Obtaining an average value on a frequency channel of the noise mask; Detecting a noise frame using the average value of the voice mask; And Obtaining a mean value of speech mask on frequency and a mean value of noise mask on frequency for the noise frame, and obtaining a signal-to-noise ratio as a ratio between these mean values.

Images