KR100897555B1 - Apparatus and method of extracting speech feature vectors and speech recognition system and method employing the same - Google Patents

Abstract

An apparatus and method for extracting a speech feature vector and a speech recognition system and method employing the same are provided. The speech feature vector extracting apparatus includes: an FFT processor converting a speech signal configured in units of frames into a signal in a frequency domain; A formant emphasis unit that emphasizes the formant by suppressing the pitch harmonic components included in each frequency component with respect to the signal in the frequency domain provided from the FFT processor; And a filter bank processor configured to perform bandpass filtering on a signal in a frequency domain including each frequency component in which the formant is emphasized, using a plurality of mel scale filter banks, and the formant emphasis unit includes a magnitude of each frequency component. A harmonic removing unit for subtracting the magnitude of the neighboring lower frequency component and removing the pitch harmonic component by taking an absolute value of the subtracted result; And a smoothing portion for smoothing each frequency component whose pitch harmonic component is suppressed using a local center of gravity.

Description

Apparatus and method of extracting speech feature vectors and speech recognition system and method employing the same}

1 is a block diagram showing the configuration of a speech recognition system employing the present invention;

2 is a block diagram showing the configuration of an embodiment of a speech feature vector extraction apparatus according to the present invention;

3 is a block diagram showing the detailed configuration of the formant emphasis unit shown in FIG.

4a and 4b show the spectrum of a vowel in order to compare the performance between the invention and the prior art,

5A and 5B show a spectrogram in one sentence for comparing the performance between the present invention and the prior art, and

6A and 6B show spectrograms of filterbanks in order to compare the performance between the present invention and the prior art.

The present invention relates to speech recognition, and more particularly, to an apparatus and method for extracting a feature vector necessary for speech recognition more accurately by suppressing pitch harmonic components in order to emphasize a formant, and a speech recognition system employing the same. It is about a method.

At present, voice recognition technology is gradually expanding the application range from personal portable terminals to information appliances, computers, and large-scale telephony servers, but attempts to increase the voice recognition performance itself to improve the instability of recognition performance that varies depending on the surrounding environment. Various studies have been conducted on attempts to prevent recognition rate degradation in noise and noise environments.

Among these, in order to prevent the recognition rate from deteriorating in the noise environment, the existing mel-frequency cepstral coefficient (hereinafter referred to as MFCC) in the speech feature vector extraction process, which is the first step of speech recognition technology, is used. Various techniques for linearly or nonlinearly transforming feature vectors in consideration of temporal characteristics have been studied.

First, conventional transform algorithms considering temporal characteristics of feature vectors include cepstral mean subtraction, mean-variance normalization, on real-time mean-variance normalization of speech recognition features, P. Pujol, D. Macho and C. Nadeu, ICASSP, 2006, pp. 773-776, RelAtive SpecTrAl algorithm, Data-driven RASTA filters in reverberation, ML Shire et al, ICASSP, 2000, pp. 1627 1630), histogram normalization, quantile based histogram equalization for noise robust large vocabulary speech recognition, F. Hilger and H. Ney, IEEE Trans.Audio, Speech, Language Processing, vol. 14, no. 3, pp. 845-854), augmenting delta feature, On the use of high order derivatives for high performance alphabet recognition, J. di Martino, ICASSP, 2002, pp. 953-956).

In addition, techniques for linearly transforming feature vectors include linear discriminant analysis (LDA) and principal component analysis, optimization of temporal filters for constructing robust features in speech recognition, Jeih-Weih Hung et. Al, IEEE Trans.Audio. , Speech, and Language Processing, vol. 14, No. 3, 2006, pp. 808-832).

In addition, as a method of using a non-linear neural network (TempoRAl Patterns, referred to as 'TRAP', Temporal patterns in ASR of noisy speech, H. Hermansky and S. Sharma, ICASSP, 1999, pp. 289-292 ), Automatic speech attribute transcription (ASAT), A study on knowledge source integration for candidate rescoring in automatic speech recognition, Jinyu Li, Yu Tsao and Chin-Hui Lee, ICASSP, 2005, pp. 837-840 ) And the like are known.

On the other hand, in relation to an attempt to increase the speech recognition performance itself, there is a limit to the performance improvement since the MFCC feature vectors are extracted from the spectrum including the pitch harmonics that are less related to speech recognition.

An object of the present invention is to provide an apparatus and method for more accurately extracting feature vectors required for speech recognition by suppressing pitch harmonic components to emphasize formants.

Another object of the present invention is to provide a speech recognition system and method employing the speech feature vector extraction apparatus and method described above.

In order to solve the above technical problem, the apparatus for extracting a speech feature vector according to the present invention comprises: an FFT processor for converting a speech signal configured in units of frames into a signal in a frequency domain; A formant emphasis unit that emphasizes the formant by suppressing the pitch harmonic components included in each frequency component with respect to the signal in the frequency domain provided from the FFT processor; And a filter bank processor configured to perform bandpass filtering on a signal in a frequency domain including each frequency component in which the formant is emphasized using a plurality of mel scale filter banks.

Here, the formant emphasis unit may include: a harmonic removing unit configured to subtract the magnitude of each frequency component and the magnitude of the neighboring lower frequency component, and remove the pitch harmonic component by taking an absolute value of the subtracted result; And a smoothing portion for smoothing each frequency component whose pitch harmonic component is suppressed using a local center of gravity.

In order to solve the above technical problem, the speech feature vector extraction method according to the present invention comprises the steps of: converting a speech signal composed of frame units into a signal in a frequency domain; Reinforcing a formant by suppressing a pitch harmonic component included in each frequency component with respect to the signal in the frequency domain; And performing bandpass filtering on a signal in a frequency domain including each frequency component in which the formant is emphasized using a plurality of mel scale filter banks.

In order to solve the above other technical problem, the speech recognition system according to the present invention obtains a spectrum emphasizing the formant by suppressing the pitch harmonic components included in each frequency component with respect to the signal in the frequency domain configured in units of frames. Feature extraction unit for extracting a feature vector for speech recognition using the spectrum is highlighted; And a recognition unit that performs a recognition process on the extracted feature vector with reference to a database.

In order to solve the above other technical problem, the speech recognition method according to the present invention suppresses pitch harmonic components included in each frequency component with respect to a signal in a frequency domain configured in units of frames, thereby obtaining a spectrum that emphasizes the formant, and the formant Extracting feature vectors for speech recognition using the highlighted spectrum; And performing a recognition process on the extracted feature vector with reference to a database.

The speech feature vector extraction method and speech recognition method may be embodied as a computer-readable recording medium that records a program for execution in a computer.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a speech recognition system to which the present invention is employed, and includes a noise removing unit 110, a feature vector extracting unit 130, a recognition unit 150, and a database 170.

Referring to FIG. 1, the noise removing unit 110 removes noise from an input voice signal. In order to remove noise from a voice signal, various known methods such as spectral subtraction may be applied.

The feature vector extractor 130 non-linearly converts the spectrum, thereby suppressing the pitch harmonic component to obtain a formant-enhanced spectrum, and extracting the MFCC feature vector from the obtained spectrum.

The recognition unit 150 calculates the similarity using the parameters stored in the learned database 170 with respect to the feature vector extracted by the feature vector extractor 130. The recognition unit 150 may use various speech recognition models such as Hidden Markov Model (HMM), Dynamic Time Warping (DTW), and neural network.

The database 170 previously learns and stores parameters of a model used by the recognizer 150. When the recognizer 150 uses the neural network model, the parameter stored in the database 170 is a weight value of each node learned by the BP (Back Propagation) algorithm, and when the recognizer 150 uses the HMM model, the parameter is stored in the database 170. The parameters stored at (170) are the state transition probabilities learned by the Baum-Welch reestimation algorithm and the probability distribution of each state.

Figure 2 is a block diagram showing the configuration of an embodiment of the speech feature vector extraction apparatus according to the present invention, a pre-processing unit 210, FFT (Fast Fourier Transform) processing unit 230, formant emphasis unit 250, filter bank A processing unit 270 and a DCT (Discrete Cosine Transform) processing unit 290 are included.

Referring to FIG. 2, the preprocessor 210 composes a frame having a length of 20 to 30 ms for every 10 msec for a voice signal, and performs a pre-emphasis process on a frame basis to perform high frequency. Emphasizes the components to enhance the consonants. The signal x (n) on which the pre-emphasis is performed may be represented by Equation 1 below.

x (n) = s (n)-α (n-1)

Here, s (n) is an audio signal, and α is usually 0.97 as a constant value used for pre-emphasis.

On the other hand, in order to prevent the frequency information from being distorted due to a sudden change in the boundary value between frames, the preprocessor 210 may express a window function in a frame unit signal in which a high frequency component is emphasized as shown in Equation 2 below. Apply the Hamming window function h (n)

h (n) = 0.6-0.4 sin (2πn / M)

Where M is the length of the Hamming window.

The FFT processor 230 converts the signal to which the window is applied to an N-point fast fourier transform (FFT) to convert the signal into a frequency domain signal. N-point FFT processing can be expressed as Equation 3 below.

Where f _s Is the sampling frequency and k = 0, 1, ..., N-1.

The formant emphasis unit 250 obtains the magnitude of each frequency component from the FFT processed signal provided from the FFT processor 230, subtracts the magnitude of the adjacent frequency component with respect to the magnitude of each frequency component, and takes the absolute value of the pitch. The harmonic component is removed and the formant is emphasized by locally smoothing the magnitude of each frequency component from which the pitch harmonic component is removed.

The filter bank processing unit 270 may narrow the low frequency region and wide the high frequency region with respect to the signal of the formant-enhanced frequency region provided by the formant enhancement unit 250 according to the human auditory characteristics. Bandpass filtering is performed using a plurality of divided filter banks. In other words, the spectrum of a specific frequency component in one frame is transformed into dimensional space through which the characteristics can be better represented through mel-scale filtering. Such mel-scale filtering may be expressed as Equation 4 below.

Where E [j] represents the output of filter bank j, J is the number of filter banks, and H _j (m) represents the transfer function of filter bank j.

The DCT processing unit 290 performs DCT processing on each filter bank signal provided from the filter bank processing unit 270 to extract the final MFCC feature vector. Currently, MFCC feature vectors, which are widely used in speech recognition technology, are represented by 12th order vectors in each frame. The m-th order MFCC feature vector C (m) output through the DCT process may be expressed by Equation 5 below.

Where J is the number of filter banks and j represents each filter bank .

3 is a block diagram illustrating a detailed configuration of the formant emphasis unit 250 illustrated in FIG. 2, and includes a size calculation unit 310, a harmonic removing unit 330, and a smoothing unit 350. Is done.

Referring to FIG. 3, the magnitude calculator 310 calculates the magnitude of each frequency component from the signal provided from the FFT processor 230. That is, since the signal provided from the FFT processor 230 is a complex number, the magnitude of each frequency component can be obtained by taking the magnitude and converting it to a real value.

The harmonic removing unit 330 suppresses the pitch harmonic component by subtracting the magnitude of each frequency component and the neighboring lower frequency component provided from the size calculating unit 310 and taking an absolute value of the subtracted result. This may be expressed as in Equation 6 below.

는 피치 하모닉 성분이 억제된 k 번째 주파수 성분을 나타낸다.here,

Denotes the kth frequency component at which the pitch harmonic component is suppressed.

The smoothing unit 350 smoothes each frequency component whose pitch harmonic component is suppressed using a local center of gravity. The smoothing process can be expressed as the following equations (7) and (8).

는 스무딩된 k 번째 주파수 성분을 나타내고, U는 국소적인 무게 중심을 구하는데 사용되는 주파수 성분의 수 즉, 윈도우의 길이를 나타내고,

는 전체 스펙트럼의 평균과 관련있는 파라미터이며, N은 FFT 포인트의 수, P는

가 전체 스펙트럼의 평균보다 큰 값이 되도록 조정하는 파라미터이다. here,

Denotes the smoothed k-th frequency component, U denotes the number of frequency components used to find the local center of gravity, i.e. the length of the window ,

Is the parameter associated with the mean of the entire spectrum, where N is the number of FFT points, P is

Is a parameter adjusted to be greater than the average of the entire spectrum.

4A and 4B show spectra of vowels in order to compare the performance between the present invention and the prior art. 4a shows a spectrum of a vowel according to the prior art, and FIG. 4b shows a spectrum of a vowel according to the invention, respectively. According to the prior art, it is difficult to distinguish the second formant and the third formant by the pitch harmonic component, but it can be seen that the present invention is clearly distinguished.

5a and 5b are diagrams showing a spectrogram of a sentence in order to compare the performance between the present invention and the prior art. 5a shows a spectrogram of a sentence according to the prior art, and FIG. 5b shows a spectrogram of a sentence according to the present invention, respectively. According to this, in the case of the present invention it can be seen that the trajectory of the formant can be accurately tracked. 6A and 6B are diagrams showing spectrograms of filter banks in order to compare performance between the present invention and the prior art. Similarly, in the present invention, the spectrogram is stable, and thus the trajectory of the formant is more clearly shown. It can be traced.

Next, in order to verify the effect of the present invention, a speech recognition experiment for English utterance was performed. Voice data is a TIMIT corpus provided by the Linguistic Data Consortium (LDC) in the United States. This database is a standard for measuring phoneme recognition performance due to the added level of phonemes. On the other hand, this database divides the entire United States into eight regions, collecting voices of people who speak the language of each region, and consists of about 6,000 speech sentences. In addition, the speech sentences were divided into learning sentences and test sentences to be used for speech recognition.

The algorithm used for phoneme recognition is HMM, which uses the KTK tool provided by the University of Cambridge. For the performance comparison, the MFCC feature vector of the prior art used a 39th order feature vector including a delta coefficient and a delta-delta coefficient based on the 13th order, and a 20th-order filter bank was used. One frame was 20 ms in length, 512-point FFT processing was performed, and the pre-emphasis constant was 0.97. Meanwhile, in the present invention, 0.97 is used as a pre-emphasis constant, a Hamming window of 20 msec, a 512-point FFT, and a 20th order filter bank, and U used in the smoothing unit 350 is 5 to determine the current frequency component. Smoothing was performed by comparing the values to the values corresponding to the surrounding four frequency components.

In addition, P is based on a value corresponding to four times the average of the entire spectrum by using 4, so that even if the size of the spectrum is too small, such as in the silent section, the small value fluctuations largely change to the local center of gravity smoothing. This prevents it from growing like a formant component.

Table 1 shows the phoneme recognition experiment results for the feature vector obtained by the present invention and the feature vector obtained by the prior art in the above experimental environment.

TABLE 1

Recognition rate accuracy # Hit # delete # how # Insert # Total word Prior art 73.74% 70.48% 47,303 5,562 11,280 2,094 64,145 The present invention 74.23% 71.17% 47,618 5,500 11,027 1,963 64,145

Looking at Table 1, as a result of using the feature vector according to the present invention, 74.23% is higher than the conventional phoneme recognition rate of 73.34%, it can be seen that the improved results including the insertion error.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, which are also implemented in the form of a carrier wave (for example, transmission over the Internet). It also includes. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. And functional programs, codes and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

As described above, according to the present invention, the speech spectrum obtained by FFT processing is suppressed from the pitch harmonic component to obtain a speech spectrum emphasizing the formant, and a more accurate feature vector is extracted from the speech spectrum and used for speech recognition. There is an advantage that can be improved.

Although the present invention has been described with reference to the above embodiments, it is merely illustrative, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. . Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (14)

An FFT processor for converting a voice signal configured in units of frames into a signal in a frequency domain; For the signal in the frequency domain provided from the FFT processor, the pitch harmonic component included in each frequency component is subtracted by subtracting the magnitude of each frequency component and the magnitude of the neighboring lower frequency component and taking the absolute value of the subtracted result. Formant emphasis to suppress the formant emphasis; And And a filter bank processor configured to perform bandpass filtering on a signal in a frequency domain including each frequency component of which the formant is emphasized using a plurality of mel scale filter banks. The apparatus of claim 1, wherein the formant emphasis unit smoothes each frequency component of which the pitch harmonic component is suppressed by using a local center of gravity. Converting a voice signal configured in units of frames into a signal in a frequency domain; For the signal in the frequency domain, by subtracting the magnitude of each frequency component and the magnitude of the neighboring lower frequency component and taking the absolute value of the subtracted result, the form is suppressed by suppressing the pitch harmonic component included in each frequency component. Emphasizing; And And performing bandpass filtering on a signal in a frequency domain including each frequency component of which the formant is emphasized using a plurality of mel scale filter banks. delete 4. The method of claim 3, further comprising smoothing each frequency component of which the pitch harmonic component is suppressed using a local center of gravity in the formant enhancement step. The method of claim 5, wherein the smoothing step is

(here,

Denotes the kth frequency component at which the pitch harmonic component is suppressed,

Denotes the smoothed k-th frequency component, U denotes the number of frequency components used to find the local center of gravity,

Is the parameter associated with the mean of the entire spectrum, where N is the number of FFT points, P is

Is a parameter that is larger than the average of the entire spectrum) Speech feature vector extraction method characterized in that performed by. By subtracting the magnitude of each frequency component and the neighboring lower frequency component with respect to the signal in the frequency domain configured in units of frames, and taking the absolute value of the subtracted result, the pitch harmonic component included in each frequency component is suppressed. A feature extractor for obtaining a formant-enhanced spectrum and extracting a feature vector for speech recognition using the formant-enhanced spectrum; And And a recognition unit performing a recognition process on the extracted feature vector with reference to a database. delete The speech recognition system of claim 7, wherein the feature extractor smooths each frequency component of which the pitch harmonic component is suppressed using a local center of gravity. By subtracting the magnitude of each frequency component and the neighboring lower frequency component with respect to the signal in the frequency domain configured in units of frames, and taking the absolute value of the subtracted result, the pitch harmonic component included in each frequency component is suppressed. Obtaining a formant-enhanced spectrum and extracting feature vectors for speech recognition using the formant-enhanced spectrum; And Speech recognition method comprising the step of performing a recognition process for the extracted feature vector with reference to a database. The speech recognition method of claim 10, wherein the feature extraction step smoothes each frequency component whose pitch harmonic component is suppressed using a local center of gravity. The method of claim 11, wherein the smoothing step is

(here,

Denotes the kth frequency component at which the pitch harmonic component is suppressed,

Denotes the smoothed k-th frequency component, U denotes the number of frequency components used to find the local center of gravity,

Is the parameter associated with the mean of the entire spectrum, where N is the number of FFT points, P is

Is a parameter that is larger than the average of the entire spectrum) Speech recognition method, characterized in that carried out by. A computer-readable recording medium describing a program capable of executing the speech feature vector extraction method according to any one of claims 3, 5 and 6. A computer-readable recording medium describing a program capable of executing the voice recognition method according to any one of claims 10 to 12.

Images