KR100842754B1 - Method and Apparatus for Speech Recognition using reliability of articulatory feature - Google Patents

Abstract

The present invention comprises the steps of: extracting a speech feature by receiving a speech signal; extracting at least one articulation feature corresponding to the extracted speech feature; calculating articulation feature scores for each phoneme in accordance with the extracted articulation feature; And calculating an observation score according to the per-articulation feature score and the observed probability value of the extracted voice feature, wherein the per-phoneme feature feature searches for at least one phoneme corresponding to the at least one articulation feature, And a weighted sum of the reliability of the articulation features corresponding to each one of the phonemes.

Speech Recognition, Articulation Features, Reliability

Description

Method and apparatus for speech recognition using reliability of articulation features {Method and Apparatus for Speech Recognition using reliability of articulatory feature}

1 is a flow chart schematically showing a speech recognition method using the reliability of the articulation feature according to an embodiment of the present invention.

2 is a diagram illustrating a method for extracting a high reliability articulation feature using a speech feature according to an embodiment of the present invention.

3 is a flowchart illustrating a method of calculating an observation score according to an exemplary embodiment of the present invention.

4 is a view for explaining a method for calculating the articulation characteristic score for each phoneme according to an embodiment of the present invention.

5 is a flow chart schematically illustrating a method for recognizing speech further using additional speech features according to a preferred embodiment of the present invention.

<Explanation of symbols for main parts of the drawings>

401: articulation feature selection unit

403 Phoneme

405 and 407: How to add phoneme articulation feature score

The present invention relates to a method and apparatus for speech recognition using reliability of articulation features.

Researches in the field of speech recognition have been reported to improve the recognition performance by building a recognition model based on the speech utterance process. Among them, the mainstream method is to create a classifier or detector based on Articulatory Feature and obtain a voice recognition result directly from the input, or input the result value of another recognition module from the detector. The speech recognition method was used as a value. There are various ways to classify or create a classifier or detector based on articulation features, and to use the results from the detector.

As a method of using the result values from the speech recognition method, a method of performing speech recognition using only these result values, a weighted sum of likelihoods of the existing HMM recognition module, and a method of performing other recognition modules The method used as an input feature vector was mainly used.

However, the existing studies commonly used the results from detectors based on these articulation features in the recognition process regardless of the reliability. Therefore, there was always a possibility that a low reliability result adversely affects the overall recognition performance.

And since the results from all the detectors are applied to all phoneme models in common, the speech features that apply only to specific detectors cannot be used.

An object of the present invention is to improve the speech recognition performance by selecting only the high reliability results among the results from additional detectors based on articulation features or other feature vectors, and reflecting the selected high reliability results in the existing recognition process. It is.

In addition, the present invention has another object to improve the speech recognition performance by applying a new method that reflects the result from the additional detector to the existing recognition process.

Another object of the present invention is to improve the speech recognition performance by allowing the result value limited to a specific phone to be reflected in the main recognition process.

In order to achieve the above objects, according to an aspect of the invention, the step of receiving a speech signal to extract a speech feature, extracting at least one articulation feature corresponding to the extracted speech feature, the extracted articulation Calculating an articulation feature score for each phoneme in correspondence with a feature; and calculating an observation score according to the articulation feature score for each phoneme and an observation probability value of the extracted speech feature, wherein the articulation feature score for each phoneme is The speech recognition method may be provided by searching for at least one phoneme corresponding to the at least one articulation feature, and calculating the weighted reliability of the articulation feature corresponding to each of the at least one found phoneme.

In a preferred embodiment, the phoneme-specific articulation feature score retrieves at least one phoneme corresponding to the at least one articulation feature, and for a phoneme corresponding to all of the at least one articulation feature of the at least one searched phoneme. It may be characterized by weighting and calculating articulation characteristic reliability.

및

중 어느 한 수식에 의하되, 상기

는 t 번째 프레임의 임의의 관측 확률 값들의 차이의 평균이며, i는 음소 인덱스이고, t는 프레임 인덱스이고, j는 조음 특징의 인덱스이고,

및

는 실험에 의해 미리 결정된 임의의 상수이고, L은 검출된 조음 특징의 개수이고,

및

는 신뢰도가 반영된 조음 특징 점수인 것을 특징으로 할 수 있다. 또한, 상기 관측 확률 값들의 차이의 평균은

의 수식에 의하되, 상기 t는 프레임 인덱스이 고 상기 b_{j ,t}는 프레임 t에서 구한 관측 확률 값 중에서 j 번째로 큰 관측 확률 값을 의미하며, M은 실험에 의해 미리 결정된 임의의 상수인 것을 특징으로 할 수 있다. In addition, the observation score is

And

According to any one of the above formulas,

Is the mean of the differences between any observed probability values of the t th frame, i is the phoneme index, t is the frame index, j is the index of the articulation feature,

And

Is any constant predetermined by the experiment, L is the number of articulation features detected,

And

May be an articulation feature score reflecting reliability. In addition, the average of the difference between the observed probability values is

According to the formula, wherein t is the frame index, and b _{j , t} is the jth largest observation probability value of the observed probability value obtained in the frame t, M is characterized in that an arbitrary constant predetermined by the experiment can do.

Referring to another aspect of the present invention, a voice feature extractor for receiving a voice signal to extract a voice feature, an articulation feature extractor for extracting at least one articulation feature corresponding to the extracted voice feature, and the extracted articulation And a search unit for calculating the articulation feature score for each phoneme in accordance with the feature and calculating the observation score according to the articulation feature score for each phoneme and the observed probability value of the extracted speech feature, wherein the articulation feature score for each phoneme is at least the A speech recognition apparatus may be provided, wherein at least one phoneme corresponding to one articulation feature is searched and the reliability of the articulation feature corresponding to each of the at least one searched phoneme is added and calculated.

Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart schematically illustrating a speech recognition method using reliability of articulation features according to an exemplary embodiment of the present invention.

Referring to FIG. 1, first, a voice signal is received from the outside (step 101). Then, the speech feature of the received speech signal is extracted (step 103). The extraction of speech features may be extracted using algorithms such as Mel-Frequency Cepstral Coefficient (MFCC), Linear Predictive Coding (LPC), Perceptual linear prediction (PLP), and the like.

Thereafter, the articulation features of the received speech signal may be extracted (step 105). There may be various types of articulation features, and after extracting each articulation feature, the articulation features may be combined to determine a reliable articulation feature. In general, to detect N articulation features, N articulation feature detectors should be used. Commonly used articulation feature detectors include Gaussian Mixture Models (GMMs), artificial neural networks, and super vector machines (SVMs).

When the articulation feature is extracted in this way, the reliability of the extracted articulation feature is then obtained (step 107), which will be described in detail with reference to FIGS. 2 and 4. When the high reliability articulation feature is obtained by obtaining articulation reliability (step 109), the word string is searched using the speech feature and the high reliability articulation feature extracted in step 103 (step 111), and the resulting speech recognition is performed (step 111). Step 113).

2 is a diagram illustrating a method of extracting a high reliability articulation feature by using a voice feature according to an exemplary embodiment of the present invention.

Referring to Fig. 2, first, a voice feature is extracted from a received voice signal (step 201). When the speech feature is extracted in this way, N articulation feature detectors are used to detect N articulation features corresponding to the speech feature (step 203).

Here, the articulation characteristics are distinctive features of the phoneme. It can be divided into purity and long sound. At this point, there will be 15 articulation features, and therefore 15 articulation feature detectors.

On the other hand, the articulation characteristics may be classified into voicey, articulation methods, articulation positions, and roundness in addition to the distinctive features of the phoneme.

Here, voiced is to distinguish whether the voice is voiced, unvoiced or silent, and the articulation method is to distinguish whether the sound is vowel, rupture, friction, non-noise, oily or approach sound, and the position of articulation is positive sound, tooth sound or oral sound, or research sound It is to distinguish whether it is high vowel, low vowel, legend vowel, hustle vowel or mute.

The method of extracting articulation features based on the division of articulation features is also used in another embodiment of the present invention to be described with reference to FIG. 5.

The reliability of the articulation feature value detected by the articulation feature detector is obtained. In the present invention, the articulation feature value is converted into a value between 0 and 1, and the converted value is used as the reliability (step 205). When the reliability of the detected articulation feature is equal to or higher than the value previously determined by the experiment, it is regarded as having high reliability and extracted as a high reliability articulation feature (step 207).

3 is a flowchart illustrating a method of calculating observation scores according to an exemplary embodiment of the present invention.

Referring to FIG. 3, first, a high reliability modulation feature is extracted by the method described in FIG. 2 (step 301). Thereafter, the articulation feature score for each phoneme is calculated using the extracted articulation feature (step 303). A method of calculating the articulation feature score for each phoneme will be described in detail with reference to FIG. 4.

Then, the observation score is calculated (step 307). The calculation of the observation score is obtained by using the observation probability value 305 of the speech feature extracted from the input speech and the articulation characteristic score for each phoneme obtained in step 303.

The observation scores are first sorted in large order by extracting the largest M observation probability values among the observation probability values of the current frame t.

, (j=1,2,…,M)라 하고, 이렇게 정렬된 관측 확률 값들의 차이의 평균을 This sort of observation probability

, (j = 1,2,…, M), and the average of the differences of the observed probability values

[Equation 1]

Obtain as

Using the average of the difference of observed probability values

You can calculate the observation score,

Observation score

[Formula 2]

혹은

or

Calculate as

및

는 실험에 의해 미리 결정된 임의의 상수이다. L은 검출된 신뢰도 높은 조음특징의 개수이며

및

는 신뢰도가 반영된 조음특징 점수이다. Where i is the phoneme index and t is the frame index. j is the index of the articulation feature

And

Is any constant predetermined by the experiment. L is the number of reliable modulation features detected

And

Is an articulation score that reflects reliability.

혹은

가 0이다. Especially for phonemes that do not correspond to the detected high-reliability articulation features

or

Is 0.

Thereafter, a Viterbi search is performed using the obtained observation score (step 309).

4 is a diagram illustrating a method for calculating articulation feature scores for each phoneme according to an exemplary embodiment of the present invention.

Referring to FIG. 4, a method of applying a weight for each phoneme may be divided into two types according to a method of extracting articulation features.

First, the case where the distinctive features are the articulation features will be described mainly. As shown by reference numeral 401, articulation feature detectors exist according to the articulation features. At reference numeral 401, there are three articulation feature detectors according to articulation features A, B and C.

Each articulation feature has its own phonemes 403. In articulation feature A, there are three phonemes a1, a2, and a3. Articulation feature C includes the phonemes a1, b1, and c1.

In phonemes according to these articulation features

Phoneme a1 is included in both articulation features A, B, and C, phoneme b1 is included in articulation features B, C, and phoneme c1 is included only in articulation feature C. In addition, the phonemes a2 and a3 are included only in the articulation feature A, and the phonemes b2 are included in the articulation feature B only. Here, the phoneme a1 is included in the articulation feature A has two meanings. First, the phoneme a1 has the property of articulation feature A. Secondly, the result value from the detector of articulation feature A was more than a predetermined value in the experiment, and articulation feature A was detected with high reliability.

In this case, when the distinctive feature is an articulation feature, as shown by reference numeral 405, the reliability score of each articulation feature corresponding to each phoneme is calculated by weighting. Therefore, in the case of phoneme a1, the reliability scores of articulation features A, B, and C are all weighted, and in the case of phoneme b1, only the reliability scores of articulation features B and C are weighted. The remaining phonemes only weight the articulation feature reliability scores corresponding to each phoneme.

,

,

값들은 모두 각 조음 특징 검출기 별로 미리 결정된 가중치를 나타내며, AF_{a1 ~ c1} 값들은 모두 조음 특징 점수를 나타낸다. here

,

,

The values all represent a predetermined weight for each articulation feature detector, and the AF _{a1 to c1} values all represent articulation feature scores.

On the other hand, another method of calculating the articulation feature score is to find only the phonemes a1 having all articulation features and weight the reliability scores of articulation features A, B, and C to these a1 phones, as shown at 407. To calculate the score.

The weight at the time of weighting is any constant determined in advance by experiment.

5 is a flowchart schematically illustrating a method of recognizing a voice by further using additional voice features according to an exemplary embodiment of the present invention.

Referring to FIG. 5, first, a voice signal is received from the outside (step 501). Thereafter, voice features of the received voice signal are extracted (step 503). The extraction of speech features may be extracted using algorithms such as Mel-Frequency Cepstral Coefficient (MFCC), Linear Predictive Coding (LPC), Perceptual linear prediction (PLP), and the like.

In addition, in the extraction of the voice feature, an additional voice feature may be extracted in addition to the basic voice feature extraction (step 503) (step 505). The additional voice feature may include an FO frequency, a formant frequency, an energy value and a frequency in the formant. Band energy and the like can be used. These additional voice features can be selected to suit the characteristics of each articulation detector. When the voice feature is extracted, the articulation feature is extracted as described above with reference to FIG. 1 (step 506).

The received voice signal voice features are then used to extract articulation features and search for reliability (step 507). Such articulation feature extraction and reliability searching methods have been described with reference to FIGS. 2 and 4.

Then, if the high reliability articulation feature is extracted by searching for articulation reliability (step 509), the word string is searched using the speech feature and the high reliability articulation feature extracted in step 503 (step 511), and the resultant speech Recognition is made (step 513).

This method is similar to the method described in FIG. 1, but additional speech features can be used, so that the speech features can be extracted without actually extracting the speech features of all the phonemes in order to be combined with the recognition process of the existing dynamic programming method. There is an advantage in that a classification scheme limited to phonemes or a specific vector can be used.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

According to the present invention, the speech recognition performance can be improved by selecting only a reliable result value among the results from additional detectors based on articulation features or other feature vectors, and reflecting the selected high confidence result value in the existing recognition process. have.

In addition, the present invention can improve the speech recognition performance by applying a new method that reflects the result from the additional detector to the existing recognition process.

In addition, the present invention can improve the speech recognition performance by allowing the result value limited to a specific phone to be reflected in the main recognition process.

Claims (6)

Receiving a speech signal to extract a speech feature; Extracting at least one articulation feature corresponding to the extracted speech feature; Calculating articulation feature scores for each phoneme in correspondence with the extracted articulation features; and Comprising a step of calculating the observation score corresponding to the phoneme characteristic score of each phoneme and the observed probability value of the extracted speech feature, The articulation feature score for each phoneme is calculated by searching for at least one phoneme corresponding to the at least one articulation feature and weighting the reliability of the articulation feature corresponding to each of the found at least one phoneme. Speech recognition method characterized in that. delete The method of claim 1, The articulation feature score for each phoneme searches for at least one phoneme corresponding to the at least one articulation feature, and weights articulation feature reliability for a phoneme corresponding to all of the at least one articulation feature of the at least one found phoneme. Counting Speech recognition method characterized in that. The method of claim 1, The observation score is

And

By any of the formulas, remind

Is the mean of the differences between any observed probability values of the t th frame, i is the phoneme index, t is the frame index, j is the index of the articulation feature,

And

Is any constant predetermined by the experiment, L is the number of articulation features detected,

And

Is an articulation feature score that reflects reliability Speech recognition method characterized in that. The method of claim 1, The mean of the differences between the observed probability values is

According to the formula, wherein t is the frame index and b _{j , t} is the jth largest observation probability value from the observation probability value obtained in the frame t, M is an arbitrary constant predetermined by the experiment Speech recognition method characterized in that. A voice feature extractor configured to receive a voice signal and extract a voice feature; An articulation feature extraction unit for extracting at least one articulation feature corresponding to the extracted voice feature; Comprising a search section for calculating the articulation feature score for each phoneme in accordance with the extracted articulation feature and calculates the observation score corresponding to the articulation feature score for each phoneme and the observed probability value of the extracted speech feature, The articulation feature score for each phoneme is calculated by searching for at least one phoneme corresponding to the at least one articulation feature and weighting the reliability of the articulation feature corresponding to each of the found at least one phoneme. Speech recognition device characterized in that.

Images