KR101066472B1 - Apparatus and method speech recognition based initial sound - Google Patents

Abstract

The present invention relates to a speech-based voice recognition device and a voice recognition method, comprising: a voice input unit for receiving a voice, a feature analyzer for generating an input pattern by calculating a feature vector of a received voice signal, and a reference pattern in which reference patterns are stored in advance A consonant including a storage unit, a pattern recognition unit outputting consonant information corresponding to a reference pattern similar to the input pattern by comparing the input pattern with a reference pattern, and a decision unit determining a word to be output by searching for words corresponding to the consonant information In the speech recognition apparatus, the reference pattern is a pattern generated based on syllables in which the same vowel is combined with each consonant; The pattern recognition unit detects a similar reference pattern by performing a syllable unit comparison algorithm on the input pattern, but performs a phoneme unit comparison algorithm only when two or more similar reference patterns are detected. do. According to the present invention, a speech recognition technology suitable for an embedded system enables simple and accurate initial-based speech recognition, and outputs a result according to the importance of words retrieved from the initial-based speech recognition, thereby improving search function. There is an advantage that it can provide.

Dynamic Time Warping (DTW), Hiden Markov Model (HMM), Speech Recognition

Description

Speech based speech recognition device and voice recognition method {APPARATUS AND METHOD SPEECH RECOGNITION BASED INITIAL SOUND}

The present invention relates to a speech-based speech recognition device and a speech recognition method. More specifically, the speech recognition is performed by using syllables combining the same vowels with consonants in a consonant search through speech recognition. At the same time, the present invention provides a speech recognition apparatus and a speech recognition method capable of improving the search function by outputting a word of high importance in the initial search.

The need for in-vehicle devices by voice recognition technology is emerging for the driver's convenience and driving stability. In particular, voice recognition of a vehicle navigation device (Navigator) is in the practical stage of use.

However, the speech recognition performance of commercialized navigation system is not satisfactory. This is due to the in-vehicle noise and poor conditions of embedded systems. In a noisy environment, speech recognition systems must implement noise processing, speech signal feature modeling, speech recognition algorithms, etc., whereas embedded systems such as speech recognition systems in navigation devices need to be implemented in small databases. do.

The noise processing techniques include speech enhancement and speech enhancement techniques. Spectral Subtraction (SS) based on statistical model and Wiener filter technique using Minimum Mean Square Error (MMSE) estimation. In addition, methods such as speech enhancement considering human hearing characteristics include signal subspace speech enhancement and masking effects. Among the statistical model techniques, SS is simple but has a weak point in musical noise. In the technique using Wiener filter, speech distortion and residual noise are also generated. Among the techniques using the human auditory characteristics, the signal subspace speech enhancement technique and the masking effect technique have good performance but have a large amount of computational disadvantages. Such voice enhancement technique is a technique to control the noise around me, and it is difficult to apply to embedded system because it consumes a lot of power.

Accordingly, the present invention has been made to solve the above-mentioned conventional problems, and an object of the present invention is to provide a speech recognition device and a speech recognition method suitable for an embedded system.

Another object of the present invention is to provide a simple and accurate voice-based speech recognition device and a speech recognition method.

It is still another object of the present invention to provide a speech recognition apparatus and a speech recognition method having improved search performance by outputting a result according to the importance of a searched word in initial speech recognition.

According to a feature of the present invention for achieving the above object, the present invention comprises: a voice input unit for receiving a voice and converting it into a voice signal; A feature analyzer for generating an input pattern by calculating a feature vector of the voice signal received from the voice input unit; A reference pattern storage unit for storing reference patterns in advance in order to perform speech recognition in comparison with the input pattern; A pattern recognition unit for comparing the input pattern with the reference pattern and outputting consonant information corresponding to a reference pattern similar to the input pattern; And a decision unit determining a word to be output by searching for words corresponding to the initial information detected by the pattern recognition unit, wherein the reference patterns stored in the reference pattern storage unit are respectively included in the reference pattern storage unit. A pattern generated on the basis of syllables in which the same vowel is combined with the consonants of; The pattern recognition unit first performs a syllable unit comparison algorithm on the input pattern to detect similar reference patterns, and selectively performs a phoneme unit comparison algorithm only when two or more similar reference patterns are detected. Detecting one pattern, the determining unit is configured to include a database in which a plurality of words are stored, and searches for and outputs a word corresponding to initial information detected by the pattern recognition unit among words stored in the database, wherein the pattern recognition is performed. If more than one word corresponding to the detected first consonant information is searched, the importance of all the searched words is calculated.

Wherein the syllable unit comparison algorithm is a dynamic time warping (DTW) method; The phoneme comparison algorithm is a Hidden Markov model method; In the reference pattern storage unit, a reference pattern for the syllable unit comparison algorithm and a reference pattern for the phoneme unit comparison algorithm may be separately stored.

In addition, the pattern recognition unit calculates a Euclidean square distance between the input pattern and each reference pattern by performing the syllable unit comparison algorithm, and the calculated Euclidean square distance is less than a previously stored threshold distance. In the case of two or more, the phoneme comparison algorithm may be performed.

delete

delete

In this case, the determination unit may detect a frequency of simple nouns included in each of the words searched in the range of the searched words, and calculate the importance of each word based on the detected frequency of the simple nouns.

In addition, the determination unit may detect whether each searched word includes a simple noun with a predetermined weight, and calculate the importance of each word based on the frequency and the weight.

Here, the determination unit may output the calculated word of high importance or output only the word of highest importance.

On the other hand, the present invention, (A) receiving the voice and converting the voice signal; (B) generating an input pattern by calculating a feature vector of the voice signal; (C) detecting a similar reference pattern by comparing the input pattern with previously stored reference patterns according to a syllable unit comparison algorithm; (D) selectively detecting one similar reference pattern according to a phoneme unit comparison algorithm only when two or more similar reference patterns are detected in step (C); (E) searching for and outputting a word having a consonant corresponding to one similar reference pattern detected in step (C) or step (D),
At this time, the step (E), (E1) is a step of searching for all the words having the initial property corresponding to the detected one similar reference pattern; (E2) detecting a frequency of simple nouns included in each word searched in the range of words searched in step (E1); (E3) calculating the importance of each word based on the frequency of the simple noun detected in the step (E2); And (E4) outputting the searched word according to the importance calculated in the step (E2).

In this case, the reference pattern may be previously generated and stored separately for the reference pattern for the syllable unit comparison algorithm and the reference pattern for the phoneme unit comparison algorithm based on the syllables in which the same vowels are combined with each consonant. have.

In addition, the syllable unit comparison algorithm is a dynamic time warping (DTW) method; The phoneme comparison algorithm may be a hidden Markov model method.

delete

The step (E3) may be performed by detecting whether each searched word includes a simple noun with a predetermined weight, and calculating the importance of each word based on the frequency and the weight.

Here, the weighted simple noun may be a simple noun representing a public institution.

As described in detail above, according to the voice recognition device and the voice recognition method according to the present invention, the following effects can be expected.

In other words, by adopting a voice recognition technology suitable for the embedded system has the advantage that it is possible to simple and accurate voice-based speech recognition.

In addition, according to the present invention there is an advantage that the improved search function can be provided by outputting the result according to the importance of the searched words in the initial speech recognition.

Hereinafter, with reference to the accompanying drawings, a specific embodiment of the voice recognition device and voice recognition method according to the present invention as described above will be described in detail.

1 is a block diagram showing a schematic configuration of a general speech recognition system. The general voice recognition system includes a voice input unit 10 that first receives a user's voice into a microphone and converts the voice into a signal form.

In addition, a feature analysis unit 11 is provided to analyze a voice signal obtained by the voice input unit 10 to extract a feature vector required for recognition, and generate an input pattern to be compared with a reference pattern using the same.

The input pattern by the feature analyzer 11 is input to the pattern recognition unit 12 again. The pattern recognition unit 12 performs a comparison algorithm with respect to the input pattern and the reference pattern previously stored in the reference pattern storage unit 13 to be described later. The pattern recognition unit 12 outputs information on a reference pattern having the highest similarity as a result of the comparison between the input pattern and the reference pattern as a voice recognition result.

Here, the reference pattern storage unit 13 is a storage means for storing predetermined reference patterns for speech recognition.

On the other hand, when the pattern recognition unit 12 outputs information on a reference pattern finally determined to have the highest similarity, the determination unit 14 receives the word and searches for a word using a letter corresponding to the similar reference pattern finally determined. do. The voice recognition operation is terminated by finally outputting the retrieved word.

In such a voice recognition system, the more the number of reference patterns stored in the reference pattern storage unit 13 and compared with the input pattern, the more accurate voice recognition is possible. In this case, the capacity of the reference pattern storage unit 13 should be enormous. In addition, the performance of the pattern recognition unit 12 should also be high because the amount of calculation increases significantly when the comparison algorithm between the input pattern and the reference pattern is performed. However, as described above, since system resources are limited in an embedded system, it is required to derive an accurate voice recognition result with a minimum of resources. For this purpose, the present invention supports voice based speech recognition.

In particular, when entering the first consonants 'a', 'b', 'c', etc., do not use the names of the consonants, such as 'breath,' 'ni', 'di', and 'ga', 'me', A single vowel is combined to be pronounced and inputted as 'da', and the reference pattern is also generated and stored in response to the voice signal of the combined form.

As a result, not only the number of reference patterns to be stored in the reference pattern storage unit 13 is greatly reduced, but also the amount of calculation in the pattern recognition unit 12 is significantly reduced.

The speech recognition method according to the present invention will be described in more detail with reference to FIG. 2. 2 is a flowchart sequentially illustrating a method of speech-based speech recognition according to an embodiment of the present invention.

In the voice recognition method according to an embodiment of the present invention as shown in FIG. 2, first, a voice is input from a user (S10). The voice input from the user in step S10 may be any one, but as described above, in the embodiment of the present invention, initial-based speech recognition is performed, and based on a combination of a single vowel common to the first consonants. Since speech recognition is performed using the generated reference pattern, it will be input as a combination of syllables that have a common vowel such as 'ga', 'me', and 'da' and does not include the finality. For example, if the user wants to search for 'Hong Gil-dong', a voice such as 'Hagada' will be input.

When a voice is input from the user in this way, the input voice is converted into a voice signal, and a feature vector is extracted therefrom (S20). Here, the extraction of the feature vector may be performed by performing linear predictive coding (LPC). When the feature vector is extracted in step S20, an input pattern for the input voice signal is generated using the feature vector. The input pattern is data processed to perform a comparison algorithm with the pre-stored reference pattern.

Subsequently, in step S30, a comparison and analysis between the input pattern and the reference pattern are performed. Here, the input pattern and the reference pattern to be compared and analyzed are all generated in syllable units and compared with each other. In particular, the step S30 may use a dynamic time warping (DTW) method. The dynamic time warping is a method of recognizing a voice input with a voice having the most similar reference pattern by non-linearly matching an input pattern and a reference pattern to compensate for a difference in pronunciation speed and length between the input voice and the reference voice. to be. The criterion for determining similarity is to calculate a Euclidean distance between the input pattern and each reference pattern, and recognize the reference pattern having the smallest distance as the pattern most similar to the input pattern.

As described above, in calculating the similarity between the input pattern and each reference pattern by performing dynamic time warping in step S30, determining whether the reference pattern having similarity with the input pattern within a predetermined range is 2 or more ( S40) is performed together. That is, during the dynamic time warping, as a result of calculating the Euclidean square distance between the input pattern and each reference pattern, it is determined whether two or more reference patterns having Euclidean square distance smaller than a preset threshold value are present.

Since the voice currently input is likely to be recognized as two or more similar voices, more accurate pattern analysis is required. For example, 'ga' and 'ka', 'da' and 'ta' are similar to the signal pattern due to the similarity of pronunciation. There is a possibility.

Therefore, it is determined whether the similar reference pattern is 2 or more, and if it is 2 or more, pattern analysis of a method having better recognition rate is performed again than the dynamic time warping method (S50). That is, the speech signal is divided into phoneme units, and the pattern comparison algorithm of the phoneme units is performed by the same method as the Hidden Markov model.

Here, the hidden Markov model is a statistical model for determining hidden parameters from observed parameters based on the assumption that the modeling system is a Markov process having unknown parameters, and is widely used in the field of speech recognition. One of the ways.

In the step S60 determines the phoneme according to the pattern analysis results performed in the step S30 and S50. That is, when the pattern comparison algorithm of the phoneme unit is performed through step S50, the phoneme determined to be the most similar is determined as the input phoneme, or when only one similar reference pattern is detected in step S40, the corresponding phoneme corresponds to the corresponding reference pattern. The phoneme corresponding to the voice is determined as the input phoneme.

For example, when the user inputs the voice 'ga' and recognizes that the reference patterns corresponding to 'ga' and 'car' are similar patterns in step S30, only the phoneme part of the voice signal stored again in step S50 is repeated. By separately processing and performing the hidden Markov model, the user decides the initial consonant 'a' to be recognized as a phoneme. On the other hand, if the user inputs the voice 'I' and the similar reference pattern is recognized as 'I' in step S30, it is determined that 'b' is immediately input without going through step S50.

When phonemes are determined in this way with respect to the sequentially input voice signals, a step of searching for words using the phonemes and selecting a final result among the searched words is performed (S70). More specific execution of step S70 will be described later with reference to FIG. 4.

In operation S80, a result of the finally recognized result is displayed, such as displaying the searched result on the display device.

According to this method, first, the number of reference patterns to be compared can be reduced by initial speech-based speech recognition, thereby saving memory and reducing computation amount. In addition, by using dynamic time warping for syllable unit patterns, which do not require much computation, the subsidiary use of the Hidden Markov Model method for phoneme unit patterns only when accuracy is required, without overloading the system It has the advantage of ensuring the accuracy of the.

In addition, referring to FIG. 3, which illustrates a voice signal graph and a voice signal spectrogram in a voice recognition method according to an exemplary embodiment of the present invention, (a) shows a male voice signal of 'a, da, other', (b) shows the female voice signal of 'a, da, other', and (c) and (d) shows the male and female voice signals for the 'ga' syllable with a window of 20ms. windowing), the spectrogram, and the spectrogram of linear predictive coding (10th order).

As a result of performing syllable unit dynamic time warping on the 'ga, da, other' speech signals illustrated in FIGS. 3 (a) and 3 (b), the linear feature in the feature vector extraction through the linear prediction coding for each syllable The predictive coding coefficients were calculated up to 10th order, and the dynamic time warping distances between male and female are shown in the following table.

As shown in the table, the recognition rate was high in the case of 'a' and 'da', but the performance of the comparison algorithm by dynamic time warping for similar syllables such as 'da' and 'ta' was not high. Therefore, the recognition accuracy can be improved by additionally performing a pattern comparison algorithm of phoneme units using a hidden Markov model.

Meanwhile, the word search and final word selection steps shown in FIG. 2 will be described in detail with reference to FIG. 4. 4 is a flowchart illustrating in more detail a word search and final word selection step in a consonant-based speech recognition method according to an embodiment of the present invention.

When the phonemes for the voice input by the user are recognized by performing the steps S10 to S60 sequentially shown in FIG. 2, the control unit of the voice recognition system corresponding to the determination unit 14 of FIG. In step S71, all words corresponding to the determined phonemes are searched. The database may be, for example, a database of a destination name in the navigation device.

Then, it is determined whether more than one word searched in step S71 (S72). If there is only one word searched, the searched word may be selected as the final word and output, but if there is more than one searched word, it is distinguished to determine which word among the searched words is selected and outputted as the final word. .

If it is determined in step S72 that more than one word is searched, the searched words are further divided into simple noun units (S73). The destination name of the navigation device is often composed of compound nouns such as 'Seoul Tourist Hotel'. For example, a plurality of simple nouns such as 'Seoul', 'tourism', and 'hotel' are formed in one word. In step S73, all the words searched are divided into simple noun units.

The user inputs 'sa, ah, ah, ah, sa, ha, a' and is recognized as 'ㅅ, ㅇ, ㅇ, ㅅ, ㅎ, ㄱ' in the voice recognition device, and 'Seoul Art School' and 'Sangam Art' For example, three words were searched for 'hall,' and 'Seoul women's school.'For example, each of these words is referred to as' Seoul / Art / School', 'Sangam / Art / Hall', and 'Seoul / Women / school' It is possible to distinguish.

After dividing the simple nouns included in the searched word as described above, the frequency in the search range of each simple noun is detected (S74). That is, in the same example, the total number of words searched is three, of which two words contain 'Seoul', two words contain 'Art', two words contain 'School', 'Sangam', 'hall', and 'female' each included one.

Thereafter, a step of selectively weighting each simple noun is performed (S75). Words that have high importance or search frequency among each word but actually have low frequency in the database, such as hospitals, fire departments, schools, ward offices, city halls, etc. In comparison, since the number included in the destination name included in the database is small, it can be separately weighted.

For example, the weight for a simple noun representing a government office can be calculated by the following equation.

는 공공기관을 나타내는 단순명사를 포함하는 단어의 전체 수이고,

는 공공기관을 나타내는 단순명사 중 해당 단순명사를 포함하는 단어수가 가장 많은 단순명사를 포함하는 단어의 수이다. Where above

Is the total number of words, including simple nouns that represent public authorities,

Is the number of words including the simple noun with the largest number of words including the simple noun among the simple nouns representing public institutions.

In operation S76, the importance index of the searched words is calculated based on the frequency detected in step S74 and the weights assigned in step S75. Equation for calculating the importance index can be determined as follows.

또는

or

는 검색된 각 단어의 중요도 지수이고,

는 검색된 모든 단어 중 특정 단순명사가 포함된 단어의 수이며,

는 검색된 모든 단어의 수이다. 또한

는 위에서 설명한 가중치이다.here

Is the importance index for each word found,

Is the number of words with a specific simple noun among all words found.

Is the number of all words retrieved. Also

Is the weight described above.

The equation shown on the left is an importance index expression when the weight is not taken into consideration, that is, when the step S75 is not performed, and the equation shown on the right is a case considering the weight.

는 3이다. 그리고 ' 서울'이라는 단어의 빈도수는 2이므로, '서울'이라는 단어에 대한

는 2이다. 이와 같은 식으로, '서울예술학교'에 포함된 각 단어, '서울', '예술', '학교'에 대한

값을 각각 산출하면, 2/3, 2/3, 2/3이고, 따라서 총합

은 2가 된다.For easier understanding, for example, if you enter 'sa, ah, ah, ah, sa, ha, ga' in the example as described above, all words searched for 'Seoul Arts School', 'Seoul Women's School', ' When calculating the importance index for each of the Sangam Arts Centers, the Seoul Arts School includes three simple nouns. therefore

Is 3. And because the frequency of the word "Seoul" is 2, the word "Seoul"

Is 2. In this way, each word included in 'Seoul Art School', 'Seoul', 'Art', 'School'

If you calculate the values, respectively, it is 2/3, 2/3, 2/3, so the sum

Becomes two.

On the other hand, if the importance index is calculated in the same way for 'Seoul Women's School', it becomes 2/3 + 1/3 + 2/3, which is 5/3. And Sangam Art Center is 1/3 + 2/3 + 1/3, which is 4/3.

Therefore, the word with the highest importance index among the above three words becomes 'Seoul Art School'.

Even when the weight is considered, the total value may be obtained by adding the weights for each word together.

However, the equation for calculating the importance index in step S76 is not necessarily defined as Equation 2a described above. As another example, the following equation is illustrated.

은 검색된 단어에 포함된 단순명사의 수이다.here

Is the number of simple nouns included in the searched words.

For example, if the first consonants 'ㅁ, ㄱ, ㅅ, ㅈ' are input, and the words searched for the two are 'Mungyeongsaejae' and 'Myeongok Market', according to Equation 2a, 'Mungyeongsaejae' is included. The simple noun is one, which is 1/2 when the importance index is calculated, whereas the 'membrane market' has one word containing 'memories' and one word containing 'market' out of two searched words. Therefore, the higher the number of simple nouns included in a word regardless of frequency, the higher the importance index may appear. Therefore, the importance index may be calculated by dividing the calculated frequency by the number of simple nouns included in the word.

Then, by comparing all the importance indexes calculated in step 76 as described above (S77), the word having the highest importance index among the searched words is selected as the final word (S78).

However, the calculation of the importance index and outputting the final selected word may be performed according to a selection, displaying all the words searched in alphabetical order, or listing all the searched words in order of the calculated importance index. It is also possible.

The rights of the present invention are not limited to the embodiments described above, but are defined by the claims, and those skilled in the art can make various modifications and adaptations within the scope of the claims. It is self-evident.

1 is a block diagram showing a schematic configuration of a general speech recognition system.

2 is a flowchart sequentially showing a voice recognition method according to an embodiment of the present invention.

Figure 3 is a voice signal graph and voice signal spectrogram in the voice recognition method according to an embodiment of the present invention.

Figure 4 is a flow chart illustrating in more detail the word search and the final word selection step in the speech recognition method according to an embodiment of the present invention.

** Description of the symbols for the main parts of the drawings **

10: voice input unit 11: feature analysis unit

12: pattern recognition unit 13: reference pattern storage unit

14: decision

Claims (14)

A voice input unit which receives a voice and converts the voice into a voice signal; A feature analyzer for generating an input pattern by calculating a feature vector of the voice signal received from the voice input unit; A reference pattern storage unit for storing reference patterns for performing speech recognition in comparison with the input pattern; A pattern recognition unit for comparing the input pattern with the reference pattern and outputting consonant information corresponding to a reference pattern similar to the input pattern; And In the initial speech-based speech recognition device including a decision unit for determining a word to be output by searching for a word corresponding to the initial information detected by the pattern recognition unit, The reference pattern stored in the reference pattern storage unit, A pattern generated based on syllables in which the same vowel is combined for each consonant; The pattern recognition unit, A syllable unit comparison algorithm is first performed on the input pattern to detect similar reference patterns. However, when two or more similar reference patterns are detected, a similar phonetic unit comparison algorithm is selectively performed to finally detect one similar reference pattern. ; The determination unit, Comprising a plurality of words stored in the database, the word stored in the database to search for the word corresponding to the initial consonant information detected by the pattern recognition unit, and outputs, the word corresponding to the initial consonant information detected by the pattern recognition unit If more than one is found, the initial speech recognition apparatus characterized in that it calculates the importance of all the searched words. The method of claim 1, The syllable unit comparison algorithm is Dynamic Time Warping (DTW) method; The phoneme comparison algorithm, Hidden Markov model method; In the reference pattern storage unit, A reference pattern for the syllable unit comparison algorithm and a reference pattern for the phoneme unit comparison algorithm are stored separately. The method of claim 2, The pattern recognition unit, When the Euclidean square distance between the input pattern and each reference pattern is calculated by performing the syllable unit comparison algorithm, and the calculated Euclidean square distance is two or more reference patterns that are less than a previously stored threshold distance, A speech-based speech recognition device, characterized in that for performing a phoneme comparison algorithm. delete delete The method of claim 1, The determination unit, A range of searched words, the frequency of simple nouns included in each of the searched words, and based on the frequency of the detected simple noun, the initial speech-based speech recognition device characterized in that for calculating the importance of each word. The method of claim 6, The determination unit, And detecting the importance of each word based on the frequency and the weight by detecting whether a simple noun with a predetermined weight is included in each searched word. The method of claim 1, The determination unit, A first speech-based speech recognition device, characterized in that for outputting the calculated word of high importance or outputting only the word of highest importance. (A) receiving a voice and converting it into a voice signal; (B) generating an input pattern by calculating a feature vector of the voice signal; (C) detecting a similar reference pattern by comparing the input pattern with previously stored reference patterns according to a syllable unit comparison algorithm; (D) selectively detecting one similar reference pattern according to a phoneme unit comparison algorithm only when two or more similar reference patterns are detected in step (C); (E) searching for and outputting a word having a consonant corresponding to one similar reference pattern detected in step (C) or step (D), Step (E), (E1) searching for all of the words having the initial property corresponding to the detected one similar reference pattern; (E2) detecting a frequency of simple nouns included in each word searched in the range of words searched in step (E1); (E3) calculating the importance of each word based on the frequency of the simple noun detected in the step (E2); And E4 based speech recognition method, characterized in that is performed including the step of outputting a searched word according to the importance calculated in the step (E2). 10. The method of claim 9, The reference pattern is, Based on syllables in which the same vowels are combined with each consonant, a reference pattern for the syllable unit comparison algorithm and a reference pattern for the phoneme unit comparison algorithm are separately generated and stored in advance. Way. The method of claim 10, The syllable unit comparison algorithm is Dynamic Time Warping (DTW) method; The phoneme comparison algorithm, A blind-based speech recognition method, characterized in that it is a Hidden Markov model method. delete 10. The method of claim 9, Step (E3), And detecting the importance of each word based on the frequency and the weight. The method of claim 13, The weighted simple noun, A consonant based speech recognition method characterized by simple nouns representing public institutions.

Images