KR100340688B1 - Method for extracting the number of optimal allophone in order to speech recognition - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method of extracting the optimal number of variances for speech recognition.

2. The technical problem to be solved by the invention

The present invention applies a clustering technique using entropy, and measures the loss of information using entropy between each variation sound and merged sound, and selects a pair that minimizes the loss of information in the entire variation sound model while clustering as many times as desired. It is an object of the present invention to provide a method for extracting the optimal number of mutations using a technique and a computer-readable recording medium recording a program for realizing the method.

3. Summary of Solution to Invention

The present invention, the first step of loading the initial distribution information; Obtaining a loss of information using entropy and recording the minimum loss value and the corresponding pair of variation sounds for all the central phones in case of merging each pair of variation sounds belonging to one central phoneme; Obtaining a merged pair having a minimum information loss value from among the pairs of variable sounds having the minimum loss value for each center phoneme obtained in the second step, updating the distribution information, and clustering the second step by the desired number of phonemes. The third step is to repeat until.

4. Important uses of the invention

The present invention is used in a speech recognizer and the like.

Description

Optimal number extraction method for speech recognition {METHOD FOR EXTRACTING THE NUMBER OF OPTIMAL ALLOPHONE IN ORDER TO SPEECH RECOGNITION}

The present invention relates to an optimal method for extracting the number of variable sounds for speech recognition and a computer-readable recording medium recording a program for realizing the method. In particular, by applying an entropy clustering technique to an existing variable sound model A method of extracting an optimal number of variation sounds while changing the number of variation sounds, and a computer-readable recording medium having recorded thereon a program for realizing the method.

In general, acoustic models mainly used in speech recognizers on personal computers (PCs) include words, syllables, phonemes, and transition sounds. Of these, phoneme units are widely preferred because of their advantages, which are much smaller than words or syllable units. However, the phoneme unit exhibits different phonetic characteristics even though the same phoneme is influenced by the surrounding phonemes. Therefore, in order to create a high-performance speech recognizer, it is necessary to implement a sound model by subdividing the phonemes so that the surrounding environment of each phoneme can include the influence that the phoneme can have. As a method of implementing an acoustic model using the phoneme segmentation method, a method using a variation sound unit is mainly used.

Conventional techniques in this field include acoustic classification using a speech database, classification using a match score, and classification using a variation sound group tree. These prior arts are mostly classification methods based on foreign languages such as English. Therefore, in the case of implementing an acoustic model of a speech recognizer capable of recognizing Korean language, it is difficult to apply the methods and has an inefficient disadvantage.

In addition, there is a conventional technology applied to Korean language, but this method is applied based on the grouping tree of these mutant sounds after determining the combination form of Korean mutant sounds that can be generated. Therefore, since the unacceptable variations are included in the acoustic model, the number of variations is quite large. Thus, this prior art causes various problems in actual implementation in a system such as a personal computer (PC).

That is, when the acoustic model using the mutant sounds is applied and classified into Korean, it is theoretically divided into 3,000 or more, and is classified into more than 1,500 even if the frequency is small. This subdivided mutant model has the advantage that the performance of the speech recognizer can be improved compared to the acoustic model consisting of 40 phonemes, but it requires a large amount of memory in actual implementation and is difficult to operate in real time. Have Therefore, quantitative optimization of the variance model is essential.

In order to solve the above problems, the present invention applies a clustering technique using entropy, and measures a loss of information using entropy between each variation sound and a merged sound, and selects a pair in which the information loss in the entire variation sound model is minimized. It is an object of the present invention to provide a method of extracting the optimal number of variation sounds using a technique of grouping as many times as desired while repeating the work, and a computer-readable recording medium recording a program for realizing the method.

1 is an exemplary configuration diagram of a variable vocabulary speech recognizer to which the present invention is applied.

Figure 2 is a structure diagram of one embodiment of a conventional variable sound model used in the present invention.

Figure 3 is an illustration of applying the information loss calculation method used in the present invention to a conventional variable sound model.

4 is a flowchart illustrating an embodiment of a method for extracting the optimum number of variations according to the present invention.

* Explanation of symbols for the main parts of the drawings

101: voice input unit 102: pronunciation dictionary generation unit

103: acoustic modeling unit 104: speech recognition unit

105: user interface unit

In order to achieve the above object, the present invention provides a method for extracting the optimum number of variances applied to an acoustic modeling unit of a speech recognizer, comprising: a first step of loading initial distribution information; Obtaining a loss of information using entropy and recording the minimum loss value and the corresponding pair of variation sounds for all the central phones in case of merging each pair of variation sounds belonging to one central phoneme; Obtaining a merged pair having a minimum information loss value from among the pairs of variable sounds having the minimum loss value for each center phoneme obtained in the second step, updating the distribution information, and clustering the second step by the desired number of phonemes. Characterized in that it comprises a third step to repeat until.

On the other hand, the present invention, a speech recognizer having a processor, the first function for loading the initial distribution information; A second function of obtaining information loss using entropy and recording a minimum loss value and a corresponding pair of variation sounds for all the central phonemes in case of merging each pair of variation sounds belonging to one central phoneme; Obtaining a merged pair having a minimum information loss value among the pairs of variable sounds having a minimum loss value for each center phoneme obtained by the second function, updating the distribution information, and clustering the second function by a desired number of phonemes. Provided is a computer readable recording medium having recorded thereon a program for realizing a third function to be repeatedly performed.

The present invention maintains the optimal number of models in each step in each iteration. In addition, by evaluating the recognition performance of each model by extracting various variance models composed of various numbers, it is possible to obtain an optimal model showing high performance while satisfying the smallest amount of memory occupancy and real-time operation. In addition, there is an advantage that can be selected from the various models, such as the recognition performance / use memory / real-time operation or not according to the user's purpose.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration diagram of a variable vocabulary speech recognizer to which the present invention is applied.

The variable vocabulary speech recognizer is divided into a speech input unit 101, a pronunciation dictionary generator 102, a sound modeling unit 103, a speech recognition unit 104, and a user interface unit 105 in which the technique of the present invention is implemented. do.

The voice input unit 101 has a function of receiving a user's voice from a microphone and processing it into digital data in a form capable of performing voice recognition. The module detects the beginning and end of the voice spoken by the user.

The pronunciation dictionary generation unit 102 converts the recognition vocabulary input from the user interface unit 105 into a phoneme string pronunciation dictionary used in the recognizer. The input from the user interface unit 105 is a vocabulary having a 2-byte complete Hangul text form, and the output is a phonetic dictionary converted according to a pronunciation rule and a predefined ASCII phonetic notation.

The acoustic modeling unit 103 extracts the variation sounds for the phoneme sequence of the pronunciation dictionary generated from the pronunciation dictionary generator 102, connects them to each other, generates a reference pattern in units of the variation sound, and hands it to the voice recognition unit 103. Have

The voice recognition unit 104 has a function of recognizing the input voice signal using the recognition model, the pronunciation dictionary, and the acoustic model, and passes the recognition result to the user interface unit 105.

The user interface unit 105 is a central control unit for managing the entire system, and has functions such as order control of each function module, a graphical user interface (GUI), and a recognition result display.

FIG. 2 is a structural diagram of an embodiment of a conventional mute model used in the present invention, and illustrates a structure of an existing mute model used to generate an acoustic model to which the present invention is applied.

This acoustic model consists of a set of variation sounds 202 classified by the central phone 201. The central phone 201 is represented by 40 models represented by Korean consonants, vowels, and silent sounds. In addition, in each of the central phonemes, there are as many as N (N is a natural number) variation sounds modeling phonemes that may come before and after the phoneme. For example, the set of phonemes 202 that can occur with the central phone 201 in the middle of '??' is such as 'A, B, ...'.

The existing acoustic model classifies the phoneme groups that can influence Korean's speech generation into seven, five, and five groups based on phonological knowledge for the cases of initial consonants, final consonants, and vowels, respectively. The total number of variations in Korean that can be generated by this taxonomy is about 3,381 or less in consideration of variations that cannot be realized in practice. In addition, by applying the phonetic dictionary of the variable vocabulary speech recognizer, the variable phonetic model can be reduced to approximately 1,548 different phonetic models except for low frequency voices. However, the reduced-tone model, which is reduced to this extent, is a significant disadvantage in the amount of memory used and the execution time of the recognition.

The method of extracting the optimum number of variation sounds applied in the present invention measures the loss of information using entropy between each variation sound and the merged variation sound, and clusters as many times as desired while repeating the operation of selecting a pair in which the loss in the total variation sound model is minimized. Way. First, a method of calculating entropy and information loss using distribution information of a hidden Markov model (HMM) will be described. First, the count value of the codeword i with respect to the distribution d of the context a is defined as N _a , _d (i).

At this time, the output probability P and the count value N are expressed as Equation 1 below.

P _a , _d (i) = N _a , _d (i) / N _a , _d

N _a , _d = N _a , _d (i)

When expressed using the above Equation 1, entropy H is as shown in Equation 2 below.

P _a , _d (i) * log (P _a , _d (i))

If a model in which context a and context b merge with each other is called context m, the merged count N _m and _d (i) are given by Equation 3 below.

N _m , _d (i) = N _a , _d (i) + N _b , _d (i)

When the entropy H for the context b and the context m is obtained using each of the above equations, the information loss between the original model and the merged model can be obtained using Equation 4 below.

L _d (a, b) = N _m , _d * H _m , _d -N _a , _d * H _a , _d -N _b , _d * H _b , _d

FIG. 3 is an exemplary diagram of applying the information loss calculation method used in the present invention to an existing mute model, in which A and B are merged among the mute models 202 belonging to the central phone 201 '??'. Is an example.

First, the entropy H and the count value N for each transition sound are obtained. In this case, b, m, and e that are displayed in each of the transition models 301, 302, and 303 correspond to a start, middle, and end state of the hidden Markov model HMM. Next, L is the loss for the case of finally merging variance models A and B after finding information loss for each start, middle, and end state using the above values. Obtain (A, B) as shown in Equation 5 below.

L _b (A, B) = N _mb * H _mb -N _ab * H _ab -N _bb * H _bb

L _m (A, B) = N _mm * H _mm -N _am * H _am -N _bm * H _bm

L _e (A, B) = N _me * H _me -N _ae * H _ae -N _be * H _be

L (A, B) = Lb (a, b) + Lm (a, b) + Le (a, b)

In this case, the merging criterion of the mute model is to merge the pair having the smallest value among the information loss obtained for all pairs by the 40 center phonemes described above. Reduce it. Using such a method, various numbers of mute models can be generated from existing mute models.

4 is a flowchart illustrating an exemplary method for extracting the optimum number of variation sounds according to the present invention.

First, in order to operate the algorithm, the initial distribution information by the existing variable sound model is loaded (401). In this distribution information, the count value of each transition sound is stored. Next, it is checked whether the number of variation sounds in the current acoustic model exists as much as desired (402). If the number is desired, the algorithm terminates the algorithm and outputs information on the currently existing variation sound (403). This information is an acoustic model with an optimized number of variations.

On the other hand, if the result of the test 402, the number of the variation sound is more than desired, information loss is calculated for the case of merging each pair of variation sounds belonging to one central phoneme (404). The information loss calculation method is obtained by the method described with reference to FIG. 3. Among the information loss thus obtained, the smallest loss value and the corresponding pair of variation sounds are recorded (405). This process (404, 405) is repeated for all 40 center phonemes (406), and when a pair of variation sounds having a minimum loss value for each center phoneme is obtained, the smallest of them is compared and found, and the value and corresponding variation sound are found. Record the pair (407). Since the thus obtained pair of pairs is a pair that minimizes the loss of information in the whole paired model, the pairs are merged to reduce the number of pairs of pairs and update distribution information (408). After all of these processes are completed, the process returns to the process of checking the number of variation sounds (402) and checks whether the number is reduced by the desired number and repeats the following process.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible in the art without departing from the technical spirit of the present invention. It will be apparent to those of ordinary knowledge.

As described above, the present invention has an effect of quantitatively reducing the number of variation sounds in the acoustic model composed of approximately 1,500 to about 3,000 variation sounds.

Therefore, the present invention has an effect of generating an acoustic model having the number of variations as desired by the user when applied to the existing Korean variations. That is, when the variable sound acoustic model using the present invention is used in a variable vocabulary speech recognizer, the memory occupies a smaller amount of memory than the conventional acoustic model when the program is executed. In addition, since the number of categorized variation sounds is reduced, the performance of the speech recognizer relative to the input speech is also relatively high, and the complexity of the speech recognizer is also reduced.

Such effects are factors that greatly affect the performance and marketability of the speech recognizer when the variable vocabulary speech recognizer is ported to a general-purpose system such as a personal computer (PC) and commercialized. As a result of applying the present invention to a small variable vocabulary speech recognizer aimed at recognizing personal computer (PC) instructions, we have developed a context-dependent anomaly model that successfully shows high recognition performance even with 200 to 400 variations. I can see that it can be implemented.

Therefore, if the present invention is applied to the acoustic model optimization field of the Korean speech recognizer, it is very effective in terms of performance, memory usage, and complexity of the speech recognizer, and can greatly accelerate the commercialization of the variable vocabulary speech recognizer.

Claims (6)

In the optimum number of variance sound extraction method applied to the acoustic modeling unit of the speech recognizer, A first step of loading initial distribution information; Obtaining a loss of information using entropy and recording the minimum loss value and the corresponding pair of variation sounds for all the central phones in case of merging each pair of variation sounds belonging to one central phoneme; And Obtaining and merging a pair of mute pairs having the minimum information loss value among the pairs of mute pairs having the minimum loss value for each center phoneme obtained in the second step, updating the distribution information, and clustering the second step to the desired number of phonemes Third Step to Repeat Optimal variation number extraction method comprising a. The method of claim 1, The second step, A fourth step of calculating information loss using entropy between the merged variation sounds when merging pairs of variation sounds belonging to one central phoneme; A fifth step of recording the lowest loss value and the corresponding pair of mute pairs among the information loss obtained in the fourth step; And A sixth step of repeating the fourth and fifth steps for all the central phonemes to obtain a pair of variation sounds having a minimum loss value for each center phoneme Optimal variation number extraction method comprising a. The method according to claim 1 or 2, The third step, A seventh step of comparing the pairs of variation sounds having the minimum loss value for each center phoneme, finding a minimum value among them, and recording the value and the pair of variation sounds; An eighth step of reducing the number of variation sounds and updating distribution information by merging pairs which minimize the loss of information obtained in the seventh step; And If it is not checked whether the number of variation sounds is reduced by the desired number, the process proceeds to the second step; and if the number is reduced by the desired number, the ninth step of outputting information on the existing variation sounds Optimal variation number extraction method comprising a. In a speech recognizer having a processor, A first function of loading initial distribution information; A second function of obtaining information loss using entropy and recording a minimum loss value and a corresponding pair of variation sounds for all the central phonemes in case of merging each pair of variation sounds belonging to one central phoneme; And Obtaining, merging, and updating distribution information among the pairs of mute pairs having the minimum loss value of each center phoneme obtained by the second function, and updating the distribution information, and clustering the second function by the desired number of phonemes. 3rd function to repeat A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 4, wherein The second function is, A fourth function of calculating information loss using entropy between the merged variation sounds when merging pairs of variation sounds belonging to one central phoneme; A fifth function of recording the lowest loss value and the corresponding pair of mute pairs among the information losses obtained in the fourth function; And A sixth function of repeating the fourth function and the fifth function with respect to all center phonemes to obtain a pair of variation sounds having a minimum loss value for each center phoneme A computer-readable recording medium having recorded thereon a program for realizing this. The method according to claim 4 or 5, The third function, A seventh function of comparing the pairs of variation sounds having the minimum loss value for each center phoneme, finding a minimum value among them, and recording the value and the pair of variation sounds; An eighth function of reducing the number of variation sounds and updating distribution information by merging pairs which minimize the loss of information obtained in the seventh function; And If it is not checked whether the number of variation sounds has been reduced by the desired number, the process goes to the second function, and if the number is reduced by the desired number, the ninth function of outputting information on the existing variation sounds A computer-readable recording medium having recorded thereon a program for realizing this.