KR20130126570A - Apparatus for discriminative training acoustic model considering error of phonemes in keyword and computer recordable medium storing the method thereof - Google Patents

Abstract

The present invention relates to a device for discriminative acoustic model training by considering results of phoneme errors in keywords and a computer readable recording medium recording a method therefor. The present invention comprises a keyword detection unit detecting a keyword recognition result as a speech recognition result about keywords from an N-best recognition result as the speech recognition result for a word string including the keywords; a keyword training unit obtaining phoneme model parameter distribution of correct phonemes and error phonemes according to the keyword recognition results; and a keyword discriminative training unit applying a log-likelihood value of the error phonemes as a weighted value in the keyword recognition results and updating the phoneme model parameter of the error phonemes to minimize mutual information in proportion to a count value of the error phonemes. [Reference numerals] (100) Property extracting unit;(200) Investigation unit;(300) Sound model database;(400) Pronunciation dictionary database;(500) Language model database;(600) Text database;(700) Language model learning unit;(800) Sound database;(910) Keyword extracting unit;(912) Keyword database;(920) N-best alignment unit;(930) Keyword learning unit;(940) N-best learning unit;(950) Keyword discrimination learning unit;(960) N-best discrimination learning unit;(AA) Sound signal;(BB) Word row

Description

Apparatus for discriminative training acoustic model considering error of phonemes in keyword and computer recordable medium storing the method according to the present invention.

The present invention relates to a sound model discrimination learning method, and more particularly, to an acoustic model discrimination learning apparatus capable of minimizing mutual information of phonemes in an acoustic model in consideration of an error result of phonemes in a key word, and such discrimination learning. A method relates to a computer readable recording medium having recorded thereon.

Speech recognition is the identification of linguistic semantic content from speech by automatic means. In detail, a process of identifying a word or a word string and extracting a meaning by inputting a speech waveform. Such speech recognition can be classified into five categories: speech analysis, phoneme recognition, word recognition, sentence interpretation, and meaning extraction. Speech recognition has a narrow meaning from speech analysis to word recognition.

As an improvement of the human-machine interface, research and development of speech recognition technology for inputting information with voice and speech synthesis technology for outputting information with voice have been in progress for a long time. With the development of large scale integrated circuits (LSIs), speech recognition devices and speech synthesis devices that require large devices can be realized on integrated circuits of several millimeters in width.

It is currently used for checking bank balances by phone, checking stock quotes, applying for mail-order sales, looking up credit cards, and making reservations for hotel or aircraft seats. These services, however, use a word speech recognition device that recognizes a speech which is pronounced by separating a limited number of words one by one. The ultimate goal of speech recognition is the realization of a complete speech or text conversion that recognizes speech by natural utterance and accepts it as an execution command or inputs it into a document as data. It is not only to recognize words, but also to develop a speech understanding system that accurately extracts the meaning contents of continuous speech or sentences using phrase information, semantic information, and work-related information and knowledge. The research and development of these systems is in progress.

Korean Laid-Open Patent No. 2013-0067854, published on June 25, 2012 (Name: Corpus-based language model discrimination learning method and apparatus)

An object of the present invention is to provide an apparatus for discriminating and learning a sound model that can minimize mutual information of phonemes using key words in a speech recognition system, and a computer readable recording medium having recorded thereon a method therefor.

The apparatus for discriminating and learning acoustic models according to an embodiment of the present invention for achieving the above object is a speech recognition result for the key word from the N-best recognition result, which is a speech recognition result for the word string including the key word. A key word detection unit for extracting a key word recognition result, a key word learning unit for deriving a phoneme model parameter distribution of correct answer phonemes and error phonemes according to the key word recognition result, and applying a log likelihood of error phonemes from the key word recognition result as a weight, And a key word discrimination learning unit for updating a phoneme model parameter distribution of the error phoneme so that the amount of mutual information is minimized in proportion to the count value of the error phoneme.

The computer-readable recording medium according to the preferred embodiment of the present invention for achieving the above object, the key word that is the speech recognition result for the key word from the N-best recognition result that is the speech recognition result for the word string including the key word Extracting a recognition result, deriving a phoneme model parameter distribution of a correct answer phoneme and an error phoneme according to the keyword recognition result, applying a log likelihood of an error phoneme as a weight in the keyword recognition result, Updating a phoneme model parameter distribution of the error phoneme in such a manner that the amount of mutual information is minimized in proportion to a count value.

According to an exemplary embodiment of the present invention, since discriminative learning is performed according to the key word recognition result along with the discriminative learning through the N-best recognition result, the performance of discriminative learning for a specific word, that is, a key word can be improved. In this case, compared to simply performing discriminative learning based on N-best recognition results, a more efficient acoustic model can be constructed for all users or specific users through keyword recognition results. Furthermore, when updating the phonemic model parameter distribution of the acoustic model, the log likelihood of the phonemes classified as the error recognition results is applied as a weight, and the error recognition results are reflected to minimize the amount of mutual information, thereby effectively in discriminative training. This can minimize the amount of mutual information.

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the technical features of the invention.
1 is a conceptual diagram illustrating the concept of a learning method according to an embodiment of the present invention.
2 is a view for explaining a speech recognition system according to an embodiment of the present invention.
3 is a view for explaining the internal configuration of the acoustic model learning unit according to an embodiment of the present invention.
4 is a flowchart illustrating an acoustic model training method according to an exemplary embodiment of the present invention.
5 is a flowchart illustrating an acoustic model training method of an acoustic model training unit according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in describing in detail the operating principle of the preferred embodiment of the present invention, if it is determined that the detailed description of the related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. This is to more clearly communicate without obscure the core of the present invention by omitting unnecessary description.

Also, when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may be present in between . In addition, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, the terms "comprises" or "having" described herein are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or the same. It is to be understood that the present invention does not exclude in advance the possibility of the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

1A to 1D are conceptual views illustrating the concept of differential learning according to an embodiment of the present invention.

Prior to the description of FIGS. 1A to 1D, a voice recognition method according to an exemplary embodiment of the present invention will be described. In speech recognition, a sound of an input speech is analyzed and a feature vector of a predetermined dimension representing a feature amount of the speech is extracted. Thereafter, matching processing between the feature vector and the acoustic model is performed. According to the embodiment of the present invention, the matching process is made in phoneme units. As a result of the matching processing, the phoneme of the acoustic model matching the feature vector becomes the speech recognition result (recognition result). In the matching process, using a probability distribution (phoneme model parameter distribution) constituting the acoustic model, a log likelihood in which a feature vector for the acoustic model is observed is calculated as a plurality of candidates of the speech recognition result. For example, when the input word ("special") is composed of three phonemes A (ㅌ), B (B), and C (a), the result as shown in Table 1 below may be output.

Phoneme A recognition result
(Log likeness) Phoneme B recognition result
(Log likeness) Phoneme C recognition result
(Log likeness) Candidate 1 E (0.1) V (0.2) E (0.1) Candidate 2 A (1.2) V (0.2) C (1.4) Candidate 3 A (1.2) B (2.2) C (1.4)

In Table 1, the log likelihood is the probability that the recognized phoneme is the phoneme in the acoustic model. In other words, the log likelihood is the similarity (probability) between the phoneme model parameter distribution of the acoustic model and the phoneme of the input voice. When the recognition result of each candidate is calculated, the final speech recognition result is determined among the plurality of candidates based on the log likelihood. In Table 1, candidate 3 will be selected. That is, the candidate with the highest log likelihood among the plurality of candidates is selected as the closest to the input voice, and the word string corresponding to the acoustic model constituting the candidate will be output as the speech recognition result.

Discriminant learning according to an embodiment of the present invention uses both N-best recognition results and keyword recognition results. Here, if the N-best recognition result is a recognition result for a specific word string, the keyword recognition result may be a recognition result for at least one word designated as a key word among words included in the specific word string.

For example, suppose a voice signal such as "I applied for an account this month" is input. In this case, when the N-best recognition result is N = 3, it is assumed that the N-best recognition result value is shown in Table 2 below.

N-best recognition result Answer / error I applied for an account this month. Answer recognition result I applied for an February month account. Error recognition result I founded a field moon in February. Error recognition result

As shown in Table 2, the voice recognition result and the correct answer recognition result may be output as "I applied for an account this month."

In addition, when the key word is "account", the key word recognition result may be as shown in Table 3 below.

Keyword recognition result Answer / error Account Answer recognition result Account Answer recognition result Field Error recognition result

As shown in Table 3, the keyword speech recognition result correct answer recognition result may be a "account".

As described above, for the N-best recognition result, the recognition result for the word string "I applied for an account this month", and for the keyword recognition result, for the keyword "account", The N-best recognition result and the keyword recognition result may be different. That is, according to the N-best recognition result, the key word included in the recognition result that is an error may be the correct answer. And vice versa. Therefore, according to the exemplary embodiment of the present invention, the differential learning of the word string including the core word and the differential learning of the key word are performed together.

1A to 1D, a discrimination learning method according to an embodiment of the present invention will now be described. To emphasize, the discrimination learning described below can be applied to both N-best recognition results and keyword recognition results.

Referring to FIG. 1A, it is assumed that phonemes A, B, and C exist in an acoustic model as shown in Table 1 below. Reference numerals 10, 20, and 30 denote a phoneme model parameter distribution of phonemes A, B, and C. The phoneme model parameter distribution may be a probability distribution of the corresponding phoneme. Here, reference numeral 40, which is a part where the phoneme model parameter distribution of each phoneme overlaps, indicates the mutual information amount.

In the speech recognition method based on the above-described speech recognition method, when there is a large amount of mutual information, a problem of lowering the recognition accuracy for clearly distinguishing the phonemes of the input voice occurs. Thus, an embodiment of the present invention updates the phoneme model parameter distribution to minimize this amount of mutual information.

The meaning of "minimization of mutual information" according to an embodiment of the present invention will be described below. Referring to FIG. 1B, a phoneme model parameter distribution for two phonemes, ie, phonemes M and N, is shown. For example, the phoneme M may be "ㅘ" and the phoneme N may be "ㅏ".

Reference numerals 50 and 60 denote phoneme model parameter distributions for the phonemes M and N, respectively, of the current acoustic model, and reference numerals 70 and 80 assume an ideal phoneme model parameter distribution in which mutual information amounts for the phonemes M and N are minimized, respectively.

Reference numeral 90 denotes a mutual information amount between phoneme model parameter distributions 50 and 60 for phonemes M and N of the current acoustic model. Although the phoneme M is input due to the mutual information amount 90, it can be recognized as the phoneme N. Therefore, this amount of mutual information should be minimized. In order to minimize the mutual information amount 90, the phoneme model parameter distribution 50 of phoneme M of the current acoustic model should be moved to the ideal phoneme model parameter distribution 70 of phoneme M. Alternatively, the phoneme model parameter distribution 60 of phoneme N of the current acoustic model should be moved to the ideal phoneme model parameter distribution 80 of phoneme N.

As described above, the speech recognition result is output with the probability (similarity) that the phonemes of the input voice signal are the same phonemes as the phonemes stored in the acoustic model. Voices for training or learning can use known voices. Therefore, in the case of the speech recognition result or the correct answer recognition result, the corresponding phoneme is called the correct answer phoneme. In the case of the speech recognition result or the error recognition result, the corresponding phoneme is called the error phoneme. For example, when phoneme M is input, when it recognizes as phoneme M, it is called a result of correct answer recognition, and phoneme M is called correct answer phoneme. In addition, when phoneme M is input and recognized as phoneme N, it is called an error recognition result and phoneme N is called error phoneme.

Referring to FIG. 1C, it is assumed that the speech recognition result for the phoneme M is the correct answer recognition result. Therefore, in the case of phoneme M whose answer is correct, the phoneme model parameter distribution of the acoustic model is updated by using the maximum mutual information (MMI) estimation method according to the speech recognition result. The phoneme model parameter distribution 50 for phoneme M will move in the direction of the ideal phoneme model parameter distribution 70 for phoneme M. In the case of Figure 1c it was minimized to an ideal form with no mutual information at all.

Meanwhile, referring to FIG. 1D, it is assumed that an error recognition result is output as a voice recognition result for the phoneme M. For example, assume that the speech recognition result for phoneme M is output as N. This is a result of error recognition by the amount of mutual information indicated by 90. In such a case, there was no update of the acoustic model since the error recognition result was conventionally ignored.

However, according to an embodiment of the present invention, the log likelihood of phoneme N, in which the speech recognition result is an error, is derived. In this case, the log likelihood is calculated using a maximum likelihood (ML) estimation method. In addition, the count value of the error phoneme according to the error recognition result, that is, the number of times determined as the error phoneme from the error recognition result is derived. The log likelihood is then applied as a weight and the phoneme model parameter distribution of the acoustic model is updated in proportion to the count value of the error phoneme. Accordingly, the phoneme model parameter distribution 60 for phoneme N of the current acoustic model will move to the ideal phoneme model parameter distribution 80 for phoneme N. This shifts the phoneme model parameter distribution 60 of the phoneme N, which is the result of error recognition, so that the amount of mutual information with the phoneme M decreases so that an error does not appear by the amount of mutual information. Accordingly, the mutual information amount 90 for the phoneme M of the phoneme N is reduced as indicated by 93. That is, the amount of mutual information is minimized. At this time, the phoneme model parameter distribution 60 of the phoneme N varies its degree of movement according to the magnitude of the log likelihood and the count value of the error phoneme. In other words, the degree of movement of the phoneme model parameter distribution 60 of the phoneme N is larger in the log likelihood value, and the larger the count value of the error phoneme is, the wider the movement is.

As described above, minimizing the amount of mutual information means reducing the probability distribution portion 90 overlapping in the phoneme model parameter distribution of different phonemes. According to this method, it is possible to minimize the mutual information amount 90 of the phoneme M and the phoneme N in the acoustic model. In particular, according to the present invention, the performance of differential learning is improved by updating the model parameter distribution for the phonemes of the correct recognition result, and further updating the model parameter distribution for the phonemes of the incorrect recognition result. Can improve.

Furthermore, according to an embodiment of the present invention, since discriminative learning is performed according to the key word recognition result along with the discriminative learning through the N-best recognition result, the performance of discriminative learning for a specific word, that is, the key word can be improved. have. For example, a key word is a key word that is more frequently used by a specific user, a word that is difficult to pronounce for all users, a word that is difficult to pronounce for a particular user, or more errors that are preset for all users. A word corresponding to at least one of a word generated and a word having more errors than a predetermined reference value may be specified in advance. In this case, it is possible to construct a more efficient acoustic model for all users or for a specific user, compared to simply performing discrimination learning according to the N-best recognition result.

2 is a diagram for explaining a speech recognition system including an apparatus for learning acoustic model discrimination according to an embodiment of the present invention.

Referring to FIG. 2, the speech recognition system includes a feature extractor 100, a searcher 200, an acoustic model database 300, a pronunciation dictionary database 400, a language model database 500, a text database 600, Language model learner 700, speech database 800, keyword detection unit 910, keyword database 912, N-best sorter 920, keyword learner 930, N-best learner 940 The keyword discrimination learning unit 950 and the N-best discrimination learning unit 960 are included.

The feature extractor 100 extracts a feature of the voice signal from the input voice signal. Here, the voice signal may be input through a voice input device or a voice file. The feature extractor 100 performs signal processing to remove noise from an input speech signal or to improve speech recognition performance. Then, the feature extractor 100 extracts the feature vector from the speech signal of the signal-processed speech section and provides it to the searcher 200.

The searcher 200 forms a search space for speech recognition through a sound model stored in the acoustic model database 300, a language model stored in the language model database 500, and a pronunciation dictionary stored in the pronunciation dictionary database 400. Speech recognition is performed using the feature vector obtained by the feature extractor 100 from the formed search space and the input voice.

In an embodiment of the present invention, the searcher 200 may output a similarity value with respect to a previously trained model as a recognition result. The searcher 200 may obtain a lattice type recognition result through voice recognition, and may obtain a N-best recognition result from the lattice type recognition result. To this end, the searcher 200 may use a pattern matching algorithm such as a Viterbi algorithm or Dynamic Time Warping (DTW). For example, the search space may include a search space in the form of a finite state network (FSN) for recognition of small vocabulary such as command recognition and a digit recognition, and a search space in the form of a tree for recognition of large words and rapid recognition. have.

The acoustic model database 300 stores the acoustic model. Here, the acoustic model models the characteristics of the voice signal that changes in time. The acoustic modeling method may exemplify an HMM, a continuous HMM, a neural network (NN), and the like. The acoustic model database 300 according to an embodiment of the present invention may store a phoneme model parameter distribution for each phoneme.

The pronunciation dictionary database 400 stores the pronunciation dictionary. Pronunciation dictionary stores the pronunciation for the voice. The phonetic dictionary stores multiple pronunciations for a particular voice in conjunction with the acoustic model.

Language model database 500 The language model improves recognition rate by weighting recognition candidates in consideration of grammar between words so that sentences matching grammar get higher scores. The search for the optimal recognition word string also reduces the number of candidates to be compared. The language model can be selected in consideration of the number of recognized words, recognition speed, and recognition performance.

The text database 600 stores texts for generating a language model. The language model learner 700 generates or updates a language model through texts stored in the text database 600.

The voice database 800 may store a voice for learning and text (transcription data) for the voice. At this time, the text for the voice may be omitted.

A search space for speech recognition is formed using the acoustic model of the acoustic model database 300, the pronunciation dictionary of the pronunciation dictionary database 400, and the language model of the language model database 500. Is input, the feature extractor 100 extracts the feature vector from the speech signal, and the searcher 200 recognizes the speech using the extracted feature vector, and outputs the N-best recognition result as the recognition result. do.

The keyword detecting unit 910 extracts only a keyword recognition result, which is a recognition result of the keyword included in the word string, by referring to the keyword database 912 from the N-best recognition result, which is a recognition result of the word string output by the searcher 200. .

The key words stored in the key word database 912 are words that a specific user uses more than a preset number, words that are difficult to pronounce for all users, words that are difficult for pronunciation for a specific user, and all users. The word corresponding to at least one of a word having more errors than a preset value and a word having more errors than a predetermined reference value for a specific user are stored. The keyword detecting unit 910 provides the detected keyword recognition result to the keyword learning unit 930.

The keyword learning unit 930 derives a phoneme model parameter distribution of a plurality of phonemes included in the keyword based on the keyword recognition result through the voice stored in the speech database 800. For example, when the phoneme M is input, when it is recognized as M, the phoneme M is determined as a correct answer recognition result, and is called a correct answer phoneme. If the phoneme M is recognized as N when the phoneme M is input, the phoneme N is determined as an error recognition result and is called an error phoneme.

Therefore, the keyword learning unit 930 derives the phoneme model parameter of the correct answer phoneme and the phoneme model parameter of the error phoneme. Next, each of the keyword learning unit 930 and the N-best learning unit 940 delivers the correct answer phoneme and the error phoneme to the keyword learning unit 930 and the N-best learning unit 940.

The keyword discrimination learning unit 950 updates the received phoneme model parameter distribution so that the phoneme model parameter distribution received from the keyword learning unit 930 and the mutual information amount of other phoneme model parameters are minimized. At this time, the keyword discrimination learning unit 950 updates the phoneme model parameter distribution of the correct answer phoneme and the phoneme model parameter distribution of the error phoneme in the keyword recognition result.

Meanwhile, the N-best sorter 920 sorts the N-best recognition result output from the searcher 200 and provides the N-best learner 940.

The N-best learning unit 940 derives a phoneme model parameter distribution of the phonemes according to the N-best recognition result through the voice stored in the voice database 800. At this time, the N-best learner 940 derives a phoneme model parameter of a correct answer phoneme and a phoneme model parameter of an error phoneme. Next, each of the N-best learners 940 transfers the phoneme model parameters of the correct answer phoneme and the error phoneme to the N-best discrimination learner 960.

The N-best discrimination learning unit 960 updates the received phoneme model parameter distribution so that the phoneme model parameter distribution received from the N-best learning unit 940 and the mutual information amount of other phoneme model parameters are minimized. In this case, the N-best discrimination learning unit 960 updates the phoneme model parameter distribution of the correct answer phoneme and the phoneme model parameter distribution of the error phoneme in the N-best recognition result.

As described above, according to an embodiment of the present invention, discrimination learning is performed using not only N-best recognition results but also keyword recognition results extracted from N-best recognition results. Therefore, more accurate and intensive discriminant learning can be performed on key words. Furthermore, in each of the N-best recognition result keyword recognition results, discriminative learning is performed using error phonemes as well as correct phonemes. Therefore, the amount of mutual information can be minimized more effectively in differential learning.

In more detail, the apparatus for performing discriminatory learning using N-best recognition results and the apparatus for performing discriminatory learning using keyword recognition results will be described in detail.

FIG. 3 is a block diagram illustrating an apparatus for performing differential learning on a keyword recognition result in the speech recognition system of FIG. 2.

Referring to FIG. 3, a device necessary for performing discriminative learning on a keyword recognition result is an acoustic model database 300, a speech database 800, a keyword detection unit 910, a keyword database 912, and a keyword learning unit 930. And the keyword discrimination learning unit 950.

First, the keyword detecting unit 910 extracts a keyword recognition result by referring to a keyword stored in the keyword database 912 from each of the N-best recognition results output from the searcher 200. For example, as shown in Table 2, when N = 3, a keyword recognition result as shown in Table 3 is extracted from the N-best recognition result. These key words are those words where a particular user has a higher frequency than a preset number, words that are difficult to pronounce for all users, words that are difficult to pronounce for a particular user, and more errors that are preset for all users. The word may correspond to at least one of a word generated and a word having more errors than a predetermined reference value for a specific user. Here, the keyword recognition result includes a correct answer and an error recognition result. The keyword detecting unit 910 provides the keyword recognition result to the keyword learning unit 930.

The keyword learning unit 930 derives a phoneme model parameter distribution for each phoneme of the keyword. At this time, the keyword learning unit 930 refers to the speech database 800 to derive a phoneme model parameter distribution for a plurality of phonemes included in each keyword. In particular, the key word learning unit 930 derives the phoneme model parameter distribution by dividing each of the correct answer phoneme and the erroneous phoneme. Next, the keyword learning unit 930 provides the keyword discrimination learning unit 950 with a phoneme phonemic model parameter distribution divided into correct answer phonemes and error phonemes.

The keyword discrimination learning unit 950 includes an error phoneme processing module 951 and a correct answer phoneme processing module 953. Accordingly, the phoneme model parameter distribution for the error phoneme having the error recognition result is input to the error phoneme processing module 951. The phoneme model parameter distribution for the correct answer phone with the correct answer recognition result is input to the correct answer phoneme processing module 953.

The error phoneme processing module 951 updates a phoneme model parameter distribution corresponding to the error phoneme in the acoustic model stored in the acoustic model database 300. In this case, the error phoneme processing module 951 applies the log likelihood of the error phoneme as a weight in the keyword recognition result, and updates the corresponding phoneme model parameter distribution in the acoustic model in proportion to the count value of the error phoneme. In this case, the log likelihood is calculated using the maximum log likelihood (ML) estimation method. In addition, the count value of the error phoneme means the number of times that the phoneme is determined as the error phoneme. For example, as illustrated in FIG. 1D, updating the phoneme model parameter distribution corresponding to the error phoneme is not a phoneme M, but a phoneme model parameter distribution 60 of the phoneme N, which is an error phoneme. This is to move in the direction in which the amount of mutual information with the parameter distribution 50 decreases. Accordingly, the phoneme model parameter distribution 60 for phoneme N will move to the ideal phoneme model parameter distribution 80 for phoneme N. In addition, it is determined that the phoneme model parameter distribution 60 for phoneme N is moved in proportion to the magnitude of the log likelihood. In addition, the phoneme model parameter distribution 60 is determined to be moved in proportion to the count value of the error phoneme.

The correct phoneme processing module 953 updates the phoneme model parameter distribution corresponding to the correct phoneme in the acoustic model stored in the acoustic model database 300. In this case, the correct answer phoneme processing module 953 may perform the maximum mutual information amount (MMI) estimation method to update the phoneme model parameter distribution of the acoustic model for the correct answer phoneme in the keyword recognition result, and such an update may be performed. Use the count value. Here, the count value of the correct answer phoneme means the number of times determined as the correct answer phoneme for the phoneme. For example, as illustrated in FIG. 1C, updating the correct phoneme model parameter distribution corresponding to the correct phoneme may result in the phoneme model parameter distribution 50 for phoneme M of the current acoustic model being the ideal phoneme model parameter distribution for phoneme M. It is to move toward 70. In addition, the degree to which the phoneme model parameter distribution 50 for phoneme M of the acoustic model is shifted according to the number of correct phonemes is determined.

FIG. 4 is a block diagram illustrating an apparatus for performing discriminative learning on N-best recognition results in the speech recognition system of FIG. 2.

Referring to FIG. 4, an apparatus for performing discriminative learning on key word recognition results may include an acoustic model database 300, a speech database 800, an N-best sorter 920, and an N-best learner 940. And an N-best discrimination learning unit 960.

First, the N-best sorter 920 sorts each of the N-best recognition results output from the searcher 200. For example, as shown in Table 2, when N = 3, the N-best recognition result is sorted. Here, the N-best recognition result includes a correct answer and an error recognition result. The N-best sorter 920 provides the N-best recognition result to the N-best learner 940.

The N-best learning unit 940 derives a phoneme model parameter distribution for each phoneme of the word string. In this case, the N-best learner 940 derives a phoneme model parameter distribution for a plurality of phonemes included in each keyword by referring to the voice database 800. In particular, the N-best learner 940 derives a phoneme model parameter distribution by dividing each of the correct answer phoneme and the error phoneme. Next, the N-best learner 940 provides the N-best discrimination learner 960 with distributions of phoneme phonemic model parameters divided into correct answer phonemes and error phonemes.

The N-best discrimination learning unit 960 includes an error phoneme processing module 961 and a correct phoneme processing module 963. Accordingly, the phoneme model parameter distribution for the error phoneme having the error recognition result is input to the error phoneme processing module 961. The phoneme model parameter distribution for the correct answer phone with the correct answer recognition result is input to the correct answer phoneme processing module 963.

The error phoneme processing module 951 updates a phoneme model parameter distribution corresponding to the error phoneme in the acoustic model stored in the acoustic model database 300. In this case, the error phoneme processing module 951 applies the log likelihood of the error phoneme as a weight in the keyword recognition result, and updates the corresponding phoneme model parameter distribution in the acoustic model in proportion to the count value of the error phoneme. In this case, the log likelihood is calculated using the maximum log likelihood (ML) estimation method. In addition, the count value of the error phoneme means the number of times that the phoneme is determined as the error phoneme. For example, as illustrated in FIG. 1D, updating the phoneme model parameter distribution corresponding to the error phoneme is not a phoneme M, but a phoneme model parameter distribution 60 of the phoneme N, which is an error phoneme. This is to move in the direction in which the amount of mutual information with the parameter distribution 50 decreases. Accordingly, the phoneme model parameter distribution 60 for phoneme N will move to the ideal phoneme model parameter distribution 80 for phoneme N. In addition, it is determined that the phoneme model parameter distribution 60 for phoneme N is moved in proportion to the magnitude of the log likelihood. In addition, the phoneme model parameter distribution 60 is determined to be moved in proportion to the count value of the error phoneme.

The correct phoneme processing module 963 updates a phoneme model parameter distribution corresponding to the correct phoneme in the acoustic model stored in the acoustic model database 300. In this case, the correct answer phoneme processing module 963 may perform a maximum mutual information amount (MMI) estimation method to update the phoneme model parameter distribution of the acoustic model for the correct answer phoneme in the keyword recognition result, and the update may be performed by the correct answer phoneme. Use the count value. Here, the count value of the correct answer phoneme means the number of times determined as the correct answer phoneme for the phoneme. For example, as illustrated in FIG. 1C, updating the correct phoneme model parameter distribution corresponding to the correct phoneme may result in the phoneme model parameter distribution 50 for phoneme M of the current acoustic model being the ideal phoneme model parameter distribution for phoneme M. It is to move toward 70. In addition, the degree to which the phoneme model parameter distribution 50 for phoneme M of the acoustic model is shifted according to the number of correct phonemes is determined.

5 is a flowchart illustrating a method for learning acoustic model discrimination according to an embodiment of the present invention.

Referring to FIG. 5, when the voice signal is input in step S110, the feature extractor 100 extracts a feature vector of the voice signal and provides the searcher 200 with the feature vector of the voice extracted in step S115.

The searcher 200 performs speech recognition in a search space formed based on an acoustic model, a pronunciation dictionary, and a language model with respect to the speech vector input in operation S120. This speech recognition is made in phoneme units. In addition, the result of speech recognition is the result of recognition of N-best. The recognition result of N-best includes an answer recognition result and an error recognition result. After performing the speech recognition, the searcher 200 provides the keyword detection unit 910 with the recognition result of N-best in operation S125.

The keyword detecting unit 910 receives the recognition result of N-best, and extracts a keyword recognition result based on the keyword stored in the keyword database 912 from the recognition result of N-best in step S130. Here, the key words stored in the key word database 912 are words for which a specific user has a frequency higher than a predetermined number, words designated as difficult to pronounce for all users, words designated as difficult to pronounce for a specific user, and for all users. It may be a word corresponding to at least one of a word in which more errors occur in advance and a word in which more errors occur than a predetermined reference value for a specific user. Like the recognition result of N-best, the keyword recognition result includes the result of correct answer and error. Then, the keyword detecting unit 910 provides the keyword recognition result to the keyword learning unit 930 in step S135.

The keyword learning unit 930 refers to the speech of the speech database 800 according to the keyword recognition result to derive a phoneme model parameter distribution for a plurality of phonemes included in the keyword recognition result. The phoneme model parameter distributions for the derived plurality of phonemes include the phoneme model parameter distributions for the correct answer phoneme and the error phoneme. In operation S140, the keyword learning unit 930 provides the keyword discrimination learning unit 950 with a distribution of phoneme model parameters for the correct answer phoneme and the error phoneme.

Then, the key word discrimination learning unit 950 updates each phoneme model parameter distribution of the phonemes corresponding to the correct answer phoneme and the error phoneme in the acoustic model stored in the acoustic model database 300 in step S145.

In this case, referring to FIG. 1D, the key word discrimination learning unit 950 applies a log likelihood of the error phoneme as a weight and updates the phoneme model parameter distribution of the error phoneme in proportion to the count value of the error phoneme. In this case, the log likelihood may be calculated using a maximum log likelihood (ML) estimation method. In addition, the count value of the error phoneme means the number of times that the phoneme is determined as the error phoneme.

At the same time, referring to FIG. 1C, the keyword discrimination learning unit 950 updates a phoneme model parameter distribution of the correct answer phoneme. At this time, updating the phoneme model parameter distribution of the acoustic model with respect to the correct answer phoneme is performed through a method of estimating the maximum mutual information amount (MMI). In particular, such an update is proportional to the count value of the correct answer phoneme, which means the number of times the corresponding phoneme is determined as the correct answer phoneme.

On the other hand, the key word detector 910 provides the N-best recognition result received from the search unit 200 to the N-best alignment unit 920 in step S150. Then, the N-best alignment unit 920 sorts the N-best recognition results, and provides the N-best recognition results sorted in step S155 to the N-best learning unit 940.

The N-best learning unit 940 derives a phoneme model parameter distribution for a plurality of phonemes included in the N-best recognition result by referring to the stored voice in the voice database 800 according to the N-best recognition result. The phoneme model parameter distributions for the derived plurality of phonemes include the phoneme model parameter distributions for the correct answer phoneme and the error phoneme. Accordingly, the N-best learner 940 provides the phoneme model parameter distribution for the correct answer phoneme and the error phoneme to the N-best discrimination learner 960 in step S160.

Then, the N-best discrimination learning unit 960 updates each phoneme model parameter distribution of phonemes corresponding to the correct answer phoneme and the error phoneme in the acoustic model stored in the acoustic model database 300 in step S165.

In this case, referring to FIG. 1D, the N-best discrimination learning unit 960 applies a log likelihood of the error phoneme as a weight and updates the phoneme model parameter distribution of the error phoneme in proportion to the count value of the error phoneme. do. In this case, the log likelihood may be calculated using a maximum log likelihood (ML) estimation method. In addition, the count value of the error phoneme means the number of times that the phoneme is determined as the error phoneme. At the same time, referring to FIG. 1C, the N-best discrimination learning unit 960 updates a phoneme model parameter distribution of the correct answer phoneme. At this time, updating the phoneme model parameter distribution of the acoustic model with respect to the correct answer phoneme is performed through a method of estimating the maximum mutual information amount (MMI). In particular, such an update is proportional to the count value of the correct answer phoneme, which means the number of times the corresponding phoneme is determined as the correct answer phoneme.

As described above, the acoustic model discrimination learning method according to an embodiment of the present invention may be implemented as computer readable codes on a computer readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination, and includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a compact disk read only memory (CD-ROM), and a digital video disk (ROM), random access memory (RAM), flash memory, and the like, such as a magneto-optical medium such as a magneto-optical medium and a floppy disk, And hardware devices that are specifically configured to perform the functions described herein.

In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers of the technical field to which the present invention belongs.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be apparent to those skilled in the art. In addition, although specific terms are used in the specification and the drawings, they are only used in a general sense to easily explain the technical contents of the present invention and to help the understanding of the present invention, and are not intended to limit the scope of the present invention. Accordingly, the foregoing detailed description is to be considered in all respects illustrative and not restrictive. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

The present invention relates to a device for learning acoustic model discrimination in consideration of phoneme error results in a key word, and a computer readable recording medium having recorded thereon a method therefor. Since the present invention performs discriminative learning according to the keyword recognition result along with the discriminative learning through the N-best recognition result, it is possible to improve the performance of the discriminative learning for a specific word, that is, a keyword. For example, key words are words frequently used by a specific user, words that are difficult to pronounce for all users, words that are difficult to pronounce for a specific user, words that generate more preset errors for all users, and preset words for a specific user. Words that cause more errors than the standard value can be specified in advance. In this case, it is possible to construct a more efficient acoustic model for all users or for a specific user, compared to simply performing discrimination learning according to the N-best recognition result. Furthermore, when the phoneme model parameter distribution of the acoustic model is updated, the present invention applies logarithmic likelihood of the phoneme classified as an error recognition result as a weight, and reflects the error recognition result to minimize the amount of mutual information. The amount of mutual information can be minimized. Such a present invention has industrial applicability because it is not only sufficient marketable or commercially viable, but also realistically repeatable.

100: feature extraction unit 200: search unit
300: acoustic model database 400: pronunciation dictionary database
500: language model database 600: text database
700: language model training unit 800: voice database
910: keyword extraction unit 912: keyword database
920: N-best alignment unit 930: keyword learning unit
940: N-best learning unit 950: Key word discrimination learning unit
960: N-best discriminant learning unit

Claims (8)

A keyword detection unit for extracting a keyword recognition result that is a speech recognition result for the keyword from an N-best recognition result that is a speech recognition result for a word string including a keyword;
A keyword learning unit for deriving a phoneme model parameter distribution of a correct answer phoneme and an error phoneme according to the keyword recognition result; And
A keyword discriminating learning unit applying a log likelihood of an error phoneme as a weight in the keyword recognition result, and updating a phoneme model parameter distribution of the error phoneme to minimize the amount of mutual information in proportion to the count value of the error phoneme; Apparatus for discriminating learning acoustic models, characterized in that. The method of claim 1,
Further comprising a keyword database for storing the keyword,
The key words are words in which a specific user uses a frequency higher than a predetermined number, words designated as difficult to pronounce for all users, words designated as difficult to pronounce for a specific user, and more errors than preset values in all users. And a word corresponding to at least one of a word and a word having more errors than a predetermined reference value for a specific user. The method of claim 1,
An N-best learner for deriving a phoneme model parameter distribution based on the N-best recognition result; And
And an N-best discrimination learning unit for updating the phoneme model parameter distribution so that the amount of mutual information is minimized. The method of claim 1,
The key word discrimination learning unit
Apparatus for discriminating acoustic models, characterized in that for calculating the log likelihood using a maximum likelihood (ML) estimation method. The method of claim 1,
The key word discrimination learning unit
Apparatus for acoustic model discrimination learning, by applying a log likelihood of the correct answer phoneme as a weight, updating a phoneme model parameter distribution for the correct answer phoneme to minimize the amount of mutual information in proportion to the count value of the correct answer phoneme . The method of claim 5,
The key word discrimination learning unit
And a phoneme model parameter distribution for the correct answer phoneme using a method of estimating a maximum mutual information (MMI). Extracting a key word recognition result, which is a voice recognition result of the key word, from an N-best recognition result, which is a voice recognition result of a word string including a key word;
Deriving a phoneme model parameter distribution of a correct answer phoneme and an error phoneme according to the keyword recognition result; And
Applying a log likelihood of an error phoneme as a weight in the keyword recognition result, and updating a phoneme model parameter distribution of the error phoneme to minimize the amount of mutual information in proportion to the count value of the error phoneme; Computer-readable recording medium having recorded thereon a method for learning. The method of claim 7, wherein
Deriving a phoneme model parameter distribution according to the N-best recognition result; And
And updating a phoneme model parameter distribution according to the N-best recognition result to minimize the amount of mutual information. The computer-readable recording medium having recorded thereon a method for discriminating acoustic models.

Images