KR20180062003A - Method of correcting speech recognition errors - Google Patents

Abstract

A method of correcting speech recognition errors is disclosed. The method comprises: a step of determining whether a current vocabulary is an error vocabulary which has a speech recognition error, using a distance value between the position of a current vocabulary mapped to a vector space and the position of a previous vocabulary according to a word embedding technique; a step of generating a corrected vocabulary candidate having a pronunciation similar to the determined pronunciation of the error vocabulary by referring to a vocabulary pronunciation dictionary if the current vocabulary is determined to be an error vocabulary; and a step of determining a corrective vocabulary formed by restoring the error vocabulary from the generated corrective vocabulary candidate, by referring to a general domain-based language model and a specific domain-based language model.

Description

{METHOD OF CORRECTING SPEECH RECOGNITION ERRORS}

The present invention relates to a speech recognition error correction method based on a uttered context, and more particularly, to a method for correcting a speech recognition error in a speech recognition based application.

In recent years, various types of application technologies combining speech recognition technology and language processing technology have been developed and applied to various fields such as automatic translation, automatic interpretation, simultaneous interpretation, and image interpretation.

Speech recognition technology generates speech recognition errors for various reasons, and such speech recognition errors amplify errors in various application modules that use speech recognition results as input. Therefore, there has been a great deal of effort to correct speech recognition errors.

In a conventional method for correcting a speech recognition error, after the vocabularies constituting the speech-recognized result sentence are tagged to check the part-of-speech, if the part-of-speech string including the specific part-of-speech is not general, , Candidate vocabularies (similar-pronounced vocabularies) having a similar pronunciation with respect to an error vocabulary judged as having an error, selecting the most probable vocabulary among the selected candidate vocabularies, In order to correct the speech recognition error.

However, since most of the conventional speech recognition technologies use a large-capacity language model, it is difficult to determine whether the speech recognition error includes a specific speech related to a specific sentence. That is, even if the specific sentence includes a speech recognition error, it is difficult to correct the error accurately since there are many cases where there is no error in terms of semantics.

In addition, in the conventional speech recognition technology, due to the inaccuracy of the morpheme tagging, when a wrong vocabulary is incorrectly tagged with incorrect speech recognition information, the vocabulary without a speech recognition error is erroneously recognized as an erroneous vocabulary and corrected Frequent cases.

Accordingly, an object of the present invention is to provide a speech recognition error correction method capable of detecting and correcting an error vocabulary in which a speech recognition error occurs in a speech recognition sentence associated with a specific subject.

According to another aspect of the present invention, there is provided a method for correcting a speech recognition error, the method comprising the steps of: using a distance value between a position of a current vocabulary mapped in a vector space and a position of a previous vocabulary according to a word- Determining whether a speech recognition error is an error vocabulary; If the current vocabulary is determined to be an erroneous vocabulary, generating a corrected vocabulary candidate having a pronunciation similar to the determined pronunciation of the erroneous vocabulary by referring to the vocabulary pronunciation dictionary; And determining a corrective vocabulary in which the erroneous vocabulary is restored from the generated corrective vocabulary candidate by referring to a general domain-based language model and a specific domain-based language model.

According to the present invention, a specific context window is defined, and various contextual information within a defined context window is used to efficiently detect speech recognition errors. These context windows can be defined variously depending on the application field to which they are applied. For example, when applied to a lecture interpreter or an image interpreter, the size of the context window is defined as the size of the context window from the start of the utterance to a specific time point, and this size increases until the lecture or video conversation ends. That is, the present invention can more accurately correct speech recognition errors by utilizing characteristics such as semantic similarity between vocabularies that can be found in a specific context window, and repeated speech of the same vocabulary. Further, the present invention can be extended to various speech and language processing apparatuses such as lecture interpreting, video conversation translation, image interpretation, and the like.

1 is a hardware block diagram for implementing a speech recognition apparatus according to an embodiment of the present invention.
2 is a functional block diagram of the processor shown in Fig.
3 is a functional block diagram of the speech recognition error correction unit shown in FIG.
4 is a flowchart illustrating a speech recognition error correction method according to an embodiment of the present invention.

Best Mode for Carrying Out the Invention Various embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiments of the present invention are capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and the detailed description is described with reference to the drawings. It should be understood, however, that the embodiments of the invention are not intended to be limited to the specific embodiments, but include all changes and / or equivalents and alternatives falling within the spirit and scope of the various embodiments of the invention. In connection with the description of the drawings, like reference numerals have been used for like elements.

Terms such as those defined in commonly used dictionaries should be interpreted to have the meanings consistent with the contextual meanings of the related art and, unless expressly defined in the various embodiments of the present invention, It is not interpreted as meaning.

In the conventional speech recognition error correction method, speech recognition error is corrected by utilizing part-of-speech context information. In recent speech recognition technology, speech recognition is performed using a large-capacity language model. Because the error does not look like an error, there is a limit to how it can not detect or correct speech recognition errors.

Accordingly, the present invention provides a method for efficiently detecting and correcting a speech recognition error by defining a context window and utilizing various context information in the context window.

These context windows can be defined variously depending on the application field to which they are applied. For example, when applied to a lecture interpreter or an image interpreter, the size of the context window is defined as the size of the context window from the start of the utterance to a specific time, and the size of the context window increases until the lecture or video conversation ends. That is, in the present invention, it is possible to provide an advantage that the speech recognition errors can be corrected more accurately by utilizing characteristics such as semantic similarity between spoken sentence vocabularies that can be found in a specific context window and repeated speech of the same vocabulary have.

On the other hand, the speech recognition apparatus of the present invention which can more accurately correct speech recognition errors can be implemented in various electronic apparatuses or various electronic apparatuses.

The electronic device may be, for example, a robot device having artificial intelligence, or a user terminal or a server having a communication function. The user terminal may be, for example, a smartphone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, a desktop personal computer, A laptop personal computer, a netbook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device Examples may be head-mounted-devices (HMD) such as electronic glasses, electronic apparel, electronic bracelets, electronic necklaces, electronic apps, or smart watches.

1 is a block diagram of a speech recognition apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 1, a speech recognition apparatus 100 according to an embodiment of the present invention detects a speech recognition error, corrects a detected speech recognition error, and outputs a speech recognition result, 200).

The application apparatus 200 may be any kind of apparatus that provides interpretation / translation services such as automatic translation, automatic translation, simultaneous interpretation, image interpretation, lecture interpretation, video conversation translation,

When the speech recognition apparatus 100 is applied to an electronic apparatus, the speech recognition apparatus 100 includes one or more processors 110, a memory 130, a user input apparatus 140, A user output device 150, and a storage 160, each of which may communicate via the system bus 120. [ In addition, the speech recognition apparatus 100 may include a network interface connected to a network.

The processor 110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 130 and / or the storage 160.

The memory 130 and the storage 160 may include volatile or non-volatile storage media. For example, the memory 130 may include a ROM 131 and a RAM 133.

Further, the speech recognition error correction method performed in the speech recognition apparatus 100 according to the embodiment of the present invention may be implemented as a non-temporary computer-readable medium having computer-executable instructions. In one embodiment, when the speech recognition error correction method is executed by the processor 110, the computer readable instructions may perform the method according to at least one aspect of the present invention.

2 is a functional block diagram of the processor shown in FIG.

Referring to FIG. 2, the processor 110 may include a speech recognition unit 112 and a speech recognition error correction unit 114. The speech recognition unit 112 and the speech recognition error correction unit 114 may be implemented as logic and mounted within the processor 110.

The speech recognition unit 112 refers to the speech recognition learning model (not shown), recognizes the speech uttered by the user, and generates a sentence in the form of a text corresponding to the speech recognition result Word sequence) to the speech recognition calibration unit 114. Here, the speech recognition learning model may be, for example, an acoustic model, a language model of a large capacity, and a pronunciation model, May be stored in storage 160 of Figure 1 to refer to the model.

The speech recognition error correcting unit 114 includes a word position database 114-7 (shown in FIG. 3), a spoken word database 114-9 (shown in FIG. 3) that are continuously updated in the size of the defined context window, And a vocabulary pronunciation dictionary 130 (shown in FIG. 3), and the like.

The context window size may vary according to an application device 200 (shown in FIG. 1) linked to the speech recognition apparatus 100 of the present invention.

For example, when the speech recognition apparatus 100 is interworked with an application apparatus 200 (shown in FIG. 1) for processing an automatic interpretation or dialog based on a general dialogue, (N is a natural number of 1 or more) sentences before the sentence of the sentence.

As another example, when the speech recognition apparatus 100 is interworked with an application apparatus 200 (shown in FIG. 1) that processes image interpretation, lecture interpreting, etc., the context window size is changed from the start point of the video conference or lecture to the utterance Lt; RTI ID = 0.0 > sentences < / RTI >

3 is a functional block diagram of the speech recognition error correction unit shown in FIG.

3, the speech recognition error correction unit 114 includes an error word determination unit 114-1, a correction word candidate generation unit 114-3, a correction word determination unit 114-5, (114-7), the lexical location information DB 114-9, and a specific domain-based language model DB 114-11. The DBs 114-7, 114-9, and 114-11 may be stored in the storage 160 of FIG.

Error vocabulary The decision unit (114-1)

The DBs 114-7, 114-9, and 114-11 may be DBs used to detect speech recognition errors. Further, in order to detect the speech recognition error, a vocabulary pronunciation dictionary 130 and an n-gram information DB 120 may be further used, and the parts-of-speech information DB 120 and the vocabulary- The pronunciation dictionary 130 may also be stored in the repository 160 of FIG.

The DBs 114 - 7, 114 - 9, and 114 - 11 may be updated in real time whenever a sentence recognized by the speech recognition unit 112 is input. Alternatively, the part-of-speech information DB 120 and the vocabulary pronunciation dictionary 130 may be DBs constructed in advance.

Concretely, vocabularies included in the speech-recognized sentence can be stored in the speech-based vocabulary DB 114-7 in real time.

In the lexical position information DB 114-9, position information (or a vector value) in which vocabularies included in the speech-recognized sentence are mapped (or projected) in a word space can be stored in real time. Alternatively, the vocabulary location information DB 114-9 may include a lexical class clustered into vocabularies that are semantically related to the vocabularies included in the speech-recognized sentence, Information (vector value) can be stored in real time. The word space is defined as a vector space in which each vocabulary in the speech-recognized sentence is to be mapped (or projected) according to a word embedding technique. The word-embedding technique is a technique derived from a neural network language model, And is arranged close to the vector space to express the lexical meaning.

The language modeling results for the specific domain-related vocabularies within the size of the specific context window among the vocabularies included in the speech-recognized sentence can be stored in real time in the specific domain-based language model DB 114-11 . The language modeling result may be n-gram information about vocabularies related to the specific domain. Here, the specific domain may be a lecture (online lecture, real-time lecture) related to a specific topic, a video conference, and the like.

Unlike the DBs 114-7, 114-9, and 114-11, the part-of-speech information (n-gram) information DB 120 that is constructed in advance stores n-gram information Wherein the n-gram information defines a probability of a word to be displayed next from n-1 words in the past, and the n-gram is, for example, a combination of a bi-gram (n = 2) Grams (n = 3).

Unlike the DBs 114-7, 114-9, and 114-11, pronunciation symbol information for each of the vocabularies in the speech-recognized sentence can be stored in the vocabulary pronunciation dictionary 130 previously constructed .

The error lexical decision unit 114-1 determines whether or not the lexical pronunciation dictionary 130 and the vocabulary pronunciation dictionary 130 correspond to the DBs 114-7, 114-9, and 114-11, the n-gram information DB 120, , It is possible to determine an error vocabulary having a possibility of speech recognition error among the vocabularies in the speech recognized sentence.

In order to determine an error vocabulary having a possibility of the speech recognition error, the error word determination unit 114-1 calculates a speech recognition error value obtained by quantifying the possibility of speech recognition error for each of the vocabularies, And when the speech recognition error value exceeds the threshold value, the target vocabulary is determined as an error vocabulary having a high possibility of speech recognition error.

The speech recognition error value E can be calculated by the following equation.

The variable A is used to calculate a difference between the position information (vector value) of the vocabulary recognized at the current point in time in the vector space (hereinafter referred to as a 'current vocabulary') and the position information (vector value) of the vocabulary class clustered up to the previous point Is a distance value (A), and the variable w1 is a weight assigned to the A. The error word determination unit 114-1 may calculate the distance value A by referring to the lexical positional information DB 114-9. In the field where speech recognition is applied, lectures and videoconferences have a specific domain (a specific topic), so vocabulary uttered by a speaker is semantically related to each other and is mapped to a relatively short distance in the word space. For this reason, the greater the distance between the vocabulary at a certain point and the vocabulary class that has been uttered until now, the greater the possibility of error.

The variable B is the pronunciation similarity between the current vocabulary and the vocabulary recognized at the previous time (hereinafter referred to as the previous vocabularies), and the variable w2 is a weight assigned to the B. The error word determination unit 114-1 may calculate the similarity between the phonetic symbols of the current vocabulary and the pronunciation symbols of the previous vocabularies by a stochastic method by referring to the vocabulary pronunciation dictionary 130. [ In conferences or lectures on the same subject, the speaker tends to repeatedly utter vocabulary that is often used on the same topic. Therefore, if there are vocabularies similar in pronunciation among the vocabulary collected in the vocabulary DB 207, at least one vocabulary among similar vocabulary words is likely to be an incorrect vocabulary recognized as a false vocabulary. For example, a vocabulary called 'learning' meaning 'learning' and a vocabulary 'running' meaning 'running' have similar pronunciations to each other although they have different meanings. If these vocabularies exist in the spoken-word DB 207, a sentence including any one of these vocabularies may mis-recognize the 'learning' as 'running', or conversely, the 'running' It is more likely to be the result of speech recognition. In this respect, the pronunciation similarity between the current vocabulary and the previous vocabulary is an important parameter in determining the speech recognition error.

The variable C is an n-gram probability value, and the variable w3 is a weight assigned to the B. The error word determination unit 114-1 may calculate an n-gram probability value for the vocabulary by referring to the speech phrase database 120. [ The words constituting the error vocabulary will have a smaller probability of the next word to appear from the past n-1 words. That is, the frequency of error vocabulary that will appear next from the past n-1 vocabularies will be found at a relatively low frequency. In this regard, the n-gram probability value (C) can also be an important parameter for detecting speech recognition errors.

As described above, according to an embodiment of the present invention, the possibility of speech recognition error is detected by comparing the position information (vector value) of the vocabulary recognized as speech at the current point in the vector space (hereinafter referred to as the 'current vocabulary' (A) between the position information (vector value) of the vocabulary class, the pronunciation similarity degree (B) between the current vocabulary and the vocabulary recognized at the previous time (hereinafter referred to as the previous vocabularies) -gram) probability value (C) on the basis of the three main factors can be judged whether the word is an error vocabulary. The weights w1, w2 and w3 assigned to each variable can be determined heuristically.

Meanwhile, in the present embodiment, an example of judging the possibility of speech recognition error of a vocabulary recognized by considering all three main factors is described, but without considering all three main factors, one or two major It is possible to determine the possibility of speech recognition error by considering only the element.

Candidate for correctional vocabulary Generating unit (114-3)

The corrective vocabulary candidate generating unit 114-3 refers to the vocabulary pronunciation dictionary 130 about the error vocabulary determined to have a speech recognition error by the error vocabulary determination unit 114-1, Vocabulary candidate). That is, the calibration lexical decision unit 114-5 searches the vocabulary pronunciation dictionary 130 for vocabulary words having pronunciation similar to the pronunciation of the error vocabulary, and generates the retrieved vocabularies as the corrected vocabulary candidates.

Corrective vocabulary The decision unit (114-5)

The calibration lexical decision unit 114-5 refers to the general domain based language model 140 and furthermore a specific domain based specific language model 114-11 to generate the corrected lexical word candidate generator 114 3) to determine the corrective vocabulary from the corrective vocabulary candidates. Thus, by using only the general domain-based language model 140, it is possible to solve the conventional problem that it is difficult to determine whether a specific sentence associated with a specific topic includes a speech recognition error.

As is well known, the generic domain-based language model 140 is a model that is learned from a large-capacity corpus and can be used to select vocabularies associated with a general topic as the most appropriate calibration vocabulary for a given corrective vocabulary candidate (E.g., conditional probability, score, word count, n-gram model data, frequency data, context frequency, etc.).

A specific domain-based language model 114-11 may be generated from the language model generation unit 114-13 and may be used to rank word candidates associated with a particular topic handled in a video conference and lecture, (E.g., conditional probability, score, word count, n-gram model data, frequency data, context frequency, etc.) that can be used to select the most appropriate calibration vocabulary within the candidate Lt; / RTI >

The calibration lexical decision unit 114-5 may determine a calibration lexical decision index to determine a calibration vocabulary from the corrected vocabulary candidates through the following Equation (2).

Here, the context frequency 1 is a context frequency for each corrective vocabulary included in the corrective vocabulary candidate with reference to the specific domain-based language model 114-11, and the context frequency 2 is a language model based on the general domain- Is the context frequency for each corrective vocabulary included in the corrective vocabulary candidate, with reference to equation (130). W4 is a weight given to calculate a calibration lexical decision index, wherein the context frequency based on the specific domain-based language model (114-11) is calculated based on the characteristics of the videoconference or lecture, May be set higher than the context frequency based on the model (140).

When the most appropriate corrective vocabulary is determined according to the above-mentioned Equation 2, the error vocabulary is restored to the determined corrective vocabulary and input to the application apparatus 200 (shown in FIG. 1) do.

On the other hand, the context frequency in Equation (2) is replaced with terms such as probability, conditional probability, score, word count, n-gram model data, and frequency data that can be used to select the most appropriate calibration vocabulary in the corrected vocabulary candidate .

FIG. 4 is a flowchart illustrating a method of correcting a speech recognition error according to an embodiment of the present invention, and a description overlapping with those described in FIGS. 1 to 3 will be briefly described or omitted.

Referring to FIG. 4, in step S410, a process of performing speech recognition on a speech of a speaking person performed in a video conference and a lecture related to a specific topic is performed.

In step S420, a determination is made as to whether the vocabulary recognized according to the result of speech recognition is an error vocabulary in which a speech recognition error occurs. The determination process can be based on a result of comparing the threshold value with the speech recognition error value E of the fish vocabulary calculated according to Equation (1).

If it is determined in step S430 that the recognized vocabulary is an error vocabulary, the process proceeds to step S440. If it is determined that the recognized vocabulary is not an error vocabulary, the process returns to step S410 to repeat steps S410 to S420.

If the vocabulary recognized as a speech vocabulary is recognized as an error vocabulary, a process of generating a corrected vocabulary candidate (or correct answer vocabulary candidate) is performed by referring to the vocabulary pronunciation dictionary 130 for the error vocabulary in step S440.

Next, in step S450, referring to the general domain-based language model 140 and the language model 114-11 based on a specific domain (specific subject), a process of determining a calibration vocabulary among the generated corrected vocabulary candidates Thereby completing a series of all processes related to the speech recognition error correction method.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood that various modifications and applications not illustrated in the drawings are possible. For example, each component specifically shown in the embodiments of the present invention can be modified and implemented. It is to be understood that all changes and modifications that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (1)

Determining whether the current vocabulary is an error vocabulary in which a speech recognition error occurs using a distance value between a position of a current vocabulary mapped in a vector space and a position of a previous vocabulary according to a word embedding technique;
If the current vocabulary is determined to be an erroneous vocabulary, generating a corrected vocabulary candidate having a pronunciation similar to the determined pronunciation of the erroneous vocabulary by referring to the vocabulary pronunciation dictionary; And
Determining a corrected vocabulary in which the erroneous vocabulary is restored in the generated corrected vocabulary candidate by referring to a general domain-based language model and a specific domain-based language model
And a speech recognition error correction method.

Images