JP2014149490A - Voice recognition error correction device and program of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice recognition error correction device in which correction work is easy and which can estimate a correct correction word string with high accuracy.SOLUTION: A voice recognition error correction device 20 includes: correction instruction input means 21 for inputting a correction instruction to a voice recognition error; voice recognition error part recognition means 22 for acquiring a correction word string candidate and a recognition score by voice recognition or by handwritten character recognition; recognition hypothesis integration means 26 for integrating a branch with which the correction word string candidate and the recognition score are associated into the voice recognition error part of a hypothesis lattice input from a voice recognition device 10; and hypothesis re-scoring means 27 for re-scoring the integrated score by using an acoustic score and the recognition score, estimating the combination of branches in which the re-scored integrated score becomes maximum as the correct correction word string and correcting the voice recognition error part by the estimated correction word string.

Description

  The present invention relates to a speech recognition error correcting apparatus and a program for correcting a speech recognition error included in a word string indicating a speech recognition result of program sound.

  Conventionally, voice recognition technology has been used for the production of captions for broadcast programs. Since the speech recognition result includes a recognition error, a corrector (operator) that corrects the recognition error is arranged and a character string in which the recognition error is corrected is broadcast as subtitles.

  As a conventional technique for correcting this recognition error, an invention has been proposed in which two pairs of a person who points to a recognition error and a person who corrects the pointed word are corrected in several sets (Patent Document 1). In addition, an invention has been proposed in which a sentence as a recognition result is shared by one to several persons and an error in the sentence in charge is corrected without dividing roles as in the prior art (Patent Document 2). In the inventions described in Patent Documents 1 and 2, by using a touch panel, an error part of a recognition result displayed on the screen is specified by touching, and if necessary, correction is made according to the type of recognition error. The character string for inputting is input using the keyboard.

  Since the operation for the correction is performed between the touch on the screen and the keyboard operation, it is difficult to perform a quick correction operation unless the user has mastered not only the correction procedure but also its movement. Then, the invention for practicing this operation is proposed (patent document 3). Furthermore, the character input necessary to correct substitution errors and omission errors is often input using a standard keyboard, and the corrector not only gets used to the correction operation described above, but also quickly enters the keyboard. There is a need for technology that can input Japanese.

  In addition, to reduce the input burden on the keyboard, a correction palette for homonyms is presented, a palette is presented to present the manuscript related to the recognition target, and character strings on the palette are used to correct recognition errors. An invention has been proposed (Patent Document 4). However, the invention described in Patent Document 4 cannot cover all recognition errors.

  In addition, a method for correcting the error part by having the re-speaker speak again in the lispeaking subtitle production has also been proposed. In this case, the corrector must edit the recognition result appropriately so that the character string obtained is inserted at the appropriate place, and has a skilled skill to perform this work in cooperation with the re-speaker. Is required.

  Here, in order to efficiently obtain a necessary correction character string, an invention has been proposed in which a correction section or a vicinity of a correction section is reevaluated using a fixed section in which correction is not specified as a constraint condition (Patent Document 5). There has also been proposed an invention in which a section that needs to be corrected is recognized again using a detailed user dictionary (Patent Documents 6 and 7). In the inventions described in Patent Documents 6 and 7, it is difficult to correct with high accuracy when the recognition target speech itself is unclear or an erroneous utterance.

  Also, an invention is provided in which a correction character string is input from a keyboard, and the correction section of the hypothesis lattice obtained from the recognition result is constrained by the correction character string and recognized again, thereby automatically correcting recognition errors other than the correction section. It has been proposed (Patent Document 8). In the invention described in Patent Document 8, the burden of inputting a character string from the keyboard still remains.

  In addition, an invention has also been proposed in which a corrected portion is recognized by recognizing the voice restated by the corrector to identify a corrected portion (Patent Document 9). In the invention described in Patent Document 9, even if a correction location is specified by voice, it is necessary to input a correction character string using a keyboard, and correction work cannot be easily performed. Therefore, the corrector rephrases the entire sentence containing the error, and at that time, emphasizes the recognition error part and utters, identifies the correction part, and replaces that part with the re-recognized speech recognition result. Has been proposed (Patent Document 10). Furthermore, an invention has also been proposed in which a corrector presents an exhaustive correction candidate using a voice in which a misrecognized portion is restated, and the corrector selects a desired correction candidate (Patent Document 11). Furthermore, the invention which complements the input of handwritten characters using the speech recognition result has also been proposed (Patent Document 12).

JP 2001-60192 A Japanese Patent No. 3998615 JP 2004-240234 A Japanese Patent No. 3986209 Japanese Patent No. 4709987 Japanese Patent No. 4902617 JP 2000-187497 A JP 2011-197410 A Japanese Patent No. 4784120 JP 2003-316386 A JP 2001-92493 A JP 2007-18290 A

  However, in the inventions described in Patent Documents 10 and 11, the rephrased speech is used for correcting the speech recognition error part. However, if only the restated speech is used for speech recognition, a recognition error is included. Therefore, it is impossible to estimate a correct corrected word string with high accuracy, and an additional input from the corrector is required. Here, it is conceivable to optimize the acoustic model used for speech recognition of the speech recognition error part for the corrector. In this case, the speech recognition result of the speech recognition error part is different in tendency of error from the recognition result of the program speech using the acoustic model for unspecified speakers.

  Further, in the invention described in Patent Document 12, the speech recognition result and the handwritten character recognition result do not work complementarily, but only assist the input of the handwritten character. Since the corrector's handwritten character recognition result is different in modal (mode) from speech recognition, the program speech recognition result and the error tendency are different.

  From the above, if the speech recognition result of the speech recognition error part having different error tendency and the recognition result of the handwritten character are complementarily integrated with the recognition result of the program sound, a correct corrected word string can be estimated with high accuracy. it can.

  Therefore, an object of the present invention is to provide a speech recognition error correction apparatus and a program thereof that can be easily corrected and can accurately estimate a correct correction word string.

  In view of the above-described problems, the speech recognition error correction apparatus according to the first invention of the present application corrects a speech recognition error included in a word string indicating a speech recognition result of program sound with a correct correction word string. The speech recognition error part recognition means, hypothesis lattice integration means, and speech recognition error part correction means are provided.

  According to such a configuration, in the speech recognition error correcting device, the corrector manually selects either speech recognition or handwritten character recognition depending on the reason why the speech recognition of the program sound is incorrect. For example, in the case of an acoustic reason such as an unclear utterance of a program sound or a wrong word, voice correction is selected because speech correction is selected and an utterance of a voice recognition error portion is input. In addition, in the case of a homonym, the handwritten character recognition is selected and the handwritten character of the voice recognition error portion is input because correction can be performed more quickly and accurately.

  Further, the speech recognition error correcting device recognizes the speech of the speech recognition error part or recognizes the handwritten character of the speech recognition error part by the speech recognition error part recognition means, and corrects a correct word that is a candidate for a correct correction word string A column candidate and a recognition score for each corrected word string candidate are obtained.

  Here, the voice recognition and the handwritten character recognition by the corrector are different in the tendency of error from the voice recognition of the program voice. Therefore, the speech recognition error correction device uses the hypothesis lattice integration unit to link the corrected word string candidate and the recognition score to the nodes of the branches located at the start point and end point of the speech recognition error part of the input hypothesis lattice. Connect hypotheses lattices by connecting.

  This hypothesis lattice is composed of a branch in which each word evaluated in the speech recognition of the program sound and the acoustic score of each word are associated, and a node of the branch indicating the position of each word. Represents the evaluation contents of.

  Further, the speech recognition error correction device uses the acoustic score and the recognition score to calculate the integrated score for each branch path from the start point to the end point in the speech recognition error part of the integrated hypothesis lattice by the speech recognition error part correction unit. The branch path with the highest calculated integrated score is estimated as a correct corrected word string.

Moreover, the speech recognition error correction apparatus according to the second invention of the present application is characterized in that speech recognition is performed using an acoustic model for a specific speaker unique to the corrector.
According to such a configuration, the speech recognition error correction device can accurately recognize the corrector's utterance and estimate a correct corrected word string with higher accuracy.

The speech recognition error correction device according to the third invention of the present application is characterized in that the speech recognition error partial correction means calculates the weighted sum of the acoustic score and the recognition score as an integrated score for each branch path.
According to such a configuration, the speech recognition error correction device can accurately calculate the integrated score by the weighted sum, and can estimate a correct corrected word string with higher accuracy.

Further, the speech recognition error correction device according to the fourth invention of the present application is the recognition score calculated by the speech recognition error partial correction means for each branch path by the posterior probability of the acoustic score and the logarithmic likelihood calculation formula set in advance. The total with the posterior probability is calculated as an integrated score.
According to such a configuration, the speech recognition error correction apparatus can accurately calculate the integrated score using the log likelihood calculation formula, and can estimate a correct corrected word string with higher accuracy.

  Further, the speech recognition error correction device according to the fifth invention of the present application is such that the speech recognition error partial recognition means uses a pronunciation dictionary or speech recognition used for speech recognition of program speech when all the results of handwritten character recognition are hiragana or katakana. A phoneme string word corresponding to hiragana or katakana notation is read out from a pronunciation dictionary used for speech recognition of an erroneous part, and is used as a corrected word string candidate.

  According to such a configuration, the speech recognition error correcting apparatus treats all words of the phoneme string corresponding to the hiragana or katakana notation as the corrected word string candidates including the kanji notation corresponding to the hiragana or katakana. As a result, even when the corrector cannot immediately remember the kanji notation of the voice recognition error portion, the voice recognition error correction device can input the voice recognition error portion in hiragana or katakana and quickly correct it.

  The speech recognition error correction apparatus according to the first invention of the present application is for causing a computer having hardware resources such as a CPU (Central Processing Unit) and storage means (for example, a memory and a hard disk) to cooperate as the above-described means. The voice recognition error correction program can also be realized (the sixth invention of the present application). This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

According to the present invention, the following excellent effects can be obtained.
According to the first and sixth inventions of the present invention, the result of speech recognition of the corrector's utterance or recognition of the corrector's handwritten character and the hypothesis lattice are complementarily integrated, and an integrated score is calculated from the integrated hypothesis lattice. A correct corrected word string can be estimated with high accuracy. Furthermore, according to the first and sixth inventions of the present application, it is not necessary for the corrector to use the keyboard, and the correction work can be easily performed.

According to the second invention of the present application, it is possible to accurately recognize a corrector's utterance and to estimate a correct corrected word string with higher accuracy.
According to the third and fourth aspects of the present invention, it is possible to accurately calculate an integrated score and estimate a correct corrected word string with higher accuracy.
According to the fifth aspect of the present invention, it is possible to quickly correct a speech recognition error even when the corrector cannot immediately remember the kanji notation.

It is a block diagram which shows the structure of the speech recognition error correction apparatus which concerns on embodiment of this invention. It is explanatory drawing explaining the integration | stacking of the hypothesis lattice in the recognition hypothesis integration means of FIG. 1, (a) shows an example of the word used for correction | amendment of a speech recognition error, (b) is a correction word sequence candidate and recognition score. An example of a branch and a hypothesis lattice associated with each other is shown. 2A and 2B are explanatory diagrams for explaining the integration of hypothesis lattices by the recognition hypothesis integration unit of FIG. 1, in which FIG. 1A shows an integrated hypothesis lattice and FIG. 1B shows a recognition score associated with a branch path; It is a flowchart which shows operation | movement of the speech recognition error correction apparatus of FIG.

[Outline of caption generation system]
With reference to FIG. 1, the outline of the caption generation system 1 according to the embodiment of the present invention will be described in detail.
The caption generation system 1 recognizes program audio and corrects the audio recognition error by a corrector when there is an error in the audio recognition result. As shown in FIG. 1, the caption generation system 1 includes a speech recognition device 10, a speech recognition error correction device 20, and a display device 30.

  The voice recognition device 10 receives a program voice that is the voice of a broadcast program, and generates a word string (speech recognition result word string) indicating a voice recognition result by voice recognition of the input program voice. The speech recognition means 11, the acoustic model 13, the language model 15, and the pronunciation dictionary 17 are provided.

  The speech recognition means 11 recognizes program speech by using an acoustic model 13, language model 15, and pronunciation dictionary 17, which will be described later, a speech recognition result word string (maximum likelihood word string), and a lattice of recognition hypotheses. Are generated. For example, the speech recognition means 11 evaluates the program speech using a statistical model (acoustic model 13) that shows the acoustic features of phonemes that appear in each word, and shows a statistical model that indicates the ease of connection between words (words). Using a language model 15), a speech recognition method for evaluating the likelihood of a Japanese sentence as a recognition result is used.

Here, the voice recognition unit 11 outputs the voice recognition result word string generated by the voice recognition to the voice recognition error correction device 20 and the display device 30. Further, the speech recognition unit 11 outputs a hypothesis lattice indicating the evaluation result of speech recognition to the speech recognition error correction device 20.
Details of the hypothesis lattice will be described later (FIG. 2).

The acoustic model 13 is a statistical model that shows the acoustic features of phonemes that appear in each word, such as a hidden Markov model (HMM).
The language model 15 is a statistical model indicating ease of connection between words, such as bigram or trigram.
The pronunciation dictionary 17 is, for example, a pronunciation model that indicates what phoneme sequence each word becomes, and associates the acoustic model 13 and the language model 15 with each other.

In the speech recognition error correction device 20, when the speech recognition result word string input from the speech recognition device 10 is incorrect, the corrector corrects the error in the speech recognition result word sequence. The speech recognition error correction device 20 outputs the speech recognition result word string with the corrected error as a caption to, for example, a broadcast transmission device (not shown).
The display device 30 is a display that displays the speech recognition result word string input from the speech recognition device 10 so that the corrector can visually recognize the speech recognition error.

[Configuration of speech recognition error correction device]
Next, the configuration of the speech recognition error correction device 20 will be described in detail.
As shown in FIG. 1, the speech recognition error correction device 20 includes a correction instruction input means 21, a speech recognition error partial recognition means 22, a recognition hypothesis integration means (hypothesis lattice integration means) 26, and a hypothesis rescoring means ( Voice recognition error portion correcting means) 27.

The correction instruction input means 21 is used by the corrector to input a correction instruction for the speech recognition result word string input from the speech recognition apparatus 10. For example, the correction instruction input means 21 specifies the position of the voice recognition error part by touching a word displayed on the display device 30 or a gesture using a pointing device. Select.
The gesture is to draw a symbol determined for each type of speech recognition error with a pointing device.

  Here, when the type of the speech recognition error is a drop error or a replacement error, it is necessary to specify a speech recognition error part in order to insert or replace the corrected word string in the speech recognition result word string. More specifically, the correction instruction input means 21 includes from the start time (start point) to the end time (end point) of the voice recognition error part specified by the corrector in the voice recognition result word string input from the voice recognition device 10. Is identified and output to the recognition hypothesis integrating means 26.

  The voice recognition error partial recognition means 22 is a correct word string candidate and a correction word string candidate that are correct correction word string candidates by voice recognition of an utterance input by the corrector or recognition of a handwritten character input by the corrector. Each recognition score is obtained. As shown in FIG. 1, the speech recognition error part recognition unit 22 includes a speech recognition unit 23, a handwritten character recognition unit 24, and a switch 25.

  Similar to the speech recognition apparatus 10, the speech recognition means 23 includes an acoustic model, a language model, and a pronunciation dictionary (not shown), and recognizes speech for a speech recognition error part. The voice recognition means 23 includes, for example, a microphone (not shown) for inputting the corrector's speech. Further, the language model and pronunciation dictionary of the speech recognition means 23 may be the same as that of the speech recognition device 10. Then, the speech recognition unit 23 outputs the corrected word string candidate and the recognition score for each corrected word string candidate to the switch 25 as a result of the voice recognition.

  Here, it is preferable that the speech recognition means 23 recognizes the speech of the corrector by using a specific speaker-specific acoustic model (not shown). Thereby, the voice recognition means 23 can correctly recognize the speech of the corrector.

  The handwritten character recognizing means 24 recognizes a handwritten character with respect to the voice recognition error part. The handwritten character recognition means 24 includes, for example, a tablet terminal and a stylus pen (not shown) for recognizing a corrector's handwritten character. And the handwritten character recognition means 24 outputs the correction word sequence candidate and the recognition score for every correction word sequence candidate to the switch 25 as a result of handwritten character recognition.

Here, the case where the result of handwritten character recognition is all hiragana or katakana is considered. In this case, the handwritten character recognizing means 24 reads out the phoneme string words corresponding to the hiragana or katakana notation from the pronunciation dictionary 17 provided in the speech recognition device 10 or the pronunciation dictionary provided in the speech recognition means 23, and corrected. It is preferable to use word string candidates. Thereby, the handwritten character recognizing means 24 can quickly correct the voice recognition error even when the corrector cannot immediately remember the kanji notation.
Note that the corrector can manually set which of the pronunciation dictionary 17 or the pronunciation dictionary of the speech recognition means 23 is to be used.

The switch 25 outputs one of the corrected word string candidate and the recognition score input from the speech recognition unit 23 or the handwritten character recognition unit 24 to the recognition hypothesis integration unit 26.
Here, the corrector manually selects either speech recognition or handwritten character recognition depending on the reason for the speech recognition error.
For example, when the voice recognition error is an acoustic reason such as an unclear utterance or misrepresentation of the program sound, the voice recognition is preferable to correct it more quickly and accurately. The recognition means 23 side is selected, and the speech for the speech recognition error part is input to the speech recognition means 23.
In addition, for example, in the case where the voice recognition error is a homonym, handwritten character recognition is preferable in order to correct more quickly and accurately. In this case, the corrector selects the handwritten character recognizing means 24 side with the switch 25 and inputs the handwritten character corresponding to the voice recognition error part to the handwritten character recognizing means 24.

  Thus, both the speech recognition means 23 and the handwritten character recognition means 24 output the corrected word string candidate and the recognition score. Therefore, even if the corrector selects either speech recognition or handwritten character recognition, the processing of the recognition hypothesis integrating means 26 can be made common, and the configuration of the speech recognition error correcting device 20 can be simplified.

  The recognition hypothesis integration unit 26 connects the branches associated with the corrected word string candidates and the recognition scores to the nodes of the branches located at the start and end points of the speech recognition error part of the hypothesis lattice input from the speech recognition apparatus 10. In this way, the hypothesis lattice is integrated. Further, the recognition hypothesis integration unit 26 outputs the integrated hypothesis lattice to the hypothesis rescoring unit 27.

<Integration of hypothesis lattice>
The hypothesis lattice integration by the recognition hypothesis integration means 26 will be described in detail with reference to FIGS. 2 and 3 (see FIG. 1 as appropriate).
Here, the word shown in FIG. 2A will be described as an example. That is, word w ₁ = “many”, word w ₂ = “remember”, word w ₃ = “淘汰”, word w ₄ = “to do”, word w ₅ = “similar”, word w ₆ = “ It is assumed that the word w ₇ = “do” and the word w ₈ = “do”.
As shown in FIG. 2B, the word string “Watch the video” is preceded by the voice recognition error part, and the word string “I was able to follow” follows the voice recognition error part. .

In the upper part of FIG. 2B, branches 10 to 12 in which the corrected word string candidates w _{1 to} w ₆ input from the speech recognition error partial recognition means 22 and the recognition scores P _{1 to} P ₃ are associated are shown. Illustrated. That is, the branch 10 indicates that the recognition score of the corrected word string candidate {w ₁ , w ₂ } = “remember a lot” is P ₁ . Further, the branch 11 indicates that the recognition score of the modified word string candidate {w ₁ , w ₃ , w ₄ } = “hesitantly” is P ₂ . The branch 12 indicates that the recognition score of the modified word string candidate {w ₅ , w ₆ } = “similar” is P ₃ .

A hypothesis lattice input from the speech recognition apparatus 10 is illustrated in the lower part of FIG.
This hypothesis lattice represents the evaluation contents in the speech recognition of the program sound. In other words, the hypothesis lattice is a directed graph composed of a branch that associates a word evaluated by voice recognition of program audio and an acoustic score for each word, and a node of a branch that indicates the start time (position) of the word. The acoustic score L is a score indicating how much the input program sound seems to be a word.

In FIG. 2 (b), the branches of the hypothetical lattice are indicated by arrows, and the nodes are indicated by black circles. Further, in the hypothesis lattice of FIG. 2 (b), the start time of speech recognition errors moiety is a T _S, the end time is T _E. Further, the node representing the nodes and end time T _E indicating the starting time T _S, the branch representing the maximum likelihood path (shown by a bold line), and the branch other than the most likely path (shown by broken lines) are connected .

In the hypothesis lattice of FIG. 2B, for each branch, a branch number that uniquely identifies the branch, a word w and an acoustic score L associated with the branch are illustrated. That is, branch 1 indicates that the acoustic score of the word w ₁ = “many” is L ₁ . The branch 2 indicates that the acoustic score of the word w ₂ = “remember” is L ₂ . The branch 3 indicates that the acoustic score of the word w ₁ = “many” is L ₃ . The branch 4 indicates that the acoustic score of the word w ₃ = “淘汰” is L ₄ . The branch 5 indicates that the acoustic score of the word w ₄ = “Yes” is L ₅ . The branch 6 indicates that the acoustic score of the word w ₇ = “done” is L ₆ . The branch 7 indicates that the acoustic score of the word w ₈ = “O” is L ₇ .

That is, in the hypothesis lattice of FIG. 2 (b), the node indicating the start time T _S, 3 single branch is input. Further, from the node showing the start time _{T S,} the branch 1,3 branches. Branches 4 and 7 branch from the node at the tip of branch 3. Branches 1,7 are joined to the same node, the branch from the node 2 is output, the tip of the branch 2 are joined to the node indicating the end time T _E. Branches 5 and 6 branch from the node at the tip of branch 4. In addition, the branches 5, to join the node to indicate the end time T _E. Therefore, in the hypothesis lattice of FIG. 2B, between the start time T _S and the end time T _E , the path H _{1 of the} branch 1-2, the path H _{2 of the} branch 3-4-5, and the branch that path _{H 3} of 3-7-2, there are three paths.

The recognition hypothesis integrating unit 26 integrates the upper branches 10 to 12 in FIG. 2B into the lower hypothesis lattice in FIG. First, the recognition hypothesis integrating unit 26 determines whether the corrected word string candidate associated with the branches 10 to 12 matches the word string associated with each branch of the paths H _{1 to} H ₃ .

Here, when the corrected word string candidate matches the word string of the paths H _{1 to} H ₃ , the recognition hypothesis integrating unit 26 associates the recognition score P of the matching corrected word string candidate with the paths H _{1 to} H ₃ .
For example, the modified word string candidate {w ₁ , w ₂ } of the branch 10 = “remember a lot”, which matches the word string {w ₁ , w ₂ } of the path H ₁ of the branch 1-2. For this reason, the recognition hypothesis integration unit 26 associates the recognition score P ₁ of the corrected word string candidate {w ₁ , w ₂ } with the path H ₁ of the branch 1-2.
Further, for example, modify the word sequence candidates _{{w 1, w 3, w} 4} branches 11 = a "number selection to" word string path of _{H 2} branches 3-4-5 _{{w 1,} w _3, matches w ₄ }. For this reason, the recognition hypothesis integration unit 26 associates the recognition score P ₂ of the modified word string candidate {w ₁ , w ₃ , w ₄ } with the path H ₂ of the branch 3-4-5.

On the other hand, when the corrected word string candidate does not match the word paths of the branch paths H _{1 to} H ₃ , the recognition hypothesis integration unit 26 adds the branch path from which the corrected word string candidate is obtained to the hypothesis lattice and adds it. The recognition score P is associated with the branch path.
For example, the modified word string candidate {w ₅ , w ₆ } = “similar” of the branch 12 is the word string {w ₁ , w ₂ } = “remembers” of the path H ₁ of the branch 1-2, branch 3 The word string {w ₁ , w ₃ , w ₄ } of the path H ₂ of −4-5 = “satisfy”, the word string {w ₁ , w ₈ , w ₂ of the path H ₃ of the branch 3-7-2 } = Does not match any of “Remember many”.

Therefore, the recognition hypothesis integrating unit 26 newly generates branches 8 and 9 in which the words w ₅ and w ₆ included in the modified word string candidate {w ₅ , w ₆ } of the branch 12 are respectively associated. At this time, since the acoustic scores L ₈ and L _{9 of} the words w ₅ and w ₆ do not exist, the recognition hypothesis integrating unit 26 requests the speech recognition apparatus 10 to calculate the acoustic scores L ₈ and L ₉ . Then, the recognition hypothesis integration unit 26 associates the acoustic scores L ₈ and L ₉ input from the speech recognition apparatus 10 with the branches 8 and ₉ in response to this request.

Furthermore, the recognition hypothesis integration unit 26, as shown in FIG. 3 (a), connecting the path H ₄ of the generated branches 8-9 to the hypothesis lattice. Specifically, the recognition hypothesis integration unit 26, from the node showing the start time T _S, diverts the branches 8. Further, the recognition hypothesis integration unit 26 connects the branch 9 to the node at the tip of the branch 8, extending to the nodes of the end time T _E. Then, the recognition hypothesis integration unit 26, the path _{H 4} branches 8-9 associates the recognition score _{P 3} modifications word string candidates that match the word sequence of the pathway _{H 4 {w 5, w 6} }.

It should be noted that there is no recognition score P _{4 of} the modified word string candidate {w ₁ , w ₈ , w ₂ } that matches the word string of the path H ₃ of the branch 3-7-2. In this case, the recognition hypothesis integration unit 26 generates a branch 13 for associating the recognition score P _4, and requests the calculation of the recognition score P ₄ to speech recognition errors partial recognition unit 22. In response to this request, the recognition hypothesis integration unit 26 associates the recognition score P ₄ input from the speech recognition error partial recognition unit 22 with the branch 13.
Further, regarding the branch 3-4-6, since the branch 6 is not connected to the node of the maximum likelihood path, it is not necessary to associate a recognition score.
As a result, as shown in FIG. 3B, recognition scores P _{1 to} P ₄ are associated with the paths H _{1 to} H ₄ of the integrated hypothesis lattice.

Returning to FIG. 1, the description of the configuration of the speech recognition error correction apparatus 20 will be continued.
The hypothesis rescoring means 27 calculates an integrated score L ′ using the acoustic score L and the recognition score P for each branch path H in the hypothesis lattice input from the recognition hypothesis integration means 26, and rescores the hypothesis lattice. It is something to ring. Then, the hypothesis rescoring means 27 estimates the path of the branch having the highest calculated integrated score as a correct corrected word string, and corrects the speech recognition error part with the estimated corrected word string.

<Rescoring Hypothesis Lattice>
With reference to FIG. 3, the rescoring by the hypothesis rescoring means 27 will be described in detail (see FIG. 1 as appropriate).
The hypothesis rescoring means 27 calculates, for each branch path H, the weighted sum of the acoustic score L and the recognition score P as an integrated score L ′. That is, the hypothesis rescoring means 27 calculates the total value of the values obtained by multiplying the acoustic score L of each branch and the weight a (n) of each branch. Further, the hypothesis rescoring means 27 multiplies the recognition score P of the modified word string candidate that matches the word string of each path H and the weight b (m) of the branch associated with this recognition score P, as described above. Add to the total value.
Note that n is a branch number associated with the acoustic score L (in this embodiment, 1 ≦ n ≦ 9), and m is a branch number associated with the recognition score P (in this embodiment, 10 ≦ m ≦ 13).

In FIG. 3A, the hypothesis rescoring means 27 assigns a weight a (1) to the acoustic score L ₁ of the branch 1 for the path H ₁ of the branch 1-2, as shown in the following equation (1). The sum of the multiplied value, the value obtained by multiplying the acoustic score L ₂ of the branch 2 by the weight a (2), and the value obtained by multiplying the recognition score P ₁ of the path H ₁ of the branch 1-2 by the weight b (10). Is calculated as an integrated score L ′ (H ₁ ).

Moreover, the hypothesis rescoring unit 27, the route of _{H 2} branches 3-4-5, as shown in the following formula (2), multiplied by the weight a (3) to the acoustic score _{L 3} branches 3 values A value obtained by multiplying the acoustic score L ₄ of the branch 4 by the weight a (4), a value obtained by multiplying the acoustic score L ₅ of the branch 5 by the weight a (5), and a route H _{2 of the} branch 3-4-5 The sum of the value obtained by multiplying the recognition score P ₂ by the weight b (11) is calculated as the integrated score L ′ (H ₂ ).

Moreover, the hypothesis rescoring unit 27, the route _{H 3} branch 3-7-2, as shown in the following formula (3), multiplied by the weight a (3) to the acoustic score _{L 3} branches 3 values And the value obtained by multiplying the acoustic score L ₇ of the branch 7 by the weight a (7), the value obtained by multiplying the acoustic score L ₂ of the branch 2 by the weight a (2), and the path H _{3 of the} branch 3-7-2. the sum of the value obtained by multiplying the weight b (13) to the recognition score _{P 4} of the, calculated as total score _{L'(H 3).}

Moreover, the hypothesis rescoring unit 27, the route _{H 4} branches 8-9, as shown in the following formula (4), a value obtained by multiplying the weight a (8) to the acoustic score _{L 8} branches 8, a value obtained by multiplying the weight a (9) to the acoustic score _{L 9} branch 9, the sum of the value obtained by multiplying the weight b (12) to the recognition score _{P 3} of the path _{H 4} branches 8-9, the total score L Calculated as' (H ₄ ).

  Here, the weights a (n) and b (m) represent the reliability of the recognition hypothesis (recognition score) of the speech recognition error partial recognizing means 22 with the reliability of the recognition hypothesis (acoustic score) of the program audio being a￣. Assuming b そ れ ぞ れ, it can be expressed by the following equations (5) and (6), respectively.

  Since the recognition hypothesis of the speech recognition error partial recognition means 22 is more reliable than the recognition hypothesis of program audio, it is often set in advance so that the reliability b 信 頼 is higher than the reliability a￣. Further, C (n) and C (m) are weights set in advance to align the dynamic range of the acoustic score L and the recognition score P, and indicate the input complexity and the number of parameters for speech recognition and handwritten character recognition. In general, the input complexity of speech recognition is evaluated by an amount obtained by weighting the exclusive time (number of frames) of the word. Further, in the case of handwritten character recognition, evaluation is performed with an amount weighted to the number of characters and the number of strokes.

  As described above, the hypothesis rescoring means 27 can calculate the integrated score L ′ by expanding the hypothesis.

  Thereafter, the hypothesis rescoring means 27 constrains the maximum likelihood path other than the speech recognition error part in the hypothesis lattice, and indicates a language score indicating the sentence-likeness of the entire hypothesis sentence calculated from the language model of the speech recognition apparatus 10; The evaluation is performed again using the integrated score L ′, and the hypothesis having the highest total score of both is estimated as a correct corrected word string. In the example of FIG. 3, the hypothesis re-scoring means 27 calculates the language score after connecting the word candidates of the speech recognition error part with the word strings “watching the video” and “being able”. Then, the evaluation is performed again with the integrated score L ′. Thereafter, the hypothesis rescoring means 27 corrects the error part of the speech recognition result word string input from the speech recognition device 10 to a correct corrected word string.

[Operation of voice recognition error correction device]
With reference to FIG. 4, the operation of the speech recognition error correction apparatus 20 will be described in detail.
The speech recognition error correction device 20 receives a hypothesis lattice from the speech recognition device 10 (step S1).
The speech recognition error correcting device 20 specifies the speech recognition error part by the correction instruction input means 21 (step S2), and selects the type of speech recognition error (step S3).

In the speech recognition error correcting device 20, the corrector inputs utterances or handwritten characters by the speech recognition error partial recognition means 22, and calculates a recognition score for each corrected word string candidate (step S4).
The speech recognition error correction apparatus 20 integrates the branch in which the recognition score for each candidate word string candidate is associated with the speech recognition error part of the hypothesis lattice input in step S1 by the recognition hypothesis integration unit 26 (step S5). ).

The speech recognition error correction device 20 calculates an integrated score for each branch path in the hypothesis lattice by the hypothesis re-scoring means 27, performs re-scoring using the language score and the integrated score L ′, and calculates the total score of both. Is estimated as a correct corrected word string (hypothesis rescoring: step S6).
The speech recognition error correcting device 20 corrects the speech recognition error part with the estimated corrected word string by the hypothesis re-scoring means 27 and outputs it as a correction result (step S7).

  As described above, the speech recognition error correction apparatus 20 according to the embodiment of the present invention complementarily integrates speech recognition results having different error tendencies and handwritten character recognition results into the speech recognition error portion of the hypothesis lattice, Since an integrated score is calculated from the integrated hypothesis lattice, a correct corrected word string can be estimated with high accuracy.

Furthermore, the speech recognition error correction device 20 allows the corrector to easily correct the speech recognition error, and the corrector can concentrate on finding and correcting the speech recognition error without being troubled by the correction operation. it can.
Furthermore, the speech recognition error correction device 20 can use an input method that does not require special skills such as handwritten characters and utterances, and the time and effort required to know the correction operation when performing the correction work is reduced. As a result, the speech recognition error correction apparatus 20 can make more people involved in the correction work, and can expand the caption program and reduce the production cost.

(Modification)
Note that the speech recognition error correction apparatus 20 is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit thereof.
The hypothesis rescoring means 27 may calculate the sum of the posterior probability corresponding to the acoustic score L and the posterior probability corresponding to the recognition score P for each branch path as the integrated score L ′.

  Specifically, the hypothesis rescoring means 27 can calculate the posterior probability L (i) corresponding to the acoustic score L using a forward backward algorithm. Also, the hypothesis rescoring means 27, when the recognition score P corresponds to the log likelihood as in the case of the acoustic score L, the posterior probability P ′ (m) corresponding to the recognition score P is expressed by the following equation (7). It can be calculated using the log likelihood calculation formula defined in.

When the posterior probability is obtained in this way, the hypothesis rescoring means 27 performs the posterior corresponding to the acoustic score L ₁ of the branch 1 for the path H ₁ of the branch 1-2, as shown in the following equation (8). probability L (1), and the posterior probability L (2) corresponding to the acoustic score _{L 2} branches 2, the posterior probability P'(10) corresponding to the recognition score _{P 1} of the path _{H 1} branches 1-2 The sum is calculated as the integrated score L ′ (H ₁ ).

The hypothesis rescoring means 27 can calculate the integrated score L ′ in the same way for the other routes H, and thus detailed description thereof is omitted.
Further, the hypothesis rescoring means 27 can use a method of approximately calculating the posterior probability L (i) other than the forward backward algorithm.

DESCRIPTION OF SYMBOLS 1 Subtitle production | generation system 10 Speech recognition apparatus 11 Speech recognition means 13 Acoustic model 15 Language model 17 Pronunciation dictionary 20 Speech recognition error correction apparatus 21 Correction instruction input means 22 Speech recognition error partial recognition means 23 Speech recognition means 24 Handwritten character recognition means 25 Switch 26 Recognition hypothesis integration means (hypothesis lattice integration means)
27 Hypothesis rescoring means (speech recognition error partial correction means)
30 Display device

Claims (6)

A speech recognition error correction device that corrects a speech recognition error included in a word string indicating a speech recognition result of a program sound with a correct correction word string,
A corrected word string candidate that is a candidate for the correct corrected word string and a corrected word string candidate for each of the corrected word string candidates by voice recognition of a voice recognition error part by the corrector's utterance or handwritten character recognition of the voice recognition error part by the corrector Speech recognition error partial recognition means for obtaining a recognition score of
A hypothesis lattice composed of each word evaluated in the speech recognition of the program sound and a branch in which the acoustic score of each word is associated and a node of the branch indicating the position of each word is input and input. A hypothesis lattice that integrates the hypothesis lattice by connecting a branch associated with the corrected word string candidate and the recognition score to the node of the branch located at the start point and the end point of the speech recognition error part of the hypothesis lattice. Integration means,
An integrated score is calculated using the acoustic score and the recognition score for each branch path from the start point to the end point in the integrated speech recognition error part of the hypothesis lattice, and the calculated integrated score is the highest. Speech recognition error portion correcting means for estimating a branch path as the correct corrected word string;
A speech recognition error correction apparatus comprising:   2. The speech recognition error correction apparatus according to claim 1, wherein the speech recognition error partial recognition means performs speech recognition using an acoustic model for a specific speaker unique to the corrector.   3. The speech according to claim 1, wherein the speech recognition error part correcting unit calculates a weighted sum of the acoustic score and the recognition score as the integrated score for each path of the branch. Recognition error correction device.   The speech recognition error part correcting unit is configured to calculate, for each path of the branch, a sum of a posterior probability of the acoustic score and a posterior probability of the recognition score calculated by a logarithmic likelihood calculation formula set in advance. The speech recognition error correction apparatus according to claim 1, wherein the speech recognition error correction apparatus is calculated as follows.   The speech recognition error part recognition means, when the result of the handwritten character recognition is all hiragana or katakana, from the pronunciation dictionary used for speech recognition of the program sound, or the pronunciation dictionary used for speech recognition of the speech recognition error part, 5. The speech recognition error correction apparatus according to claim 1, wherein a phoneme string word corresponding to the hiragana or katakana notation is read out and used as the corrected word string candidate. 6. .   A speech recognition error correction program for causing a computer to function as the speech recognition error correction device according to claim 1.

Images