JP5271299B2 - Speech recognition apparatus, speech recognition system, and speech recognition program - Google Patents

Description

  The present invention relates to a voice recognition device, a voice recognition system, and a voice recognition program, and more particularly, to a voice recognition device, a voice recognition system, and a voice recognition program for improving accuracy of voice recognition.

  Conventionally, voice recognition processing has been performed for the purpose of adding subtitles to a live broadcast program such as a television in real time. When such a speech recognition process is used, the current speech recognition technology is not perfect, and an error of several percent occurs in the output recognition word string. Therefore, it is common to arrange one or several operators who correct errors in real time in the subsequent stage of the speech recognition device and send out text corrected by errors as subtitle broadcast (Non-patent Document 1, (Refer nonpatent literature 2.).

  However, the information on the words that are corrected in real time in this way is not currently fed back to the speech recognition device, and which word was misrecognized and how was it correctly recognized, Information useful for subsequent speech recognition processing is not effectively utilized. In recent years, research has been made to feed back a speech recognition result including an error to a speech recognition device to improve future recognition accuracy. For example, a method called a cache model (see, for example, Non-Patent Document 3). ) And a method of feeding back the text whose error has been corrected to the speech recognition apparatus (for example, see Non-Patent Document 4).

Ando et al., “Broadcast News Subtitle Production System Using Speech Recognition,” IEICE Transactions, vol. J84-D-II, no. 6, pp. 877-887, 2001 Honma et al., “Real-time caption production system using both direct and lispeak speech recognition”, Transactions of the Video Information Society, vol. 63, no. 3, pp. 331-338, 2009 Non-Patent Document 3: Kenji Kita, “Stochastic Language Model”, University of Tokyo Press, pp. 77, 1999 Honma et al., “Improvement of free speech recognition for news talk programs”, Proceedings of the Spring Meeting of the Acoustical Society of Japan, 3-Q-17, pp. 243-244, 2009

  However, any of the conventional methods described above is limited to correcting the language model representing the chain appearance probability of words, and is reflected only in future speech recognition processing. For this reason, it is not possible to correct the acoustic model representing the characteristics of the voice, or to correct the accuracy of the correct answer quickly, because it is not possible to perform correction such as sequential correction on a part that has already been recognized but is not yet confirmed as subtitle text. There was a problem.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide a speech recognition device, a speech recognition system, and a speech recognition program for improving accuracy of speech recognition.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

According to the first aspect of the present invention, in a speech recognition apparatus that performs speech recognition using a speech recognition result and an error correction result for an input speech, an acoustic analysis unit that extracts an acoustic feature amount of the input speech, and a preset A word lattice generation means for generating a word lattice comprising a network of candidate words corresponding to the acoustic feature quantities of the input speech obtained by the acoustic analysis means using the acoustic model, the language model, and the pronunciation dictionary; An acoustic model for learning the acoustic model using a maximum likelihood word string selection means for selecting a maximum likelihood word string from the word lattice obtained by the word lattice generation means, and a word string modified with respect to the maximum likelihood word string A word lattice for the modified word string using an identification learning means and an acoustic model learned by the acoustic model identification learning means; Possess a word lattice reconstruction means for reconstructing further the best word string selecting means uses the word lattice obtained by the word lattice reconstruction means, word string after the corrected word sequence The most likely word string for is selected .

  According to the first aspect of the present invention, it is possible to improve the accuracy of voice recognition.

  In the invention described in claim 2, the word lattice reconstructing means replaces an incorrect word with a correct word among the candidate words included in the initial word lattice for the input speech, and is lacking correct The word lattice is entirely or partially reconstructed by newly inserting a word and deleting a word that cannot be connected to a correct word.

  According to the second aspect of the present invention, the corrected portion can be quickly reflected in the word lattice, so that the accuracy of correct speech recognition can be further improved.

  In the invention described in claim 3, when the acoustic model identification learning unit acquires a correct word string for the same input speech a plurality of times, only the statistical information of the latest correct word string is used, and the latest correct word string is used. The acoustic model is trained by deleting statistical information of old correct word strings other than the strings.

  According to the third aspect of the present invention, even when the same part is corrected again due to an operator's mistake or for some reason, the accuracy of the model is improved by causing the acoustic model to learn only the latest correct word string. Can be made.

  The invention described in claim 4 is characterized in that the word lattice reconstructing means reconstructs a word lattice repeatedly until the corrected word string becomes the correct word string.

  According to the fourth aspect of the present invention, the correction contents can be reflected in real time. In addition, the accuracy of speech recognition can be further improved by performing a plurality of reconstructions.

  According to a fifth aspect of the present invention, there is provided the speech recognition apparatus according to any one of the first to fourth aspects and an error correction apparatus that performs error correction on a speech recognition result obtained from the speech recognition apparatus. In the speech recognition system, the error correction device includes a word string display unit that displays the latest recognition word sequence sequentially input from the speech recognition device on a screen, and a recognition word sequence displayed by the word sequence display unit. Error correction means for correcting an error with respect to the information, and an information output means for outputting a correct word string obtained by the error correction means to an external device and / or feeding back to the voice recognition device. And

  According to the fifth aspect of the present invention, for example, it is possible to repeatedly obtain a speech recognition result for the same utterance and automatically change the characters on the screen of the error correction device to the latest state. Therefore, correction and confirmation of the speech recognition result can be performed quickly and accurately.

According to a sixth aspect of the present invention, in a voice recognition program for performing voice recognition using a voice recognition result and an error correction result for an input voice, a computer analyzes an acoustic analysis for extracting an acoustic feature quantity of the input voice. Generating a word lattice comprising a network of candidate words corresponding to an acoustic feature quantity of the input speech obtained by the acoustic analysis means , using a means, a preset acoustic model, a language model, and a pronunciation dictionary Means for selecting a maximum likelihood word string from the word lattice obtained by the word lattice generation means, and learning the acoustic model using a word string modified with respect to the maximum likelihood word string The acoustic model identification learning means and the acoustic model learned by the acoustic model identification learning means are used for the correction. To function as a word lattice reconstruction means for reconstructing the word lattice for a word string, further wherein the best word string selecting means uses the word lattice obtained by the word lattice reconstruction means, said modified word sequence The most likely word string for a later word string is selected .

  According to the sixth aspect of the present invention, the accuracy of correct speech recognition can be improved sequentially. In addition, voice recognition processing can be easily realized by installing an execution program in a computer.

  According to the present invention, the accuracy of voice recognition accuracy can be improved.

It is a figure which shows the system configuration example of the speech recognition system in this embodiment. It is a figure which shows an example of the function structure in an error correction apparatus. It is a figure which shows the example of a change of the character string displayed on a recognition word string display means. It is a figure which shows an example of an initial word lattice. It is a figure which shows an example of the word lattice reconfigure | reconstructed by the replacement of a word. It is a figure which shows an example of the word lattice reconfigure | reconstructed by the addition of a word. It is a figure which shows an example of the word lattice reconfigure | reconstructed by the deletion of a word. It is a figure for demonstrating the difference of each processing time in this embodiment. It is a figure which shows an example of a function structure of the speech recognition apparatus in other embodiment. It is a flowchart which shows an example of the process sequence of speech recognition.

<About the present invention>
In the present invention, for example, confirmation of speech recognition results and error correction information are fed back online to a speech recognition apparatus, and adaptive learning of an acoustic model is performed in an identifying manner from the correspondence between correct words and incorrect words. In addition, the present invention automatically corrects a word lattice, which is a network of candidate words for speech recognition, reconstructs the word lattice, and re-scores to sequentially output recognition results with higher accuracy. It is.

  More specifically, in the present invention, for example, when speech recognition is used for the purpose of giving subtitles to a live broadcast program such as a TV in real time, a character string that is sequentially corrected online, that is, subtitle text is converted into a voice recognition device. As a result, it is possible to automatically correct a recognition word string that has already been output if it has not been confirmed as subtitle text, and to output a recognition result with higher accuracy.

  Hereinafter, preferred embodiments of a voice recognition device, a voice recognition system, and a voice recognition program according to the present invention will be described with reference to the drawings. Note that the following voice recognition is basically processed sequentially for each utterance. One utterance is a voice section surrounded by two silence sections (for example, a section that is silent in about 400 ms).

<Voice recognition system: system configuration example>
FIG. 1 is a diagram illustrating a system configuration example of a voice recognition system according to the present embodiment. A speech recognition system 1 shown in FIG. 1 includes, as a speech recognition device 10, an acoustic analysis unit 11, a word lattice generation unit 12, a language model / pronunciation dictionary 13, an acoustic model 14, and a maximum likelihood word string selection unit 15. In addition to the word lattice reconstruction means 16 and the acoustic model identification learning means 17, an error correction device 18 is further provided.

  In the present embodiment, as shown in FIG. 1, an error correction device 18 that corrects an error in a recognized word string output by the speech recognition device processing is provided outside the speech recognition device 10. 10 may have the same configuration.

  Further, the discrimination learning of the acoustic model 14 according to the present embodiment includes, for example, phoneme error minimization learning (non-patent document: D. Povey and PC Woodland, “Minimum phone error and I-smoothing for stimulated training,” Proc. IEEE ICASSP, pp. I-105-108, 2002.) can be used.

<Voice Recognition Device 10: Functional Configuration Example>
First, a specific functional configuration of the voice recognition device 10 in the voice recognition system 1 will be described.

  The acoustic analysis means 11 analyzes a voice signal (input voice) input from the outside and extracts the acoustic feature amount. As the acoustic feature amount, for example, various acoustic feature amounts such as frequency characteristics and sound power are extracted. In addition, these feature quantities are extracted by first applying a window function (such as a Hamming window) to the speech waveform of the speech signal to extract a framed waveform and analyzing the frequency of the waveform to extract various feature quantities. To do. In the present embodiment, for example, a cepstrum coefficient that is a value obtained by performing inverse Fourier transform on the logarithm of the power spectrum of a framed waveform can be used as a feature amount, and it is also used in other general speech recognition methods. Yes. In the present embodiment, for example, Mel Frequency Cepstrum Coefficients (MFCC: Mel Frequency Cepstrum Coefficients) representing voice characteristics (see, for example, Shikano et al., “Speech Recognition System”, Ohmsha, 2001, etc.) Analyzes features such as LPC (Linear Predictive Coding) coefficients that quantify the shape of the vocal tract, features such as prosody (pitch, intonation), and statistical information such as the average value and variance of these features By doing so, various feature amounts can be acquired. The acoustic analysis unit 11 outputs various acoustic feature amounts obtained by the analysis to the word lattice generation unit 12.

  The word lattice generation unit 12 uses a language model / pronunciation dictionary 13 and an acoustic model 14 stored in advance from the acoustic feature quantity obtained by the acoustic analysis unit 11 to identify a plurality of words that may be recognition candidates. A word lattice composed of a network is generated. In addition, the word lattice generation unit 12 outputs the generated word lattice to the maximum likelihood word string selection unit 15.

  The language model / pronunciation dictionary 13 stores a plurality of language models and pronunciation dictionaries necessary for speech recognition in the present embodiment set in advance. Here, as the language model, for example, a general N-gram model expressing the ease of connection between words as a probability can be used. The probability of connection of “” is 0.8 ”, etc., and the ease of connecting each word can be expressed numerically.

  The pronunciation dictionary is a file in which the pronunciation of each word is represented by a combination of vowels and consonants. For example, the pronunciation of the word “Earth” is described as “/ chi ki u: /” or the like. As shown in FIG. 1, the language model and the pronunciation dictionary may be stored in an integrated database, or may be configured as separate databases. The data of the language model / pronunciation dictionary 13 is used by the word lattice generation means 12.

  The acoustic model 14 represents the frequency characteristics of each vowel / consonant voice, and can be represented by a general hidden Markov model (HMM). In the present embodiment, the acoustic model 14 or the word lattice generation means 12 may have a speech recognition parameter. Speech recognition parameters are information on variables that adjust the accuracy and processing speed of speech recognition, such as the maximum number of words that should be retained during the speech recognition process and weighting factors that adjust the balance of each score between the language model and the acoustic model. is there.

  Furthermore, the acoustic model 14 described above can be learned to the latest by the acoustic model identification learning means 17 using the correct word string obtained from the word lattice reconstruction means 16. The data of the acoustic model 14 is used by the word lattice generation means 12 for the next utterance and the word lattice reconstruction means 16 for the utterance currently being processed.

  Further, as in the present embodiment, the language model / pronunciation dictionary 13 and the acoustic model 14 are provided inside the speech recognition apparatus 10, but may be provided outside, in which case other external devices, etc. May be updated as appropriate.

  The maximum likelihood word string selection means 15 searches for the path of the word string having the highest score among the word lattices obtained by the word lattice generation means 12, and outputs this as the initial recognition word string.

  The recognition word string obtained by the above-described processing is then subjected to error determination by the error correction device 18. In the error correction device 18, the correct word is sequentially determined and corrected in near real time by the error correction operator from the beginning of the currently processed utterance corresponding to the input speech, and these partial correct word strings are executed. Is used in an application (external device) such as subtitle broadcasting, for example, and is output to the word lattice reconstruction unit 16 to reflect error correction.

  That is, the error correction device 18 determines the presence / absence of an error by determining whether the input recognition word string is correct, and determines the result of the determination (a control signal indicating that there is OK or correction) and the data of the OK or corrected data. The contents (corrected word string (a word string that was correct from the beginning or a word string in which an error was corrected to the correct answer)) are output to the speech recognition apparatus 10.

  The word lattice reconstructing means 16 reconstructs a word lattice for a corrected word string such as a correct word string using the acoustic model 14 learned by the acoustic model identification learning means 17. The word string is, for example, a partial character string composed of one or a plurality of words corresponding to the utterance section of the input voice.

  In addition, the word lattice reconstruction unit 16 has already finished the initial speech recognition processing at present, but the final result of subtitle text and the like corresponds to an incorrect word in the word recognition word recognition of the uncertain part. The link to the word node to be removed is removed from the correct word string. Further, the word lattice reconstruction means 16 also removes the word node of the final result undetermined part that can be connected only to the word nodes to be removed, and the word lattice of the utterance currently being processed corresponds only to the correct word string. To restructure and update the word lattice.

  Further, the word lattice reconstruction unit 16 outputs the reconstructed word lattice to the maximum likelihood word string selection unit 15. Further, the word lattice reconstructing means 16 outputs the finally determined information (correct word string) to the acoustic model identification learning means 17. The confirmed correct word string is, for example, a correct word string received along with the signal when an OK control signal is received from the error correction device 18.

  Note that the word lattice reconstruction unit 16 may use a word string obtained from the error correction device 18 as a whole word string or a partial character string made up of one or more words in one utterance.

  The acoustic model identification learning means 17 inputs the correct word string information from the word lattice reconstructing means 16, identifies the correct word and the error word in the word lattice from the input correct word string, and these temporal correspondences The acoustic model 14 is discriminatively adaptively learned by using the attachment and is updated. The acoustic model identification learning means 17 recalculates and updates the scores of all words in the word lattice using the adaptively learned acoustic model 14. However, the acoustic model identification learning means 17 does not recalculate the score of a word to be removed, which will be described later.

  The update timing of the acoustic model 14 in the acoustic model identification learning means 17 is preferably when the correct word string finally determined from the word lattice reconstruction means 16 is input. In addition, for example, the correct word string from the error correction device 18 is input by the word lattice reconstructing means 16 and updated when the word lattice reconstructing means 16 outputs it to the acoustic model identification learning means 17. You may do it. Thereby, the update process of the acoustic model 14 can be performed every time the reconstruction process by the word lattice reconstruction unit 16 is performed. Furthermore, the acoustic model identification learning means 17 may perform updating at a certain time interval or every certain character amount, at the timing of receiving an execution control signal, or the like.

  Furthermore, when the acoustic model identification learning means 17 acquires the correct word string for the same input speech a plurality of times, for example, only the latest correct word string statistical information is used from among them, and the other old correct word strings are used. May be deleted, and the acoustic model 14 may be identified and learned.

  After the above reconstruction is completed, the maximum likelihood word string selection means 15 searches again the path of the word string having the highest score among the word lattices, and outputs this as a recognized word string after error reflection. Thereafter, the above-described error correction and correct word string feedback, acoustic model 14 and word lattice update are repeated until the end of the utterance.

  The error correction device 18 repeatedly acquires a recognition word string for input speech of one utterance, and always presents the maximum likelihood word string based on the latest word lattice to an operator who performs error correction. In addition, the error correction device 18 outputs the correct word string corrected by the operator to the speech recognition device 10.

  Here, the speech recognition apparatus 10 according to the present embodiment assumes a method of sequentially confirming recognition results early even during utterance. For the sequential speech recognition, for example, an early-determined conventional method as disclosed in Japanese Patent No. 3834169 can be used. As a result, the word lattice, the recognized word string, and the correct word string correspond sequentially from the beginning of the utterance instead of the entire utterance, and a more correct recognition result is obtained earlier, for example, a delay time of about 0.5 seconds. Will be output. Note that the early-determined conventional method applied in the present embodiment is not limited to this.

<Error Correction Device 18: Functional Configuration Example>
Next, a specific functional configuration example of the error correction device 18 in the voice recognition system 1 described above will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a functional configuration in the error correction apparatus. The error correction device 18 shown in FIG. 2 is configured to include a recognized word string display means 21, an error correction means 22, and an information output means 23.

  When the error recognition device 18 sequentially inputs the latest recognition word string from the speech recognition device 10, the error correction device 18 displays the character string on the screen by the recognition word string display means 21. The recognized word string display means 21 is composed of a screen such as a touch panel or a monitor, for example. The recognized word string display means 21 displays the recognized word character string to be displayed on a plurality of lines with line breaks in accordance with the size of the screen and the content of the character string. At this time, since the recognized word string is divided for each word, the recognized word string display means 21 displays the word words collectively so as not to break the line between the divided words.

  The error correction means 22 is, for example, an OK or error signal input by a user (error correction operator) using an input device such as a touch panel, a keyboard, or a mouse, and a corrected or correct answer character string. To decide. That is, the error correction means 22 confirms the recognition word string displayed on the screen and determines whether or not the correction is made. If there is a correction result, the error correction means 22 shows the content of the corrected data. input.

  When the control signal for the obtained correct word string is OK, the information output means 23 outputs the correct word string to an application (external device) such as caption broadcasting or feeds it back to the voice recognition device 10. Process. Note that both the output to the application and the feedback to the speech recognition apparatus 10 described above may be performed, or either one may be performed.

  Further, when the control signal for the correct word string is NG, the information output means 23 outputs the correct word string input by the error correction operator to the voice recognition device 10 and feeds it back. Output to the application. The information output means 23 can also send the above-described OK and NG control signals to the speech recognition apparatus 10. Thereby, the speech recognition apparatus 10 can easily obtain the result of error correction.

<Example of changing a character string in the recognized word string display means 21>
Next, an example of changing a character string in the recognized word string display means 21 will be described with reference to the drawings. FIG. 3 is a diagram showing an example of changing the character string displayed on the recognized word string display means. As shown in FIG. 3, the character string obtained from the speech recognition device 10 is displayed on the display screen 30 of the error correction device 18.

  As an example of speech recognition, for example, when the input speech “next / no / news / is” is speech-recognized as “follow / ha / New York / do”, first, the recognized word string display means 21 of the error correction device 18 is used. The voice recognition results are displayed on the screen one after another in almost real time while synchronizing with a delay of about 0.5 seconds from the input voice (FIG. 3 (a)).

  The display screen 30 of the error correction apparatus 18 can display, for example, a maximum of about 16 characters × 10 lines, and is clearly indicated by a partition 31 such as a symbol “|” in order to clarify a word boundary. . The symbol of the partition 31 is not limited to the above in the present invention, and may be “/”, “@”, “#”, or may be enclosed in parentheses.

  When the error correction device 18 displays the initial erroneous recognition result “continue” on the display screen 30, the error correction device 18 “follows” by an input means such as a touch panel and a keyboard of the operator who performs error correction (FIG. 3B). The error correction means 22 executes an instruction to correct the correct word “next”. In addition, the execution result rewrites the display screen 30 as such (FIG. 3C).

  The corrected text is used as a correct word string in an application such as subtitle broadcasting and fed back to the speech recognition apparatus 10. The speech recognition apparatus 10 performs identification learning of the acoustic model 14 and reconstruction of the word lattice, and the correct word string “no / news / is” for the unconfirmed part is again recognized as the recognized word string to the error correction apparatus 18. Sent.

  The error correction device 18 always displays the latest recognized word string on the display screen 30 (FIG. 3D). Therefore, the operator does not need to correct any further errors, and can correct the remaining words automatically by correcting only the first word, and can immediately determine the correct word string. . Note that the operator can confirm and output the recognition result in units of one line on the display screen 30. Is this one line unit exactly the same as the speech segment cut out as one utterance unit in speech recognition? As shown in 3 (e), it may be shorter. Thus, the error correction operator can execute the confirmation and correction of the recognition result without delay from the input voice even during the utterance without waiting for the utterance to end. In addition, the correction in this embodiment mentioned above may be performed for every word divided | segmented by the partition 31, and may be performed by the character string which consists of several words.

<Specific examples of word lattice reconstruction>
Next, a specific example of word lattice reconstruction will be described with reference to the drawings. FIG. 4 is a diagram illustrating an example of an initial word lattice. FIG. 5 is a diagram illustrating an example of a word lattice reconstructed by word replacement. FIG. 6 is a diagram illustrating an example of a word lattice reconstructed by adding words. FIG. 7 is a diagram illustrating an example of a word lattice reconstructed by deleting words.

  Assume that the utterance content of the input voice is “next / no / news /”. Here, the symbol “/” represents a word boundary (partition 31). In the present embodiment, the acoustic feature quantity is extracted by the acoustic analysis means 11, and the word lattice generation means 12 generates a network of a plurality of words (word lattices) that can be recognized as recognition candidates, for example, as shown in FIG. Suppose that

  Here, the beginning of the utterance in FIG. 4 represents silence immediately before the utterance (phonetic symbol sil), the end of the utterance represents silence immediately after the utterance (phonetic symbol sil), and other words may be candidates for the input speech. Represents a word. In addition to kanji kana notation of words, these word nodes have phonetic symbols represented by vowels and consonants, scores indicating the likelihood of correct answers, and information on the word end times in the input speech And Furthermore, each link represented by an arrow between word nodes represents a word connection possibility.

  In the examples of FIGS. 4 to 7, in order to show the contents of the invention more clearly, the word nodes that should be correct are connected with solid links, and the word nodes that should be incorrect are connected with broken links. Yes. In FIGS. 4 to 7, it is assumed that a word node written at the top of the figure has a higher score.

  The maximum likelihood word string selection means 15 searches for the path of the word string having the highest score from the initial word lattice shown in FIG. 4 and outputs it as an initial recognition word string. , A recognition word string “succeeding / ha / new york / do” is output.

  In the error correction device 18, when the erroneous word “follow” is corrected to the correct word “next” by the user (error correction operator) or the like, this partial correct word string “next” is converted into the word lattice reconstruction means. 16 and the acoustic model identification learning means 17.

  In the present embodiment, as an example, a real-time type is assumed in which the speech recognition apparatus 10 sequentially confirms the recognition results early even during speech.

  The acoustic model identification learning means 17 identifies the correct word “next” and the error word “follow” in the word lattice from the partial correct word string, and uses these temporal correspondences to generate the acoustic model 14. Is adaptively learned and updated.

  Accordingly, the acoustic model 14 is configured so that the score for the correct pronunciation / ts, u, g, i / is higher than the score for the incorrect pronunciation / ts, u, z, u, i, t, e /. Adaptive learning is done, and this updates the scores of all words in the word lattice. As a result, as shown in FIG. 5, for example, the scores of the words “s” and “s” are reversed, and the word “s” is higher than the score of the word “s”. Can happen. This corresponds to acoustic error correction that has been difficult with the conventional feedback method.

  The word lattice reconstructing means 16 removes the error word “follow” from the word lattice based on the partial correct word string, and the word nodes “ha”, “which can only be connected to the error word“ follow ”. "New York" is also removed. As a result, the word lattice of the unconfirmed part as the final result of the caption text or the like is reconstructed reflecting the confirmation and error correction information up to the present time.

  In this example, as shown in FIG. 5, the word sequence “next / no / news / is” is selected as the recognized word string by the maximum likelihood word string selecting means 15 and re-output. In addition, when the subsequent word “NO” is determined to be correct by a user (an operator who corrects an error) or the like, the word “bathing” or the word “was” that can only be connected from the wrong word “GA”. Will also be removed automatically.

  As described above, in the present embodiment, there is a possibility that the subsequent recognition error can be automatically corrected by simply performing an error correction and a correct word confirmation operation from the first word. Thereafter, by repeating error correction and feedback, and updating the acoustic model and word lattice until the end of the utterance, the error correction results can be sequentially reflected in the recognition result even during the utterance.

  In the example of error correction described above, the correct word is included in the word lattice, and the word lattice is reconstructed based on an instruction from the user or the like. On the other hand, when the error correction operator inputs a word that is not included in the word lattice as a correct word, a new word node is added to the word lattice, for example, as “topic” in FIG. In addition, as shown in FIG. 7, when the user instructs to delete the word “to”, the word node “to” is deleted and a new node from the word node “news” to the end of the utterance is deleted. Add a link. If all the uppermost columns (initial recognition results) in FIG. 4 are correct, the user confirms the word without changing the word, so that the correct word and the error word are the same as when the word is changed. This information can be used to perform discriminative adaptive learning and update of the acoustic model 14.

  As described above, according to the present embodiment, it is possible to quickly and accurately configure an appropriate lattice for every pattern of replacement, addition, and deletion for the reconstruction of the lattice. In the present embodiment, the above-described word replacement, addition, and deletion can be applied in appropriate combination.

  The speech recognition apparatus 10 can reconstruct the word lattice as described above, and can output a more correct word string of the undetermined portion as a recognized word string to the error correction apparatus 18 again. Therefore, since the error correction device 18 always displays the latest recognized word string, the error correction operator does not need to correct any further errors, and the correct word string may be determined immediately.

<Difference in processing time in this embodiment>
Next, differences in processing times in the present embodiment will be described with reference to the drawings. FIG. 8 is a diagram for explaining a difference in each processing time in the present embodiment. In FIG. 8, it is assumed that the speech from the start of speech to time T1 has already been input to the speech recognition apparatus.

  Since the speech recognition apparatus 10 requires a time difference of several tens of milliseconds in the acoustic analysis unit 11, the word lattice generation unit 12 performs speech recognition processing (acoustic score and language score calculation and partial word lattice) at the time T2. Generation). In addition, since a time difference of several hundred milliseconds is required to output a stable and reliable maximum likelihood word string, the maximum likelihood recognition word string up to time T3 is output, and the same character string is erroneous. It is displayed on the correction device 18. The operator who corrects the error monitors the delayed voice almost in synchronization with the displayed character string, and it takes some time to determine whether the answer is correct or incorrect, and to correct the correct word. The correct word strings up to are determined.

  Since the word lattice has been generated until time T2, the word lattice between time T2 and time T4 is reconstructed based on the correct word string. Here, if the maximum likelihood word string is also automatically changed, the final result undetermined portion between time T3 and time T4 is updated on the display screen of the error correction device 18. .

<Other embodiments>
Here, in the present embodiment described above, the acoustic model 14 is updated in the acoustic model identification learning means 17. However, the present invention is not limited to this. For example, the above-described learning is performed as a language model / pronunciation. The same can be applied to the language model included in the dictionary 13, and more accurate speech recognition can be realized by discriminating and learning the language model. Here, the content described above will be described as another embodiment with reference to the drawings.

  FIG. 9 is a diagram illustrating an example of a functional configuration of a speech recognition apparatus according to another embodiment. In addition, in the speech recognition apparatus 40 shown in FIG. 9 and the speech recognition apparatus 10 in FIG. 1 described above, the same reference numerals are given to components having substantially the same functions, and the specific description here is as follows. Omitted.

  In FIG. 9, the speech recognition device 40 is provided with language model identification learning means 41 as compared with the above-described embodiment. In other words, in the present embodiment, the language model / pronunciation dictionary 13 is sequentially updated in the same manner as the acoustic model 14 so as to correspond to the correct word string in which the language model is also input using the language model identification learning unit 41. More accurate voice recognition can be realized.

  At this time, the word lattice reconstruction means 16 can acquire data not only from the acoustic model 14 but also from the language model / pronunciation dictionary 13 to reconstruct the word lattice.

  In addition to the above-described embodiment, the present invention has a configuration including only the word lattice reconstruction means 16 for reconstructing a word lattice without performing identification learning for an acoustic model, a language model, or the like. The same effect can be obtained.

<Execution program>
Here, the voice recognition device in the present invention described above includes a CPU, a volatile storage medium such as a RAM (Random Access Memory), a nonvolatile storage medium such as a ROM, an input device such as a mouse, a keyboard, and a pointing device, A computer having a display unit for displaying images and data and an interface for communicating with the outside can be used.

  Therefore, each function of the speech recognition apparatus 10 can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  That is, an execution program (voice recognition program) for causing a computer to execute the processes in the above-described configurations is generated, and the voice recognition process is realized by installing the program in, for example, a general-purpose personal computer or server. be able to.

  Next, the speech recognition process procedure by the execution program in this invention is demonstrated using a flowchart.

<Example of voice recognition processing procedure>
FIG. 10 is a flowchart illustrating an example of a voice recognition processing procedure. FIG. 10 shows a flowchart of the entire error sequential correction type speech recognition apparatus.

  First, when the operation of the entire speech recognition apparatus is started, when a speech to be recognized starts to be input (S01), acoustic analysis is first performed to extract a preset acoustic feature amount (S02). Also, the beginning of speech is detected (S03). Thereafter, a network of a plurality of words that are possible candidates for recognition, that is, a word lattice is generated using the language model, pronunciation dictionary, acoustic model, and the like described above (S04).

  When the word lattice is generated, a stable and reliable maximum likelihood word string is selected and output to an error correction procedure or a procedure for feeding back (S05). In the error correction procedure, an error correction and correct answer determination procedure is executed in parallel with the speech recognition process. These procedures correspond to the processing performed by the error correction device 18 described above.

  Here, it is determined whether there is an input of a correct word string by the error correction procedure described above (S06). If there is an input of a correct word string (YES in S06), acoustic model identification learning (S07), The word lattice is reconstructed (S08), and the maximum likelihood word string is selected and output (S09). At this time, the maximum likelihood word string is corrected again and the correct answer is confirmed in parallel with the subsequent speech recognition process in the error correction procedure (S10).

  Next, when the result confirmation process is performed after the end of S10 or without input of the correct word string in the process of S06 (NO in S06), it is determined whether or not the speech recognition process has reached the end of speech (S11). ), If the end of the utterance has not been reached yet (NO in S11), the process returns to S04, and the processing after generation of the word lattice is repeated. If the utterance end has been reached (YES in S11), it is next determined whether or not the entire speech recognition process is to be terminated (S12). If the entire speech recognition process has not been completed yet (NO in S12), the process returns to S03, and the process after the start of the next utterance is detected. If an instruction to end the voice recognition process is received (YES in S12), the entire process ends.

  Note that the processing procedure for detecting the start and end of the utterance described above can be operated in any known utterance section detection method. For example, the technique described in Japanese Patent Application Laid-Open No. 2007-233148 filed by the present applicant. Etc. can be used. In addition, using a language model, a pronunciation dictionary, and an acoustic model, a speech recognition processing procedure for generating a network of a plurality of possible words as recognition candidates, that is, a word lattice, is performed by any known speech recognition method. For example, the technique described in Japanese Patent No. 3834169 can be used. In addition, the above-mentioned method is not limited to the content described in the said gazette in this invention.

  As described above, according to the present invention, the accuracy of correct speech recognition can be improved. Specifically, confirmation of speech recognition results and error correction information are fed back to the speech recognition device online, and an acoustic model is identified and adaptively learned from correspondence between correct and incorrect words, and candidate words for speech recognition are also identified. By automatically correcting the word lattice that is a network, reconstructing the word lattice, and re-scoring, it is possible to sequentially output recognition results with higher accuracy.

  That is, according to the present invention, for example, even a part that has already been voice-recognized, but has not yet been finalized as a final result such as subtitle text, can quickly improve the accuracy of accuracy and reduce the work load of the error correction operator. Can provide the final result such as subtitle text more correctly and with less delay time.

  In addition, according to the present invention, in addition to caption production, confirmation and error of speech recognition results online, such as making minutes and transcripts at meetings, parliaments, lectures, courts, etc., voice input with mobile phones, etc. It can be widely used in a phonetic character conversion system to which correction is applied.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 1 Speech recognition system 10,40 Speech recognition apparatus 11 Acoustic analysis means 12 Word lattice generation means 13 Language model and pronunciation dictionary 14 Acoustic model 15 Maximum likelihood word sequence selection means 16 Word lattice reconstruction means 17 Acoustic model identification learning means 18 Error correction Device 21 Recognition word string display means 22 Error correction means 23 Information output means 30 Display screen 31 Partition 41 Language model identification learning means

Claims (6)

In a speech recognition apparatus that performs speech recognition using a speech recognition result and an error correction result for an input speech,
Acoustic analysis means for extracting an acoustic feature of the input speech;
Word lattice generation means for generating a word lattice consisting of a network of candidate words corresponding to the acoustic feature quantities of the input speech obtained by the acoustic analysis means using a preset acoustic model, language model, and pronunciation dictionary; ,
Maximum likelihood word string selection means for selecting a maximum likelihood word string from the word lattice obtained by the word lattice generation means;
Acoustic model identification learning means for learning the acoustic model using a word string modified with respect to the maximum likelihood word string;
Using said acoustic model learned by the acoustic model identification learning means, possess a word lattice reconstruction means for reconstructing the word lattice for said modified word sequence,
Further, the maximum likelihood word string selection means selects a maximum likelihood word string for a word string after the corrected word string using the word lattice obtained by the word lattice reconstruction means. Voice recognition device. The word lattice reconstruction means includes:
Of the candidate words included in the initial word lattice for the input speech, the wrong word is replaced with the correct word, the missing correct word is newly inserted, and the word that cannot be connected to the correct word is deleted. The speech recognition apparatus according to claim 1, wherein the word lattice is reconfigured in whole or in part. The acoustic model identification learning means includes
When the correct word string for the same input speech is acquired a plurality of times, only the latest correct word string statistical information is used, and the old correct word string statistical information other than the latest correct word string is deleted, and the acoustic model The speech recognition apparatus according to claim 1, wherein learning is performed. The word lattice reconstruction means includes:
The speech recognition apparatus according to claim 3, wherein the word lattice is reconstructed repeatedly until the corrected word string becomes the correct word string. A speech recognition system comprising: the speech recognition device according to any one of claims 1 to 4; and an error correction device that performs error correction on a speech recognition result obtained from the speech recognition device.
The error correction device includes:
Word string display means for displaying the latest recognized word string sequentially input from the voice recognition device on a screen;
Error correction means for correcting an error for the recognized word string displayed by the word string display means;
A speech recognition system comprising: an information output means for outputting a correct word string obtained by the error correction means to an external device and / or feeding back to the speech recognition device. In a speech recognition program for performing speech recognition using speech recognition results and error correction results for input speech,
Computer
Acoustic analysis means for extracting an acoustic feature of the input speech;
Word lattice generation means for generating a word lattice comprising a network of candidate words corresponding to the acoustic feature quantities of the input speech obtained by the acoustic analysis means using a preset acoustic model, language model, and pronunciation dictionary;
Maximum likelihood word string selection means for selecting a maximum likelihood word string from the word lattice obtained by the word lattice generation means;
Acoustic model identification learning means for learning the acoustic model using a word string modified with respect to the maximum likelihood word string; and
Using the acoustic model learned by the acoustic model identification learning means, function as word lattice reconstruction means for reconstructing a word lattice for the corrected word string ,
Further, the maximum likelihood word string selection means selects a maximum likelihood word string for a word string after the corrected word string using the word lattice obtained by the word lattice reconstruction means. Speech recognition program.

Images