JP2010055044A - Device, method and system for correcting voice recognition result - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for correcting a voice recognition result, capable of correcting a recognition error without user's time and effort, when the error arises in the recognition result. <P>SOLUTION: A feature amount data of voice are transmitted to a server device 120. Voice recognition is performed in the sever device 120, and the recognition result is received from the server device 120 by a receiving section 235. An error section indicating section 240 indicates an error section where the recognition error arises, on the basis of a degree of confidence or the like. An error section feature amount extracting section 260 extracts a feature amount data of the error section, and a correction section 270 performs correction processing by performing recognition processing on the recognition result in the extracted error section. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a speech recognition result correcting apparatus, a speech recognition result correcting method, and a speech recognition result correcting system for correcting speech-recognized data.

Japanese Patent Application Laid-Open Publication No. 2003-228867 discloses a technology that outputs voice input to a portable terminal to a server, recognizes the voice on the server, and transmits the recognition result to the portable terminal. 1 is known.
JP 2003-295893 A

  However, if there is an error in the recognition result recognized by the server, the correction is not considered. In general, when there is an error in the recognition result, it can be considered that the user corrects it by manual operation, but this is very time-consuming. For example, it takes time and effort for the user to grasp a sentence as a recognition result, to recognize an error, to specify the erroneous part, and to correct it.

  Therefore, in the present invention, a speech recognition result correction apparatus, a speech recognition result correction method, and a speech recognition result correction system capable of correcting a recognition error without any user effort when there is an error in the recognition result. The purpose is to provide.

  In order to solve the above-described problem, a speech recognition result correction apparatus according to the present invention includes an input unit that inputs speech, a calculation unit that calculates feature data based on the speech input by the input unit, and the calculation A recognition error occurs in the storage unit that stores the feature amount data calculated by the unit, the acquisition unit that acquires the recognition result for the voice input by the input unit, and the recognition result that is recognized by the acquisition unit. Extracting the feature amount data corresponding to the error section specified by the specifying unit from the feature amount data stored in the storage unit, and using the extracted feature amount data Correction means for correcting the recognition result obtained by the acquisition means by performing re-recognition.

  The speech recognition result correction method of the present invention includes an input step for inputting speech, a calculation step for calculating feature amount data based on the speech input by the input step, and a feature amount calculated by the calculation step. A storage step for storing data, an acquisition step for acquiring a recognition result for the speech input by the input step, and an error section in which a recognition error has occurred in the recognition result recognized by the acquisition step Extracting the feature amount data corresponding to the error section specified by the specifying means from the feature amount data stored in the storing step, and performing re-recognition using the extracted feature amount data. A correction step for correcting the recognition result obtained by the acquisition step, That.

  According to the present invention, the feature amount data of the input voice is stored, and the error section where the recognition error has occurred is designated in the recognized recognition result for the voice. Then, the recognition result is corrected by re-recognizing the feature data in the designated error section. Accordingly, since a necessary part of the recognized result is corrected, correction processing can be easily performed and a correct recognition result can be obtained. Accordingly, correction processing can be easily performed without imposing a burden on the user, and a correct speech recognition result can be obtained.

  In the speech recognition result correction apparatus according to the present invention, the acquisition unit receives a speech input by the input unit and transmits a speech to the speech recognition device, and receives a recognition result recognized by the speech recognition device. Preferably, the designation means designates an error section in which a recognition error has occurred in the recognition result received by the reception means.

  According to the present invention, the input voice is transmitted to the voice recognition device, and the recognition result recognized by the voice recognition device is received. Then, in the received recognition result, an error section in which a recognition error has occurred is specified, and the recognition result in the specified error section is corrected. Accordingly, since a necessary part of the recognized result is corrected, an error in speech recognition can be easily corrected, and a correct recognition result can be obtained.

  In the speech recognition result correcting apparatus of the present invention, it is preferable that the specifying unit specifies an error section by accepting a user operation.

  According to the present invention, by accepting a user operation, an error interval can be specified, an error interval can be specified more easily, and a correct speech recognition result can be obtained.

  In the speech recognition result correction apparatus of the present invention, it is preferable that the specifying unit determines an error interval based on a reliability of the recognition result given to the recognition result, and specifies the determined error interval. .

  According to this invention, an error interval can be automatically specified by determining an error interval based on the reliability of the recognition result given to the recognition result, and specifying the determined error interval. An error interval can be specified easily.

  In the speech recognition result correcting apparatus of the present invention, it is preferable that the specifying unit calculates a reliability of the recognition result, determines an error interval based on the reliability, and specifies the determined error interval. .

  According to the present invention, the reliability of the recognition result is calculated, the error interval is determined based on the reliability, the determined error interval can be specified, and the error interval can be specified more easily. . Furthermore, even when a server device or the like performs voice recognition, it is not necessary to calculate reliability from the server device, and a more convenient device can be provided.

  The speech recognition result correcting apparatus according to the present invention includes at least one word immediately before the error section specified by the specifying means, or at least one word immediately after, or both the immediately preceding word and the immediately following word. The correction means further comprises a specifying means for specifying a recognition result forming any one of the words, and the correction means uses the recognition result specified by the specifying means as a constraint condition, and in accordance with the constraint condition, It is preferable that feature amount data corresponding to a section including a word is extracted from the storage unit and a recognition process is performed on the extracted feature amount data.

  According to the present invention, the recognition result forming either at least one word immediately before the specified error section, at least one word immediately after, or both of the immediately preceding word and the immediately following word is obtained. Using the identified recognition result as a constraint condition, it is possible to perform a process for recognizing feature data stored in advance according to the constraint condition. Thereby, more accurate recognition processing can be performed, and thus a correct speech recognition result can be obtained.

  The speech recognition result correcting apparatus according to the present invention includes at least one word immediately before the error section specified by the specifying means, or at least one word immediately after, or both the immediately preceding word and the immediately following word. The correction means further includes a specifying means for specifying any recognition result, and the correction means uses the recognition result specified by the specifying means as a constraint condition, and according to the constraint condition, the feature amount data corresponding to the error section is obtained. It is preferable to perform recognition processing on the extracted feature value data extracted from the storage means.

  According to the present invention, the recognition result forming either at least one word immediately before the specified error section, at least one word immediately after, or both of the immediately preceding word and the immediately following word is obtained. Using the identified recognition result as a constraint condition, it is possible to perform a process for recognizing feature data stored in advance according to the constraint condition. That is, according to the present invention, recognition processing can be performed using feature amount data only in error sections. Thereby, more accurate recognition processing can be performed, and thus a correct speech recognition result can be obtained.

  Further, the speech recognition result correction apparatus of the present invention is word information that is information for specifying at least one word immediately before the error section specified by the specifying means, or word information of at least one word immediately after, Or word information specifying means for specifying word information of a word in a recognition result forming either the word information of the immediately preceding word or the word information of the immediately following word, and the correcting means includes the word information Using the word information specified by the specifying means as a constraint condition, in accordance with this constraint condition, feature quantity data corresponding to the word immediately before the error section and the section including the word immediately after is extracted from the storage means, and the extracted feature quantity It is preferable to perform recognition processing on the data.

  According to this invention, more accurate recognition processing can be performed by performing correction processing using word information for specifying a word as a constraint.

  For example, it is preferable that the word information includes one or more of part-of-speech information indicating the word part-of-speech and reading information indicating how to read the word.

  Further, the speech recognition result correcting apparatus according to the present invention is based on the word information, the at least one word immediately before the error section specified by the specifying means, the at least one word immediately after, or the word immediately before and It is further provided with an unknown word determination means for determining whether or not the recognition result word forming either of the immediately following words is an unknown word, and the recognition result word is an unknown word by the unknown word determination means. If it is determined that there is, it is preferable that the correction means corrects the recognition result based on the word information.

  According to the present invention, when an unknown word is used, a more accurate speech recognition result can be obtained by performing recognition processing using word information as a constraint condition.

  The speech recognition result correction apparatus according to the present invention further includes connection probability storage means for storing connection probabilities between words, and the correction means performs correction processing on the word in the error section and before or after the word. It is preferable to create a connection probability with a word and update the connection probability stored in the connection probability storage means using the connection probability.

  According to the present invention, since the connection probability between words is stored and the connection probability changes every time correction processing is performed, more accurate speech recognition can be performed by calculating and updating the connection probability. The result can be obtained.

  The speech recognition result correcting apparatus according to the present invention further includes a constraint condition storage unit that stores the word information specified by the word information specifying unit or the word specified by the specifying unit as a constraint condition, and the correction unit The correction processing is preferably performed in accordance with the constraint conditions stored in the constraint condition storage means.

  As a result, it is possible to store a word or word information as a constraint condition and perform correction processing according to the stored constraint condition as necessary, and it is not necessary to generate a constraint condition every time correction processing is performed. Rapid correction processing (voice recognition processing) can be performed.

  The speech recognition result correction apparatus according to the present invention further includes a reception unit that receives character information from a user, and the correction unit corrects the recognition result in the error section using the character information received by the reception unit as a constraint. It is preferable to carry out the treatment.

  According to the present invention, the user can directly specify the character that is the constraint condition, can perform more accurate recognition processing, and thus can obtain a correct speech recognition result.

  The speech recognition result correcting apparatus according to the present invention further includes a time information calculating unit that calculates an elapsed time in the recognition result based on the recognition result received by the receiving unit and the feature amount data stored in the storage unit. Further, it is preferable that the specifying unit specifies an error section based on the time information calculated by the time information calculating unit.

  According to the present invention, the elapsed time in the recognition result can be calculated based on the received recognition result and the stored feature amount data, and the error section can be designated based on this time information. As a result, even when the recognition result does not include time information, it is possible to extract appropriate feature amount data corresponding to the error section.

  The speech recognition result correction apparatus according to the present invention preferably further includes display means for displaying the recognition result corrected by the correction means, and the display means does not display the recognition result acquired by the acquisition means. . Thereby, since the recognition result which may have a recognition error is not displayed, a misunderstanding is not given to a user.

  The speech recognition result correction apparatus according to the present invention includes a case where the recognition result obtained by re-recognition by the correction unit is the same as the recognition result obtained by the acquisition unit, or included in each of these recognition results. If there is a deviation in the time information to be displayed, it is determined that a recognition error has occurred, and the display means preferably does not display the recognition result. Thereby, it can prevent displaying an incorrect recognition result.

  Further, in the speech recognition result correction apparatus according to the present invention, the designation means designates a start point of an error section by a user operation, and based on the reliability of the recognition result given to the recognition result acquired by the acquisition means. It is preferable to specify the end point of the error interval. Thereby, the correction method suitable for the user's input habit can be realized, and a user-friendly device can be provided.

  In the speech recognition result correcting apparatus according to the present invention, it is preferable that the specifying unit specifies a start point of an error section by a user operation, and specifies an end point of the error section with a predetermined number of recognition units based on the start point. Thereby, the correction method suitable for the user's input habit can be realized, and a user-friendly device can be provided.

  Further, in the speech recognition result correcting apparatus according to the present invention, the designation unit designates a start point of an error section by a user operation, and determines an end point of the error section based on a predetermined phonetic symbol in the recognition result acquired by the acquisition unit. It is preferable to specify. Thereby, the correction method suitable for the user's input habit can be realized, and a user-friendly device can be provided.

  In the speech recognition result correction apparatus of the present invention, when the acquisition unit acquires a recognition result, the acquisition unit acquires a plurality of recognition candidates as a recognition result, and the specifying unit specifies a start point of an error interval by a user operation. Preferably, the end point is designated based on the number of recognition candidates acquired by the acquisition unit. Thereby, the end point based on the reliability of the recognition result can be designated, and the correction process can be realized efficiently.

  In the speech recognition result correction apparatus of the present invention, the speech recognition result correction apparatus further includes a calculation unit that calculates an average value of sections including error sections of the feature amount data calculated by the calculation unit, and the correction unit extracts the extracted feature amount data. It is preferable to subtract the average value calculated by the calculation means from the above and perform re-recognition processing using the data obtained by the subtraction as feature quantity data. As a result, correction processing can be performed on the sound from which the characteristics of the sound collector that inputs sound such as a microphone are removed, and more accurate correction (voice recognition) can be realized.

  Further, in the speech recognition result correcting apparatus according to the present invention, an input unit for inputting speech, an acquisition unit for acquiring a recognition result for the speech input by the input unit, and a recognition result recognized by the acquisition unit Designation means for designating an error section in which an error has occurred; notification means for requesting the external server to re-recognize the error section by notifying the external server of the error section designated by the designation means; Receiving means for receiving a recognition result of an error section re-recognized in the external server in response to a request from the notification means.

  Further, in the speech recognition result correction method of the present invention, in the input step of inputting speech, the acquisition step of acquiring the recognition result for the speech input by the input step, and the recognition result recognized by the acquisition step, A designation step for designating an error section in which an error has occurred; a notification step for requesting the external server to perform re-recognition processing of the error section by notifying the external server of the error section designated by the designation step; A receiving step of receiving a recognition result of the error section re-recognized in the external server in response to the request in the notification step.

  The speech recognition result correcting apparatus according to the present invention further comprises subword section specifying means for specifying a subword section in the recognition result acquired by the acquiring means, wherein the correcting means is an error specified by the specifying means. Further, in the section, feature amount data corresponding to the subword section specified by the subword section specifying unit is extracted from the storage unit, and re-recognition is performed using the extracted feature amount data. It is preferable to perform correction of the recognized recognition result.

  As a result, the recognition result can be corrected using the feature data corresponding to the subword section, and more accurate correction processing can be performed. That is, re-recognition can be performed according to an unknown word section such as a subword section.

  The speech recognition result correction apparatus of the present invention further includes a dividing unit that divides the recognition result acquired from the acquiring unit into a plurality of sections according to the subword section specified by the subword section specifying unit,

  It is preferable that the correction means corrects the recognition result for each divided section divided by the dividing means.

  Thereby, the recognition target can be shortened by dividing the recognition result into a plurality of sections, and more accurate recognition processing can be performed.

  The dividing means in the speech recognition result correcting apparatus of the present invention uses the end point of the subword section as the end point of one divided section, and the start point of the subword section as the start point of the next divided section of the one divided section. It is preferable to divide the recognition result.

  As a result, the subword section is included in any of the divided sections. Therefore, when the recognition process is performed, the subword section is always included, so that the recognition process using the subword character string as a constraint condition can be performed.

  Further, the correcting means in the speech recognition result correcting apparatus of the present invention executes correction of the recognition result for each divided section divided by the dividing means, and uses the subword section as a constraint condition in correcting each divided section. It is preferable.

  Thereby, when the recognition process is performed, the subword section is always included, and the recognition process using the subword character string as a constraint condition can be performed.

  Further, in the speech recognition result correcting apparatus of the present invention, the correcting means holds a hypothesis including a subword character string described in the subword section designated by the subword section designating means as a recognition search process, and from the hypothesis The correction is preferably performed by selecting the final recognition result.

  Thereby, the recognition process using the subword character string can be performed without fail.

  The speech recognition result correcting apparatus according to the present invention preferably further comprises a dictionary adding means for adding a subword character string in the subword section specified by the subword section specifying means to a dictionary database for recognition processing.

  As a result, subword character strings can be accumulated, and can be used effectively for future recognition processing, and more accurate recognition processing can be performed.

  The speech recognition result correction apparatus according to the present invention preferably further includes a dictionary database generated by a user, and the correction unit preferably performs a correction process using a character string obtained by converting a subword character string according to the dictionary database. .

  As a result, subword character strings can be accumulated, and can be used effectively for future recognition processing, and more accurate recognition processing can be performed.

  The speech recognition result correction system of the present invention performs speech recognition based on the speech recognition result correction device and the speech transmitted from the speech recognition result correction device, and transmits the recognition result to the speech recognition result correction device. A server device. This speech recognition result correction system is the same as the speech recognition result correction device described above in terms of operational effects except that the category is different.

  According to the present invention, a necessary part of the recognized result can be corrected, correction processing can be easily performed, and a correct recognition result can be obtained.

  Embodiments of the present invention will be described with reference to the accompanying drawings. Where possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

<First Embodiment>
FIG. 1 shows a system configuration of a communication system including a client apparatus 110 that is a voice recognition result correction apparatus according to the present embodiment and a server apparatus 120 that recognizes a voice transmitted from the client apparatus 110 and returns the result to the client apparatus 110. FIG. In the present embodiment, the client device 110 is a mobile terminal such as a mobile phone, for example. The client device 110 inputs voice uttered by the user, transmits the input voice to the server device 120 using wireless communication, and the server device 120. The recognition result which is a reply from can be received.

  The server device 120 includes a speech recognition unit, performs speech recognition on the input speech using a database such as an acoustic model and a language model, and returns the recognition result to the client device 110.

  Next, the configuration of the client device 110 will be described. FIG. 2 is a block diagram illustrating functions of the client device 110. The client device 110 includes a feature amount calculation unit 210 (input unit, calculation unit), a feature amount compression unit 220, a transmission unit 225 (acquisition unit, transmission unit), a feature amount storage unit 230 (storage unit), and a reception unit 235 ( (Acquisition unit, reception unit), error section specifying unit 240 (designating unit), context section specifying unit 250 before and after error section 250 (identifying unit), error section feature quantity extracting unit 260, correcting unit 270 (correcting unit), acoustic model holding unit 281 , A language model holding unit 282, a dictionary holding unit 283, an integration unit 280, and a display unit 290.

  FIG. 3 is a hardware configuration diagram of the client device 110. As shown in FIG. 3, the client device 110 shown in FIG. 2 physically includes a CPU 11, a RAM 12 and a ROM 13 as main storage devices, an input device 14 such as a keyboard and a mouse as input devices, and an output from a display or the like. The computer 15 includes a device 15, a communication module 16 that is a data transmission / reception device such as a network card, an auxiliary storage device 17 such as a hard disk, and the like. Each function described in FIG. 2 has the input device 14, the output device 15, and the communication module 16 under the control of the CPU 11 by reading predetermined computer software on the hardware such as the CPU 11 and the RAM 12 shown in FIG. 3. This is realized by reading and writing data in the RAM 12 and the auxiliary storage device 17. Hereinafter, each functional block will be described based on the functional blocks shown in FIG.

  The feature amount calculation unit 210 is a part that inputs a user's voice input from a microphone (not shown) and calculates feature amount data indicating a speech recognition spectrum and indicating acoustic features from the input voice. . For example, the feature amount calculation unit 210 calculates feature amount data indicating an acoustic feature represented by a frequency such as MFCC (Mel Frequency Cepstrum Coefficient).

  The feature amount compression unit 220 is a portion that compresses the feature amount data calculated by the feature amount calculation unit 210.

  The transmission unit 225 is a part that transmits the compressed feature value data compressed by the feature value compression unit 220 to the server device 120. The transmission unit 225 performs transmission processing using Hyper Text Transfer Protocol (HTTP), Media Resource Control Protocol (MRCP), Session Initiation Protocol (SIP), and the like. The server device 120 performs reception processing and reply processing using these protocols. Further, the server device 120 can decompress the compressed feature amount data, and perform voice recognition processing using the feature amount data. Since the feature amount compression unit 220 is for data compression in order to reduce communication traffic, the transmission unit 225 can transmit the feature amount data as it is without being compressed. .

  The feature amount storage unit 230 is a part that temporarily stores the feature amount data calculated by the feature amount calculation unit 210.

  The receiving unit 235 is a part that receives the voice recognition result returned from the server device 120. The speech recognition result includes text data, time information, and reliability information. The time information indicates the elapsed time for each recognition unit of the text data, and the reliability information indicates the correctness accuracy in the recognition result. It is information which shows.

For example, the information shown in FIG. 4 is received as the recognition result. In FIG. 4, the utterance content, the recognition content, the voice section, and the reliability are described in association with each other, but the utterance content is not actually included. Here, the number shown in the voice section indicates the index of the frame, and the index of the first frame of the recognition unit is shown. Here, one frame is about 10 msec. Further, the reliability indicates the reliability for each recognition unit of the speech recognition result recognized by the server device 120, and is a numerical value indicating how correct it is. This is generated using a probability or the like for the recognition result, and is added to the recognized word unit in the server device 120. For example, it is described in the following references as a method of generating reliability.
References: Lee Shin-nobu, Kawahara Tatsuya, Kano Kiyohiro. "High-speed reliability calculation method based on word posterior probabilities in two-pass search algorithm", IPSJ SIG, 2003-SLP-49-48, 2003-12.
In FIG. 4, for example, “sell” as a recognition result includes 33 frames to 57 frames, and the reliability is 0.86.

  The error section specifying unit 240 is a part that specifies an error section based on the speech recognition result received by the receiving unit 235. The error interval specification unit 240 can specify an error interval based on reliability information included in the speech recognition result transmitted from the server device 120, for example.

  For example, FIG. 4 shows that the recognition result is 905 (Kyumarugo), the time information is 9 frames (90 msec), and the reliability is 0.59 as the recognition result. The reliability of “where”, which is the recognition result, is 0.04. Then, the error section specifying unit 240 can determine that the reliability is equal to or less than a predetermined threshold value, and can specify the section as an error section. For example, if the reliability is set to 0.2 or less, it is determined that the “where”, “de”, and “tofu” portions are incorrect, and that portion is designated as the error section. can do. This threshold value is a numerical value set in advance on the client device 110 side. Note that it may be variably set depending on the individual difference of the voice, the amount of noise (noise), or the reliability calculation method. In other words, the reliability decreases further when there is a lot of noise, so the threshold is set low, and the reliability added to the speech recognition result is generally low or conversely high. In such a case, the reliability may be changed according to the level of reliability. For example, the threshold value may be set based on the median value of reliability, or the threshold value may be set based on the average value.

  The client device 110 includes a reliability calculation unit (not shown) that calculates the reliability information of the recognition result, and the error section specification unit 240 is based on the reliability information calculated in the client device 110. An error interval may be designated.

  The context specifying unit 250 before and after the error section is a part that specifies words (at least one recognition unit) recognized before and after the error section based on the error section specified by the error section specifying unit 240. In the following, the case where only one word is used will be described as an example. FIG. 5A shows a conceptual diagram when one recognition unit (context before and after the error interval) recognized before and after the error interval is designated. As shown in FIG. 5A, the speech section of the word before the error section and the speech section of the word after the error section are specified before and after the error section of the recognition result.

  The error section feature quantity extraction unit 260 is a part for extracting feature quantity data of the error section (which may include at least one recognition unit before and after) specified by the context specification section 250 before and after the error section from the feature quantity storage unit 230. is there.

  The correction unit 270 is a part that recognizes again the feature amount data extracted by the error section feature amount extraction unit 260. The correction unit 270 performs speech recognition using the acoustic model holding unit 281, the language model holding unit 282, and the dictionary holding unit 283. Further, the correction unit 270 performs speech recognition using the words (front and back contexts) indicated in the preceding and following speech intervals specified by the error interval preceding and following context specifying unit 250 as constraint conditions. FIG. 5B shows a conceptual diagram when the recognition process is performed based on the word specified by the context specifying unit 250 before and after the error section. As shown in FIG. 5B, when the word W1 in the previous section and the word W2 in the subsequent section of the error section are used as constraint conditions, the recognition candidates are limited. Therefore, recognition accuracy can be improved. In the example of FIG. 5B, the recognition candidates can be narrowed down to A to Z, an appropriate candidate can be selected from the narrowed back, and the recognition process can be performed efficiently.

  Further, the correction unit 270 may perform the correction process based on the dependency relationship with the preceding and following words, the utilization form, and the like. For example, the correction unit 270 extracts a plurality of recognition candidates A to Z for the words in the error section, calculates a score for each correction candidate based on the dependency relationship with the preceding and following words W1 and W2, and calculates the score High correction candidates may be used as recognition results.

  In addition, the correction unit 270 also includes the word information and the front and rear for specifying the word even when the word W1 in the previous section and the word W2 in the subsequent section are not included in the language model holding unit 282 and the dictionary holding unit 283. Correction processing (re-speech recognition processing) can be performed using word information for specifying the word as a constraint.

  For example, the client device 110 has received part-of-speech information indicating the part-of-speech of the word W1 and the word W2 from the server device 120 as the word information, and the correction unit 270 constrains the part-of-speech information of the word W1 and the word W2. Correction processing is performed as a condition. Thereby, more accurate correction processing, that is, voice recognition processing can be performed. Specifically, out of the word information added to the speech recognition result received by the receiving unit 235, the error section specifying unit 240 extracts word information before and after the error section (or any one), and the correcting section To 270. The correction unit 270 corrects the portion designated with this word information as a constraint condition. The conceptual diagram is shown in FIG. As shown in FIG. 24, part-of-speech information A (for example, a particle) is set as a constraint condition for word W1, and part-of-speech information B (for example, a verb) is set for word W2. The correction unit 270 can perform more accurate speech recognition processing by performing correction processing so as to satisfy the part-of-speech information A and the part-of-speech information B, respectively.

  The word information is not limited to the part-of-speech information, and may be information for specifying a word other than the word such as how to read.

  If the necessary word information is not included in the speech recognition result, the sentence to be recognized is converted into a well-known morphological analysis system (for example, “tea bowl”, “Mecab”), a Japanese dependency analysis tool (for example, “Nanban” Word information can be generated by analyzing using ")". That is, in the modified example of the client device 110 shown in FIG. 25, a word information analysis unit 251 is newly added, and the word information analysis unit 251 is a well-known morphological analysis system, Japanese dependency as described above. It consists of analysis tools and can analyze the speech recognition result. Then, the analyzed result is outputted to the context specifying unit 250 before and after the error section, and the context specifying unit 250 before and after the error section extracts the word information of the words before and after the error section based on the word information and outputs the word information to the correcting unit 270. be able to.

  The processing for generating the word information may be performed by the client device 110 or the server device 120. However, it is more processing amount at the client device 110 to instruct the server device 120 to perform the processing and to receive the processing result. Can be reduced.

  The above processing is particularly effective when the words W1 and W2 are unknown words. An unknown word is a word that is not included in the language model holding unit 282 or the dictionary holding unit 283. For example, the correction unit 270 (unknown word determination unit) determines whether or not the words W and W2 are unknown words. If the words W and W2 are unknown words, the correction unit 270 is included in the recognition result transmitted from the server device 120. Correction processing is performed using the existing word information as a constraint condition.

  In the client device 110, the constraint condition may be registered. That is, in the modified example of the client device 110 shown in FIG. 25, the word in the specified error section and the words before and after (or at least one of) the word, or a set of the word information is used as a restriction condition. You may memorize | store in the condition memory | storage part 285 (restraint condition memory | storage means). As a result, the correction unit 270 is the same as the word in the error section specified by the error section specifying unit 240, and when the preceding and succeeding words are the same, the restriction condition stored in the restriction condition storage unit 285 is stored. Correction processing can be performed according to conditions. Therefore, the process can be performed quickly. That is, even if an unknown word is detected after the next time, the constraint condition can be applied only by reading the constraint condition already registered. Since it is not necessary to newly create a constraint condition, the constraint condition can be set with less processing.

  In addition, according to the result corrected by the correction unit 270, the word probabilities in the error section and the word connection probabilities before and after that may be updated. That is, the connection probabilities are stored in the language model holding unit 282 and the dictionary holding unit 283 that function as connection probability storage means, and the connection probabilities calculated and created by the correction unit 270 each time correction processing is performed appropriately are language models. It may be updated in the holding unit 282 and the dictionary holding unit 283.

  Further, the correction unit 270 determines whether or not the re-recognized recognition result is the same as the recognition result recognized by the server device 120 in this error section, and in that case, the recognition result is sent to the integration unit 280. It is preferable not to display the recognition result on the display unit 290 without outputting.

  Similarly, when a recognition unit is deviated between the recognition result recognized by the correction unit 270 and the recognition result recognized by the server device 120 in this error section, it is similarly determined as a recognition error. It is preferable that the recognition result is not displayed on the display unit 290 without outputting the recognition result to the integration unit 280.

  For example, when the correspondence relationship between the speech section and the recognition result in FIG. 4 is different, more specifically, in the speech section, the frame index is 0-9 as the recognition result in the server device 120. In that case, “905 ( If the frame index is 0-15 or “90555 (Kyumarugogo)” in the re-recognition in the correction unit 270, the voice is indicated. The correspondence between the section and the recognition result is shifted between the recognition result and the re-recognition result. For this reason, it can be determined as a recognition error. In that case, the correction unit 270 performs processing such as not outputting so that the recognition result is not displayed on the display unit 290.

  Further, when the receiving unit (not shown) that receives character information from the user can determine that the above-described recognition error has occurred, the correcting unit 270 receives the received character ( For example, the recognition result in the error section may be corrected using the kana) as a constraint condition. That is, when there is some character input for the recognition result of the error section, the recognition processing in the remaining part may be performed on the premise of the character. In this case, when the recognition error is determined, the character input can be accepted by the accepting unit.

  Note that the correction unit 270 prevents erroneous recognition again by performing a voice recognition process different from the recognition process performed in the server device 120. For example, the recognition process is performed by changing the acoustic model, the language model, and the dictionary.

  The acoustic model holding unit 281 is a database that stores phonemes and their spectra in association with each other. The language model holding unit 282 is a part that stores statistical information indicating the chain probability of words, characters, and the like. The dictionary holding unit 283 holds a database of phonemes and texts, and stores, for example, an HMM (Hidden Marcov Model).

  The integration unit 280 is a part that integrates the text data outside the error section and the text data re-recognized by the correction unit 270 in the speech recognition result received by the reception unit 235. The integration unit 280 integrates the text data re-recognized by the correction unit 270 according to an error section (time information) indicating a position where the text data is integrated.

  The display unit 290 is a part that displays text data obtained by integration in the integration unit 280. The display unit 290 is preferably configured to display and recognize the result recognized by the server device 120. In addition, when the result of re-recognition by the correction unit 270 and the recognition result recognized by the server device 120 in the error section are the same, it is preferable to display the recognition result so as not to be displayed. In such a case, it may be displayed that the recognition is impossible. Further, even if the time information is deviated between the recognition result obtained by re-recognizing in the correction unit 270 and the recognition result obtained by recognizing in the server device 120, it is displayed because it may be erroneous. It is preferable to display that the recognition is not possible.

  Further, it is not always necessary to execute the re-recognition process, and it may be determined whether to execute the re-recognition process according to the length of the error interval. For example, when the error section is one character, the re-recognition process is not executed, and correction by another method such as character input is performed.

  An operation of the client apparatus 110 configured as described above will be described. FIG. 6 is a flowchart showing the operation of the client device 110. The feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104).

  Next, voice recognition is performed in the server device 120, and the recognition result is transmitted from the server device 120 and received by the receiving unit 235 (S105). Then, an error section is specified by the error section specifying unit 240 from the speech recognition result, and the preceding and following contexts are specified based on the specified error section (S106). Based on the error section including the preceding and following contexts, the feature data is extracted from the feature storage unit 230 by the error section feature extraction unit 260 (S107). Based on the feature amount data extracted here, speech correction is performed again by the correction unit 270, and text data in the error section is generated (S108). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and the text data obtained by being correctly recognized is displayed on the display unit 290 (S109).

  Next, the processing in S106 to S108 will be described in more detail. FIG. 7 is a flowchart showing the detailed processing. This will be described with reference to FIG.

  An error section is specified by the error section specifying unit 240 based on the recognition result (S201 (S106)). Based on this error section, the word section W1 before the error section (FIG. 5A) is specified and stored by the context specifying section 250 before and after the error section (S202). Further, the word section W2 (FIG. 5 (a)) after the error section is specified and stored by the context section before and after the error section 250 (S203). Next, the start time T1 (FIG. 5A) of the word W1 is specified by the context specifying unit 250 before and after the error section (S204), and the end time T2 of the word W2 (FIG. 5A) is specified. Are stored (S205).

  In this way, the feature data of the section from the start time T1 to the end time T2, which is the error section obtained by adding one word before and after (one recognition unit) to the error section, is the error section feature quantity extraction unit 260. (S206 (S107)). Setting of the constraint condition with the word W1 as the start point and the word W2 as the end point is performed in the correction unit 270 (S207). Then, according to this constraint condition, recognition processing for the feature data by the correction unit 270 is performed, and correction processing is executed (S208).

  As described above, the operation and effect of the client device 110 according to this embodiment will be described. In the client device 110, the feature amount calculation unit 210 calculates the feature value data of the input voice, and the feature amount compression unit 220 transmits the feature amount data to the server device 120 that is a speech recognition device. On the other hand, the feature amount storage unit 230 stores feature amount data.

  The server device 120 performs recognition processing, and the reception unit 235 receives the recognition result from the server device 120. The error interval specification unit 240 specifies an error interval in which a recognition error has occurred in the received recognition result. The error interval specification unit 240 can make a determination based on the reliability. Then, the error section feature value extraction unit 260 extracts feature value data of the error section, and the correction unit 270 performs correction processing by performing re-recognition processing on the recognition result in the extracted error section. That is, the integration unit 280 integrates the re-recognized result and the recognition result received by the receiving unit 235 to perform a correction process, and the display unit 290 can display the corrected recognition result. it can. Accordingly, since a necessary part of the recognized result is corrected, an error in speech recognition can be easily corrected, and a correct recognition result can be obtained. For example, up to 70% of erroneous words can be reduced. Moreover, 60% or more of errors due to unknown words can be corrected. The reliability may be received from the server device 120 or calculated by the client device 110.

  Further, the client device 110 can perform correction processing (re-recognition processing) in accordance with the constraint conditions using the context specifying unit 250 before and after the error section. That is, it is possible to obtain a more accurate recognition result by fixing words before and after the error section and performing recognition processing according to the fixed words.

  In the present embodiment or other embodiments described later, the first recognition process is performed by the server apparatus 120. However, the present invention is not limited to this, and the first recognition process is performed by the client apparatus 110. The second recognition process may be performed in the server device 120. At that time, naturally, the error section designation processing and the like are performed in the server apparatus 120. For example, in this case, the client device 110 includes a recognition processing unit that performs a recognition process based on the feature amount data calculated by the feature amount calculation unit 210, and the transmission unit 225 includes the recognition result and the feature here. The amount data is transmitted to the server device 120.

  The server apparatus 120 includes units corresponding to an error section specifying unit 240, an error section pre- and post-error section specifying unit 250, a feature amount storing unit 230, an error section feature amount extracting unit 260, and a correcting unit 270 in the client device 110. The feature amount data transmitted from the device 110 is stored in the feature amount storage unit, the error section is specified based on the recognition result, and the context before and after the error section is specified. Based on these, the feature amount stored previously is stored. Data correction processing (recognition processing) is performed. The recognition result processed in this way is transmitted to the client device 110.

  In this embodiment or other embodiments described below, re-recognition (correction processing) is performed using the constraint conditions determined by the context specifying unit 250 before and after the error interval. Only feature data is used. Re-recognition processing may be performed without using such a constraint condition.

  Moreover, it is preferable to change the recognition method in the server apparatus 120 and the recognition method in this embodiment (or other embodiments shown below). That is, since it is necessary for the server apparatus 120 to recognize the voices of an unspecified number of users, it is necessary to be general-purpose. For example, the number of models and the number of dictionaries in the acoustic model holding unit, language model holding unit, and dictionary holding unit used in the server device 120 are large, the number of phonemes is increased in the acoustic model, and the number of phonemes is increased in the language model. The number of models and the number of dictionaries are made large, such as increasing the number of words, so that it can handle all users.

  On the other hand, the correction unit 270 in the client device 110 does not need to correspond to any user, and uses an acoustic model, a language model, and a dictionary that match the voice of the user of the client device 110. Therefore, the client device 110 needs to update each model and dictionary as appropriate with reference to correction processing, recognition processing, and character input processing at the time of mail creation.

  The client device 110 further includes a display unit 290 that displays the recognition result corrected by the correction unit 270, and the display unit 290 does not display the recognition result recognized by the server device 120. Thereby, since the recognition result which may have a recognition error is not displayed, a misunderstanding is not given to a user.

In addition, the client device 110 detects that the recognition result obtained by re-recognition in the correction unit 270 is the same as the recognition result received by the reception unit 235, or the time information included in each of the recognition results is shifted. If the error occurs, the correction unit 270 determines that the recognition error has occurred, and the display unit 290 does not display the recognition result. Thereby, it can prevent displaying an incorrect recognition result. Specifically, up to 70% of erroneous words can be reduced. Moreover, 60% or more of errors due to unknown words can be corrected.

<Second Embodiment>
Next, a description will be given of the client device 110a configured so that the user manually determines the error interval without automatically determining the error interval based on the reliability. FIG. 8 is a block diagram illustrating a function of the client device 110a that accepts an error interval by a user input. As shown in FIG. 8, the client device 110a includes a feature amount calculation unit 210, a feature amount compression unit 220, a feature amount storage unit 230, a transmission unit 225, a reception unit 235, an operation unit 236, a result storage unit 237, a user input Detection section 238, error section specifying section 240a, error section front and rear context specifying section 250, error section feature quantity extracting section 260, correcting section 270, integrating section 280, acoustic model holding section 281, language model holding section 282, dictionary holding section 283 The display unit 290 is included. The client device 110a is realized by the hardware shown in FIG.

  This client device 110a is different from the client device 110 in that it includes an operation unit 236, a result storage unit 237, a user input detection unit 238, and an error section specification unit 240a. Hereinafter, this difference will be mainly described.

  The operation unit 236 is a part that receives user input. The user can specify an error section while confirming the recognition result displayed on the display unit 290. The operation unit 236 can accept the designation.

  The result storage unit 237 is a part that stores the speech recognition result received by the reception unit 235. The stored speech recognition result is displayed on the display unit 290 so that the user can visually recognize it.

  The user input detection unit 238 is a part that detects a user input received by the operation unit 236, and outputs the input error interval to the error interval specification unit 240a.

  The error section designation unit 240a is a part that designates the section according to the error section input from the user input detection unit 238.

  Next, processing of the client device 110a configured as described above will be described. FIG. 9 is a flowchart showing processing of the client device 110a. The feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104).

  Next, voice recognition is performed in the server device 120, the recognition result is transmitted from the server device 120, received by the receiving unit 235, temporarily stored, and the recognition result is displayed on the display unit 290 (S105a). ). And a user judges an error area based on the recognition result currently displayed on the display part 290, and inputs the error area. Then, the input is detected by the user input detection unit 238, and the error interval is specified by the error interval specification unit 240. Then, the preceding and following contexts are designated based on the designated error section (S106a). Based on the error interval including the preceding and following contexts, feature amount data is extracted by the error interval feature amount extraction unit 260 (S107), speech recognition is performed again by the correction unit 270, and text data in the error interval is generated. (S108). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and correct text data is displayed on the display unit 290 (S109).

  Next, the processing in S105a to S108 described above will be described in more detail. FIG. 10 is a flowchart showing detailed processing when an error interval is designated by user input in the client apparatus 110a.

  The recognition result is received by the receiving unit 235 and displayed on the display unit 290 (S301). While confirming the recognition result displayed on the display unit 290, the user designates an error interval, and the user input detection unit 238 detects the start point of the error interval and temporarily stores it (S302). Then, the preceding word section W1 of the error section is specified and stored by the context section before and after the error section 250 (S303), and the start time T1 of the stored word W1 is specified and stored (S304).

  Further, the end point of the error section is detected by the user input detection unit 238 according to the user designation and temporarily stored (S305). Then, the word W2 after the error section is specified and stored by the context specifying unit 250 before and after the error section (S306), and the end time T2 of the stored word W2 is specified and stored (S307).

  After these processes, the feature amount data from the start time T1 to the end time T2 is extracted by the error section feature amount extraction unit 260 (S308). Setting of the constraint condition with the word W1 as the start point and the word W2 as the end point is performed in the correction unit 270 (S309). Then, according to this constraint condition, a recognition process for the feature data by the correction unit 270 is performed, and the correction process is executed (S310).

  By such processing, an error interval by user input can be designated, and thereby the recognition result can be corrected by re-recognizing.

In such a client device 110a, the display unit 290 displays the recognition result, and the user visually recognizes the recognition result, and the user designates an error section, that is, a portion to be corrected by operating the operation unit 236. Can do. Accordingly, since a necessary part of the recognized result is corrected, correction processing can be easily performed and a correct recognition result can be obtained.

<Third Embodiment>
Next, a description will be given of the client device 110b that can correctly specify an error interval when the recognition result transmitted from the server device 120 does not include time information. FIG. 11 is a block diagram showing functions of the client device 110b. The client device 110b includes a feature amount calculation unit 210, a feature amount compression unit 220, a transmission unit 225, a feature amount storage unit 230, a reception unit 235, a time information calculation unit 239, an error section specification unit 240, and an error section feature amount extraction unit. 260, an error interval context designation unit 250, a correction unit 270, an acoustic model holding unit 281, a language model holding unit 282, and a dictionary holding unit 283. This client device 110b is realized by the hardware shown in FIG. 3 as with the client device 110 of the first embodiment.

  Further, the difference from the client device 110 of the first embodiment is that the client device 110b receives a recognition result that does not include progress information from the server device 120, and the time information calculation unit 239 recognizes the result. The point is to automatically calculate the elapsed time (frame index) based on the text data. Hereinafter, the client device 110b will be described focusing on this difference.

  The time information calculation unit 239 is a part that calculates the elapsed time in the text data using the text data and the feature amount data stored in the feature amount storage unit 230 among the recognition results received by the reception unit 235. More specifically, the time information calculation unit 239 compares one word or one recognition unit of the text data by comparing the input text data with the feature amount data stored in the feature amount storage unit 230. When converted to frequency data, it is possible to calculate the elapsed time in the text data by determining to what part of the feature data matches. For example, if there is a match with one word of text data up to 10 frames of feature amount data, the one word has an elapsed time of 10 frames.

  The error section specifying unit 240b can specify an error section using the elapsed time and text data calculated by the time information calculating unit 239. The error interval designating unit 240b determines an error interval based on the reliability information included in the recognition result. Note that an error interval may be designated by user input as in the second embodiment.

  As described above, based on the error section specified by the error section specifying unit 240b, the context section specifying section 250 before and after the error section specifies an error section including the preceding and following contexts, and the feature extracting section 260 is the error section feature amount extraction section 260. The voice data of the section is extracted, and the correction unit 270 can perform the correction process by performing the recognition process again.

  Next, processing of the client device 110b will be described. FIG. 12 is a flowchart showing the processing of the client device 110b. The feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104).

  Next, voice recognition is performed in the server device 120, and the recognition result (not including the elapsed time) is transmitted from the server device 120 and received by the receiving unit 235 (S105). Then, the elapsed time is calculated by the time information calculation unit 239 from the voice recognition result and the feature amount data of the feature amount storage unit 230, and the error section is specified by the error section specifying unit 240 using the elapsed time and the voice recognition result. Is done. The preceding and following context is specified based on the specified error section by the error section preceding and following context specifying unit 250 (S106b). Based on the error sections including the preceding and following contexts, the feature data is extracted by the error section feature quantity extraction unit 260 (S107), the speech recognition is performed again by the correction unit 270, and the text data in the error section is generated. (S108). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and correct text data is displayed on the display unit 290 (S109).

  Next, further detailed processing including S106b will be described. FIG. 13 is a flowchart showing detailed processing from S105 to S108.

  The reception unit 235 receives a recognition result that does not include the elapsed time (S401), and the time information calculation unit 239 calculates the elapsed time in the text data (S402). An error section is specified from the recognition result by the error section specifying unit 240 (S403). Based on this error section, the word section W1 before the error section (FIG. 5A) is specified and stored by the context specifying unit 250 before and after the error section (S404). Further, the word section W2 (FIG. 5A) after the error section is specified and stored by the context section before and after the error section 250 (S405). Next, the start time T1 (FIG. 5A) of the word W1 is specified by the context specifying unit 250 before and after the error section (S406), and the end time T2 of the word W2 (FIG. 5A) is specified. (S407).

  The error section feature quantity extraction unit 260 extracts the feature quantity data of the section from the start time T1 to the end time T2, which is an error section obtained by adding one word before and after the error section in this way. S408). Setting of the constraint condition with the word W1 as the start point and the word W2 as the end point is performed in the correction unit 270 (S409). Then, according to this constraint condition, recognition processing for the feature data by the correction unit 270 is performed, and correction processing is executed (S410).

According to the client device 110b, the time information calculation unit 239 calculates the elapsed time in the recognition result based on the recognition result received by the reception unit 235 and the feature amount data stored in the feature amount storage unit 230. . Then, the error interval specification unit 240 can specify an error interval based on this time information. It is possible to specify the context before and after the error section specified here, and to perform correction processing based on the feature amount data. As a result, an appropriate error interval can be designated even when the recognition result does not include time information.

<Fourth Embodiment>
Next, the client device 110c that performs the correction process only from the recognition result obtained by the voice recognition in the server device 120 will be described. FIG. 14 is a block diagram illustrating functions of the client device 110c. The client device 110c includes a feature amount calculating unit 210, a feature amount compressing unit 220, an error section specifying unit 240, an error section pre- and post-context specifying unit 250, a correcting unit 270a, and a language DB holding unit 284. The client device 110c is realized by the hardware shown in FIG.

  Compared to the client device 110, the client device 110c is configured not to store feature amount data obtained by voice input, and to not use the feature amount data again during correction processing. Specifically, the feature amount storage unit 230, the error section feature amount extraction unit 260, the acoustic model holding unit 281, the language model holding unit 282, and the dictionary holding unit 283 are not provided. Is different. Hereinafter, description will be made based on the differences.

  The feature amount calculation unit 210 calculates feature amount data from the voice input, and the feature amount compression unit 220 compresses the feature amount data and transmits it to the server device 120. Then, the receiving unit 235 receives the recognition result from the server device 120. The error section specifying unit 240 specifies an error section by reliability information or a user operation, and the error section pre- and post-context specifying unit 250 specifies the pre- and post-context and specifies the error section.

  The correction unit 270a performs a conversion process on the text data specified by the error section including the preceding and following contexts based on the database stored in the language DB holding unit 284. The language DB holding unit 284 stores almost the same information as the language model holding unit 282, and stores the chain probability for each syllable.

  Further, the correction unit 270a lists word strings w (Wi, Wi + 1... Wj) that may appear in the error section. Here, the number of word strings w may be limited to K. The limit number K is the same as or close to the number P of error words (K = P-c to P + c).

Further, the correction unit 270a calculates the likelihood when all the word strings listed are limited to the preceding and following words W1 and W2. That is, the likelihood DB is obtained using the following equation (1) using the language DB stored in the terminal for all W sequences.
Likelihood P (w1 w w2) = P (W1, Wi, Wi + 1 ... Wj, W2) = P (W1) * P (Wi / W1) ... * P (W2 / Wj) of word string (W1 w W2) (1)

Further, the distance between the word string in the error section and the candidate is calculated, and this distance may be added. In this case, the following formula (2) is obtained.
Likelihood P (w1 w w2) of word string (W1 w W2) = P (W1, Wi, Wi + 1 ... Wj, W2) * P (Wi, Wi + 1 ... Wj, Werror) (2)
P (Wi, Wi + 1... Wj, Werror) indicates the distance between the error word string Werror and the candidate strings Wi, Wi + 1.

  P (Wn / Wm) in this equation is for a Bi-gram in the N-gram model, and represents the probability that Wn appears after Wm. Here, a Bi-gram example will be described, but other N-gram models may be used.

  The integration unit 280 integrates the text data thus converted by the correction unit 270a with the text data in the received recognition result, and the display unit 290 displays the integrated and corrected text data. Prior to the integration, the candidates sorted using the likelihood calculated by the correction unit 270a may be listed and selected by the user, or the candidate having the highest likelihood is automatically determined. You may make it do.

  Next, processing of the client device 110c configured as described above will be described. FIG. 15 is a flowchart showing processing of the client device 110c. The feature amount data is calculated by the feature amount calculation unit 210 based on the voice data input by voice, and the feature amount data compressed by the feature amount compression unit 220 is transmitted to the server device 120 (S502).

  The recognition result recognized by the server device 120 as a voice is received by the receiving unit 235 (S502), and an error section is specified by the error section specifying unit 240 (S503). The designation of the error section here may be based on reliability or may be designated by user input.

  Subsequently, the context before and after the error section (word) is specified by the error section front and rear context specifying unit 250 (S504). Then, re-conversion processing is performed by the correction unit 270a, and error zone candidates are listed up at that time (S505). Here, the likelihood of each candidate is calculated by the correction unit 270a (S506), a sorting process based on the likelihood is performed (S507), and the candidate group subjected to the sorting process is displayed on the display unit 290 (S508). .

In the client device 110 c, the feature amount calculation unit 210 calculates feature amount data from the input voice, the feature amount compression unit 220 compresses the feature amount data, and the transmission unit 225 transmits this to the server device 120. In the server device 120, voice recognition is performed, and the reception unit 235 receives the recognition result. Then, the correction unit 270a performs correction processing based on the error section specified by the error section specifying unit 240 and the context specifying unit 250 before and after the error section. After the integration process by the integration unit 280, the display unit 290 displays the corrected recognition result. Accordingly, since a necessary part of the recognized result is corrected, an error in speech recognition can be easily corrected, and a correct recognition result can be obtained. In this embodiment, as compared with the first embodiment, the feature data is not stored, and the feature data is not used in the re-recognition process, and the configuration is simplified. Can do.

<Fifth Embodiment>
Next, a mode in which the first speech recognition and the second speech recognition are performed in the client device 110d instead of the distributed processing in which the server device 120 performs speech recognition will be described.

  FIG. 16 is a block diagram illustrating a functional configuration of the client apparatus 110d. The client device 110d includes a feature amount calculation unit 210, a first recognition unit 226 (acquisition means), a language model storage unit 227, a dictionary storage unit 228, an acoustic model storage unit 229, a feature amount storage unit 230, an error section specification unit 240, It includes a context designation unit 250 before and after an error section, an error section feature quantity extraction unit 260, a correction unit 270, an acoustic model storage unit 281, a language model storage unit 282, a dictionary storage unit 283, an integration unit 280, and a display unit 290. Yes. The client device 110d is realized by the hardware shown in FIG.

  The client device 110d is not configured to communicate with the server device 120 with the client device 110 of the first embodiment, and the first recognition unit 226, language model holding unit 227, dictionary holding unit 228, acoustic The difference is that a model holding unit 229 is provided. Hereinafter, the difference will be mainly described.

  The first recognition unit 226 performs speech recognition on the feature amount data calculated by the feature amount calculation unit 210 using the language model holding unit 227, the dictionary holding unit 228, and the acoustic model holding unit 229.

  The language model holding unit 227 is a part that stores statistical information indicating the chain probability of words, characters, and the like. The dictionary holding unit 228 holds a database of phonemes and texts, and stores, for example, an HMM (Hidden Marcov Model). The acoustic model holding unit 229 is a database that stores phonemes and their spectra in association with each other.

  The error section specifying unit 240 inputs the recognition result recognized by the first recognition unit 226 described above, and specifies an error section. The error section pre- and post-context specifying unit 250 specifies the context before and after the error section, and the error section feature quantity extraction unit 260 extracts feature data of the error section including the front and back context. Then, the correction unit 270 performs recognition processing again based on the feature amount data. The correction unit 270 functions as a second recognition unit.

  When the integration process by the integration unit 280 is performed, the display unit 290 can display the corrected recognition result.

  Next, the operation of the client device 110d will be described. FIG. 17 is a flowchart showing processing of the client device 110d. The feature amount data of the voice input by the feature amount calculation unit 210 is calculated (S601), and the calculated feature amount data is stored in the feature amount storage unit 230 (S602). In parallel with this storage processing, the first recognition unit 226 performs voice recognition (S603).

  The error section of the recognition result recognized by the first recognition unit 226 is specified by the error section specifying unit 240 and the context section specifying unit 250 before and after the error section (S604). The feature amount data of the designated error section (including the preceding and following contexts) is extracted from the feature amount storage unit 230 by the error section feature amount extraction unit 260 (S605). Then, the voice in the error section is recognized again by the correction unit 270 (S606). The recognition result recognized here is integrated by the integration part 280, and a recognition result is displayed on the display part 290 (S607).

  Thus, since the recognition process is performed by the first recognition unit 226 and the second recognition unit (correction unit) 270 in the client device 110d, more accurate voice recognition can be performed. It is desirable that the first recognition unit 226 and the second recognition unit have different recognition methods. Thereby, even the voice that has not been recognized by the first recognizing unit 226 can be supplemented by the second recognizing unit 270, and an accurate voice recognition result as a whole can be expected.

According to the client device 110d, feature amount data is calculated from the voice input in the feature amount calculation unit 210, and is stored in the feature amount storage unit 230. On the other hand, the first recognizing unit 226 performs speech recognition processing based on the feature amount data, and the error section specifying unit 240 and the error section pre- and post-context specifying unit 250 have a recognition error in the recognized recognition result. Specify the error interval. Then, the correction unit 270 (second recognition unit) corrects the recognition result in the designated error section. Accordingly, since a necessary part of the recognized result is corrected, correction processing can be easily performed and a correct recognition result can be obtained. Further, by performing the recognition process twice in the client device 110d, it is not necessary to use the server device 120.

<Sixth Embodiment>
Next, a sixth embodiment, which is a modification of the second embodiment, will be described. This embodiment is characterized in that the end point of the error section is automatically determined.

  FIG. 18 is a block diagram illustrating a functional configuration of the client device 110f according to the sixth embodiment. The client device 110f includes a feature amount calculation unit 210, a feature amount compression unit 220, a feature amount storage unit 230, a transmission unit 225, a reception unit 235, an operation unit 236, a result storage unit 237, a user input detection unit 238, and an error section specification unit. 240c, end point judgment unit 241, error section pre- and post-context designation unit 250, error section feature quantity extraction unit 260, correction unit 270, integration unit 280, acoustic model holding unit 281, language model holding unit 282, dictionary holding unit 283, display unit 290. The client device 110f is realized by the hardware shown in FIG.

  The client device 110f is different from the second embodiment in that the error section specifying unit 240c receives only the start point of the error section, and the end point determination unit 241 determines the end point of the error section based on a predetermined condition. . Hereinafter, based on the block diagram shown in FIG. 18, it demonstrates centering on difference with 2nd Embodiment.

  Similarly to the configuration shown in the second embodiment, in the client device 110f, the reception unit 235 receives the recognition result recognized by the server device 120, and the result storage unit 237 stores the recognition result. Then, while the display unit 290 displays the recognition result, the user operates the operation unit 236 while viewing the recognition result displayed on the display unit 290, thereby specifying the start point of the error section. The user input detection unit 238 detects the start point and outputs it to the error interval specification unit 240c.

  The error section specifying unit 240c specifies an error section according to the start point specified by the user and the end point determined by the end point determination unit 241. In determining the end point of the error section, when the error section specifying unit 240c detects that the start point is specified by the user, the error section specifying unit 240c outputs that fact to the end point determining unit 241 and instructs the end point determination.

  The end point determination unit 241 is a part that automatically determines the end point of the error interval in accordance with an instruction from the error interval specification unit 240c. For example, the end point determination unit 241 compares the reliability information included in the speech recognition result received by the reception unit 25 and stored in the result storage unit 237 with a preset threshold, and the reliability is A word exceeding the threshold (or a word having the highest reliability) is determined as an error end point. Then, the end point determining unit 241 outputs the determined end point to the error section specifying unit 240c, so that the error section specifying unit 240c can specify the error section.

For example, the following voice will be described as an example. Here, for the sake of convenience, it is assumed that “activation” is designated as the start point of the error section.
<Voice content>
“Your cooperation is needed to achieve this goal.”
<Voice recognition result>
“Your cooperation is needed to revitalize this goal.”
Here, the speech recognition result is divided into words. "/" Indicates a word break.
“This / Goal / To / Activate / To / To / To / To / Everyone / Ms. / No / Cooperation / To / Need /”
As a result of the speech recognition, the reliability of “activation” is 0.1, the reliability of “no” is 0.01, the reliability of “for” is 0.4, and the reliability of “ni” is 0.6. In the case where the threshold value is 0.5, it is possible to determine that “ni” in “activation / for / for / ni” is the end point.

  The end point determination unit 241 can also determine the previous point (in the above example, “because”) of the word having a reliability equal to or higher than the threshold as the end point. Any method can be used as long as it includes an erroneous part.

  Such an error interval designation method is convenient because it is in line with the user's usual correction habits. That is, for example, in Kanji conversion, when a user designates an error, it is customary to first input a starting point, then delete the error, and input a correct word string. Since the end point is automatically determined after inputting the viewpoint, the above-described error section designation method is also suitable for the operation method, and can be operated without a sense of incongruity for the user.

  Further, the end point determination unit 241 is not limited to the above-described method when determining the end point. For example, a method of determining the end point according to a specific phonetic symbol, or a method of setting the Mth word as the end point after the start of the error start point may be used. Here, the method according to a specific phonetic symbol is a method of judging based on a pause during utterance, a short pause (reading mark) that appears at the boundary of a phrase, and a long pause (punctuation) that appears at the end of an utterance. You may make it judge based on. As a result, determination is made at sentence breaks, and more accurate speech recognition can be expected.

Specific examples are shown below. The following will be described by way of example with the same contents as described above as speech.
<Voice content>
“Your cooperation is needed to achieve this goal.”
<Voice recognition result>
“Your cooperation is needed to revitalize this goal.”

  When the user operates the operation unit 236 to set “after this target” as the start point of the error section, the end point determination unit 241 determines the pose (reading point portion) closest to this portion as the end point. To do. The error interval specification unit 240c can specify an error interval based on this end point. In the above-described example, the part “,” in “For” is designated as the end point of the error section. Note that the “,” part is in fact a state where there is no voice and there is a moment.

  In addition to the punctuation marks and punctuation marks, the specific pronunciation may be a pronunciation such as “e ~” or “that ~” or a word such as “masu” or “is”.

Next, an example of a method of setting the Mth word of the error start point transition as the end point is shown. The text shown below shows a state divided into words. "/" Indicates a word break.
“This / Goal / To / Activate / To / To / To / To / Everyone / Ms. / No / Cooperation / To / Need /”

  For example, when the starting point is “activated” and M = 3, “for” in “activated / not / for” becomes the end word. Therefore, the error section specifying unit 240c can specify “activation / no / for” as the error section. Of course, it may be other than M = 3.

Next, an example of a method of setting a word with a small number of recognition result candidates (competition number) as an end point will be described. For example, it demonstrates using the following examples.
In “this / target / do / activate / for / begin”, the following candidates are listed.
“Activation”: “Who”, “Many”, “Recommended”
“No”: “ka”, “is”
"For":-(No candidate)

  The number of candidates reflects the ambiguity of the section. The lower the reliability, the more candidates are transmitted from the server device 120. In this example, the server apparatus 120 is configured to transmit other candidates obtained based on the reliability information to the client apparatus 110 as they are, instead of transmitting the reliability information.

  In this case, since there is no candidate for “for”, it can be considered that the reliability is high. Therefore, in this example, “no” in front of the error section can determine the end point of the error section. It should be noted that the end point of the error section is not limited to the end of the error section, but may have a certain width.

  As described above, there are a method based on the reliability of the end point, a method using a specific phonetic symbol (or pronunciation), and a method of setting an error interval from the start point to the Mth, but a combination of these methods, that is, these It is good also as a format which selects one from the N-best format or the recognition result of multiple methods for the correction result of multiple methods. In this case, the recognition results may be displayed in a list in the order of the recognition result scores, and the user may select an arbitrary recognition result from the list.

  In this way, based on the error section specified by the error section specifying unit 240c, the context specifying unit 250 before and after the error section specifies a section including the preceding and following sections, and the error section feature amount extracting unit 260 converts the feature amount data into the feature amount. Extracted from the storage unit 230, the correction unit 270 performs correction processing by performing re-recognition processing on the feature amount data.

  Next, the operation of the client device 110f configured as described above will be described. FIG. 19 is a flowchart showing processing of the client device 110f.

  The feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104).

  Next, voice recognition is performed in the server device 120, the recognition result is transmitted from the server device 120, received by the receiving unit 235, temporarily stored, and the recognition result is displayed on the display unit 290 (S105a). ). Then, the user determines the start point of the error section based on the recognition result displayed on the display unit 290 and designates the start point by operating the operation unit 236. When the user input detection unit 238 detects that the start point has been designated, the end point determination unit 241 automatically determines the end point of the error section. For example, a determination is made based on the reliability included in the speech recognition result, and a place where a predetermined phonetic symbol appears is determined as an end point. Further, the Mth from the start point (M is a predetermined arbitrary value). Value) is determined as the end point.

  In this way, the start point and the end point are specified by the error section specifying unit 240c. Then, the preceding and following contexts are designated based on the designated error section (S106c). Based on the error interval including the preceding and following contexts, feature amount data is extracted by the error interval feature amount extraction unit 260 (S107), speech recognition is performed again by the correction unit 270, and text data in the error interval is generated. (S108). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and correct text data is displayed on the display unit 290 (S109).

  In addition, about the process of S105a-108 including S106c, the process substantially the same as the flowchart shown in FIG. 10 is performed, but the end point judgment part 241 automatically judges the end point part of an error area about the process of S305. The difference in saving it.

As described above, according to this embodiment, such an error section designation method can be adapted to the user's usual correction habits, and an apparatus that is very easy to use can be provided.

<Seventh Embodiment>
Next, a seventh embodiment will be described. According to this embodiment, when the user designates the first character in the error section, voice recognition is performed more correctly using the designated character as a constraint condition.

  FIG. 20 is a block diagram illustrating a functional configuration of the client apparatus 110g according to the seventh embodiment. The client device 110g includes a feature amount calculation unit 210, a feature amount compression unit 220, a feature amount storage unit 230, a transmission unit 225, a reception unit 235, an operation unit 236, a result storage unit 237, a user input detection unit 238, and an error section specification unit. 240a, an error interval pre- and post-context specifying unit 250a, an error interval feature amount extraction unit 260, a correction unit 270, an integration unit 280, an acoustic model storage unit 281, a language model storage unit 282, a dictionary storage unit 283, and a display unit 290. Has been. The client device 110g is realized by the hardware shown in FIG.

  In this client device 110g, the operation unit 236 receives a corrected character in the error section as a constraint condition from the user, and the context specification unit 250a before and after the error section receives the context and the corrected text received in the operation unit 236 before and after the error section. Characters are designated, and the correction unit 270 is characterized in that correction processing is performed by performing re-recognition processing using the context before and after the error section and the corrected character as a constraint condition.

  That is, the operation unit 236 receives an input for designating an error section from the user, and then receives a corrected character input in the error section.

  The context specifying unit 250a before and after the error section performs substantially the same processing as the context specifying unit 250 before and after the error section in the first embodiment, and specifies a word (one recognition unit) recognized before and after the error section. The corrected character received by the operation unit 236 is designated.

  The correction unit 270 can perform the re-recognition processing based on the feature amount data extracted by the error section feature amount extraction unit 260 and the constraint conditions specified by the context specifying unit 250a before and after the error section, and can execute the correction processing. .

For example, the above process will be described based on the following example.
<Voice content>
“Your cooperation is needed to achieve this goal.”
<Voice recognition result>
“Your cooperation is needed to revitalize this goal.”
In this case, the user operates the operation unit 236 to input the correct character content at the start point in the error section (in the above example, the position next to “this target”). The kana string to be entered is “for the sake of convenience”. In the following example, a case where “TA”, which is a part of the head of input, is input will be described as an example. It is assumed that the start point and end point of the error section have been determined or determined by the same method as described above.

  When the user inputs “TA” via the operation unit 236, the error interval pre- and post-context specifying unit 250a uses “this target” as the pre- and post-context and “TA” as the input character, that is, “this” “Target” is set as a constraint condition when recognizing feature amount data.

  Thus, by presenting the user with a recognition result obtained by performing speech recognition again using the user's character input content as a constraint condition, a more accurate recognition result can be presented. The correction method may be used in combination with the key character input method in addition to the voice recognition. For example, kana-kanji conversion can be considered as a key character input method. Kana-Kanji conversion has a function of comparing input character contents with a dictionary and predicting the conversion result. For example, if “ta” is entered, the word “ta” from the database is listed in order and presented to the user.

  Here, by using this function, a list of candidates for the kana-kanji conversion database and candidates obtained by speech recognition are displayed in a list, and the user may select an arbitrary candidate based on these lists. Good. The order displayed in the list may be the order of the scores given to the conversion result or recognition result, or the candidate based on kana-kanji conversion is compared with the candidate based on speech recognition, and is completely or partially matched. The candidates may be added in the order of their scores, and the order may be based on the scores. For example, in the case of a kana-kanji conversion candidate A1 “achieve” score 50 and a speech recognition result candidate B1 “achieve” score 80, the candidate A1 and the candidate B1 partially match each other. In FIG. 5, the predetermined coefficient may be multiplied and displayed based on the score obtained by addition. In the case of a perfect match, there is no need to perform adjustment processing such as multiplication by a predetermined coefficient. In addition, when the user selects Kana-Kanji conversion candidate A1 “achievement”, “reach this goal” is used as a constraint, and feature data corresponding to the remaining “yes” that has not yet been determined is recognized again. In this way, the candidate list may be displayed again.

  Next, the operation of the client device 110g configured as described above will be described. FIG. 21 is a flowchart showing the processing of the client device 110g.

  The feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104).

  Next, voice recognition is performed in the server device 120, the recognition result is transmitted from the server device 120, received by the receiving unit 235, temporarily stored, and the recognition result is displayed on the display unit 290 (S105a). ). Then, the user designates an error section based on the recognition result displayed on the display unit 290 (S106d). Further, the user inputs characters to the operation unit 236 to correct the recognition result in the error section. When the operation unit 236 accepts character input, the operation unit 236 outputs the input to the preceding and following context specifying unit 250a. The preceding and following context specifying unit 250a determines the preceding and following contexts based on the specified error section together with the input characters. It is specified. Based on the error interval including the preceding and following contexts, feature amount data is extracted by the error interval feature amount extraction unit 260 (S107), speech recognition is performed again by the correction unit 270, and text data in the error interval is generated. (S108). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and correct text data is displayed on the display unit 290 (S109).

  In addition, about the process of S105a-108 containing S106d, the process substantially the same as the flowchart shown in FIG. 10 is performed. Further, in the present embodiment, in addition to the processes in the flowchart of FIG. 10, in S309, a process for setting the character received by the operation unit 236 as a constraint condition is required. In addition, it is necessary to complete the input acceptance of the character which becomes a constraint condition by S309.

As described above, according to this embodiment, more accurate voice recognition can be performed by setting a character designated by the user in addition to the preceding and following contexts as a constraint condition.

<Eighth Embodiment>
Next, an eighth embodiment will be described. According to this embodiment, as a result of re-recognition by the correction unit 270, the same recognition result as that before the re-recognition is prevented.

  FIG. 22 is a block diagram illustrating a functional configuration of the client apparatus 110h according to the eighth embodiment. The client device 110h includes a feature amount calculation unit 210, a feature amount compression unit 220, a feature amount storage unit 230, a transmission unit 225, a reception unit 235, an operation unit 236, a result storage unit 237, a user input detection unit 238, and an error section specification unit. 240a, an error interval pre- and post-context specifying unit 250, an error interval feature amount extraction unit 260, a correction unit 270b, an integration unit 280, an acoustic model storage unit 281, a language model storage unit 282, a dictionary storage unit 283, and a display unit 290. Has been. The client device 110h is realized by the hardware shown in FIG. Hereinafter, the description will focus on the differences from the client device 110 in FIG.

  The correction unit 270b is a part that performs re-recognition processing and the like in the same manner as the correction unit 270 in FIG. Further, the correction unit 270b performs re-recognition processing based on the recognition result stored in the result storage unit 237 so as not to make the same recognition error. That is, the correction unit 270b compares the recognition result in the error section specified in the error section specifying unit 240a with the recognition result in the error section in the re-recognition search process so as not to obtain the same recognition result. The process which excludes the path | pass which contains is performed from a candidate. As a process of exclusion, the correction unit 270b does not select a candidate that is minimized as a result of multiplication by a predetermined coefficient so as to minimize the probability of the hypothesis in the candidate for the feature amount data in the error section. Like that. In the above-described method, a candidate (for example, “activation”) that may be erroneous when re-recognizing is excluded from the recognition result candidates. However, the present invention is not limited to this. When presenting the recognized recognition result, one candidate of the recognition result that may be erroneous (for example, “activation”) may not be displayed.

  The client device 110h executes almost the same processing as that in the flowchart shown in FIG. Note that the error interval recognition processing in S108 is different in that recognition processing is performed so that the same recognition result is excluded from the candidates so as not to be displayed.

As described above, since the word to be corrected is an error, the word to be corrected should not be output as the result after re-recognition. You can hide it.

<Ninth Embodiment>
Next, a ninth embodiment will be described. According to this embodiment, in the error section of the feature amount data extracted by the error section feature amount extraction unit 260, an average value is calculated, and re-recognition processing is performed using data obtained by subtracting the average value from the feature amount data. To do.

  The specific configuration will be described. FIG. 23 is a block diagram illustrating functions of the client device 110i according to the ninth embodiment. The client device 110i includes a feature amount calculation unit 210, a feature amount compression unit 220, a feature amount storage unit 230, a transmission unit 225, a reception unit 235, an error section specification unit 240, an error section front and rear context specification unit 250, an error section feature amount. Extraction unit 260, average value calculation unit 261 (calculation unit), feature normalization unit 262 (correction unit), correction unit 270 (correction unit), integration unit 280, acoustic model storage unit 281, language model storage unit 282, dictionary storage A part 283 and a display part 290 are included. The client device 110i is realized by the hardware shown in FIG. Hereinafter, the average value calculation unit 261 and the feature normalization unit 262 that are different from the client device 110 in FIG. 2 will be mainly described.

  The average value calculation unit 261 is a part that calculates an average value of error sections (or an average value including before and after the error section) in the feature amount data extracted by the error section feature amount extraction unit 260. More specifically, the average value calculation unit 261 cumulatively adds the output value (magnitude) for each frequency of each recognition unit in the error interval. Then, an average value is calculated by dividing the output value obtained by cumulative addition by the number of recognition units. For example, the recognition unit in the error section “activation / no / for” is a part delimited by a slash “/”. Assuming that the recognition frame n as each recognition unit is composed of frequencies fn1 to fn12 and the output values thereof are gn1 to gn12, the average value g1 of the frequency f1 = Σgn1 / n (in the above example, from n = 1). 3).

  That is, the frequencies f11 to f112 (output values are g11 to g112) that constitute “activation”, the frequencies f21 to f212 (output values are g21 to g212) that constitute “no”, and the frequencies f31 to f that constitute “for”. When f312 (output values are g31 to g312), the average value of the frequency f1 is calculated as (g11 + g21 + g31) / 3.

  The feature normalization unit 262 performs a process of subtracting the average value of each frequency calculated by the average value calculation unit 261 from the feature amount data composed of each frequency. Then, the correction unit 270 can perform correction processing by performing re-recognition processing on the data obtained by subtraction.

  In the present embodiment, by correcting the feature amount data using the average value calculated by the average value calculation unit 261, for example, the characteristics of a sound collecting device such as a microphone for inputting sound to the feature amount calculation unit 210 are obtained. It can be the removed data. In other words, noise at the time of microphone collection can be removed, and more accurate correction (recognition processing) for speech can be performed. In the above-described embodiment, the present invention is applied to the error section extracted by the error section feature quantity extraction unit 260. However, the feature quantity data of a section having a certain length including the error section may be used. Good.

The average value calculation unit 261 and the feature normalization unit 262 can be applied to the second to eighth embodiments described above, respectively.

<Tenth Embodiment>
In the client apparatuses 110 to 110i that are the speech recognition result correction apparatuses described in the first to ninth embodiments, the correction unit 270 performs the correction process (re-recognition process). It is not a thing. That is, the server apparatus 120 is configured to notify the error section specified by the error section specifying unit 240 to the server apparatus 120, so that the server apparatus 120 performs correction processing again, and the receiving unit 235 receives the correction result. It is good. The re-correction process in the server apparatus 120 is the correction process in the correction unit 270 of the client apparatus 110 described above. As a specific example of the notification process in the client apparatus 110, the error section specifying unit 240 calculates time information of the error section specified by the error section specifying unit 240 or time information including words before and after the error section specifying unit 240, and the transmitting unit It is possible that 225 notifies the server device 120 of the time information. In the server apparatus 120, erroneous recognition is prevented from being performed again by performing voice recognition processing different from the recognition processing performed first. For example, the recognition process is performed by changing the acoustic model, the language model, and the dictionary.

<Eleventh embodiment>
Next, the client device 110k according to the eleventh embodiment will be described. The client device 110k in the eleventh embodiment recognizes a subword section and performs a correction process using the subword character string described in the subword section. FIG. 26 is a block diagram illustrating functions of the client device 110k.

  The client device 110k includes a feature amount calculation unit 210, a feature amount compression unit 220, a transmission unit 225, a feature amount storage unit 230, a reception unit 235, an error interval specification unit 240, a subword interval specification unit 242, a division unit 243, an error interval. A feature amount extraction unit 260, a dictionary addition unit 265, a correction unit 270, an integration unit 280, an acoustic model holding unit 281, a language model holding unit 282, a dictionary holding unit 283, and a display unit 290 are configured.

  This embodiment is different from the first embodiment in that it includes a subword section specifying unit 242, a dividing unit 243, and a dictionary adding unit 265. Hereinafter, the configuration will be described focusing on this difference.

  The subword section specifying unit 242 is a part that specifies a section including a subword character string from the error section specified by the error section specifying unit 240. In the subword character string, “subword” indicating an unknown word is added as attribute information, and the subword section specifying unit 242 can specify a subword section based on the attribute information. it can.

  For example, FIG. 28 shows a diagram showing a recognition result recognized based on the utterance content in server device 120. 28, “subword” is added as attribute information to “Sanyomusen”, and the subword section designation unit 242 recognizes “Sanyomusen” as a subword character string based on the attribute information, and the character A column part can be designated as a subword section.

  In FIG. 28, a frame index is added to the recognition unit of the recognition result recognized according to the utterance content. As described above, one frame is about 10 msec. Also, in FIG. 28, the error interval specification unit 240 can specify an error interval in accordance with the same processing as described above, and “from” (second recognition unit) to “ga” (eighth recognition). (Unit) can be designated as an error interval.

  The dividing unit 243 is a part that divides the error section specified by the error section specifying unit 240 with the subword character string included in the subword section specified by the subword section specifying unit 242 as a boundary. Based on the example shown in FIG. 28, the section is divided into section 1 and section 2 based on the subword character string “Sanyoumusen”. That is, from the second recognition unit “N” to the fifth recognition unit “Sanyoumusen”, that is, from 100 msec to 500 msec in the frame index, it is divided into sections 1 and the fifth recognition unit. From “Sanyomusen” to “8” as the eighth recognition unit, that is, from 300 msec to 660 msec is divided into the sections 2.

  The dictionary addition unit 265 is a part that adds the subword character string designated by the subword section designation unit 242 to the dictionary holding unit 283. In the example of FIG. 28, “Sanyoumusen” is newly added to the dictionary holding unit 283 as one word. Further, subword readings are added to the dictionary holding unit 283, and connection probabilities of subwords and other words are added to the language model holding unit 282. As the connection probability value in the language model holding unit 282, a class dedicated to subwords prepared in advance may be used. Moreover, since the character string of the subword model is almost a proper noun, the value of the noun (proper noun) class may be used.

  With such a configuration, the error section feature quantity extraction unit 260 extracts the feature quantity data held in the feature quantity storage unit 230 according to the sections 1 and 2 obtained by the division by the division unit 243. Then, the correction unit 270 performs correction processing by performing re-recognition processing on the feature amount data corresponding to each section. Specifically, taking FIG. 28 as an example, the correction result of section 1 is “in the electric manufacturer Sanyo Musen”, and the correction result of section 2 is “San Yomusen's product has a reputation”.

  The integration unit 280 performs integration processing on the recognition results (section 1 and section 2) obtained by the correction by the correction unit 270 based on the subword character string serving as a boundary, and the recognition result received by the reception unit 235. Are integrated and displayed on the display unit 290. Taking FIG. 28 as an example, as a result of the integration, the text of the final error section is “So that the product of the electric manufacturer Sanyomusen has a reputation”.

  Next, the operation of the client device 110k configured as described above will be described. FIG. 27 is a flowchart showing the operation of the client device 110k.

  From S101 to S105, processing similar to that of the client apparatus 110 shown in FIG. 6 is performed. That is, the feature data of the voice input through the microphone is extracted by the feature amount calculation unit 210 (S101). The feature amount data is stored in the feature amount storage unit 230 (S102). Next, the feature amount data is compressed by the feature amount compression unit 220 (S103). The compressed compressed feature data is transmitted to the server device 120 by the transmission unit 225 (S104). Then, voice recognition is performed in the server device 120, and the recognition result is transmitted from the server device 120 and received by the receiving unit 235 (S105). Then, an error section is specified by the error section specifying unit 240 from the speech recognition result (S106). Note that the preceding and following contexts may be specified based on the specified error interval.

  Next, the subword section is designated and fixed by the subword section designation unit 242 (S701). At this time, the subword character string in the subword section is a user dictionary (for example, a word registered by the user in the kana-kanji conversion dictionary or a name registered in the address book / phone book) provided in the client device 110k. Etc.) may be performed to replace the word. Then, the division unit 243 divides the error section with the subword section as a boundary (S702). While performing this division processing, the dictionary adding unit 265 holds the designated subword character string in the dictionary holding unit 283 (S703).

  Thereafter, the error section feature quantity extraction unit 260 extracts the error section feature quantity data and the subword section feature quantity data (S107a), and the correction section 270 re-recognizes the error section and subword section feature quantity data. Correction processing is performed (S108a). Then, the text data in the error section and the text data received by the receiving unit 235 are integrated, and the text data obtained by being correctly recognized is displayed on the display unit 290 (S109). In the integration, the results of the sections 1 and 2 are connected using the word at the boundary as a guide. Further, when the sub-word character string is converted based on the user dictionary, the correction unit 270 may perform the correction process by performing the voice recognition process using the converted character string as a constraint condition. .

  In the present embodiment, the subword character string has been described based on the recognition result of the server. However, the subword character string may be generated by the client device 110k. In this case, after the error section designation process in the process S106 of FIG. 27, a subword character string is generated and then a subword section determination process is performed. Further, the processing in FIG. 27 described above in the client device 100k may be performed by a server or another device. Furthermore, although the correction method has been described based on recognition, other methods such as a method based on similarity between character strings may be used. In this case, the feature quantity storage unit 230 and the process of saving the acoustic feature quantity data (S102), the error section feature quantity extraction unit 260, the correction unit 270, and the acoustic feature recognition (S108a) are not necessary.

  Further, when the character string of the subword is in the dictionary holding unit 283, the information in the dictionary holding unit 283 may be used. For example, if the dictionary holding unit 283 has a word corresponding to “Sanyomusen”, for example, “Sanyo Radio”, it is not necessary to add it to the subword dictionary.

  In the previous example, when the section is divided, each of the sections 1 and 2 includes a subword section. However, this is not essential, and each divided section may not include a subword. In other words, in the second word “from” to the start of the fifth subword character string, section 1 is divided into “1”, and from the fifth subword character string end to the eighth word end, “ga” is divided into section 2. May be. In this case, the subword character string need not be added to the dictionary.

  Next, operational effects of the client device 110k according to the present embodiment will be described. In the client device 110k, the receiving unit 235 receives the recognition result from the server device 120, and the error section specifying unit 240 specifies an error section. Further, the subword section specifying unit 242 specifies a subword section in the error section. This can be determined from the attribute information added to the recognition result transmitted from the server device 120. Then, the correction unit 270 extracts feature quantity data corresponding to the subword section designated by the subword section designation unit 242 from the feature quantity storage unit 230, and performs re-recognition using the extracted feature quantity data. Then, the recognition result is corrected. As a result, correction processing for unknown words such as subwords can be performed. That is, re-recognition can be performed according to an unknown word section such as a subword section.

  Further, in the client device 110k of this embodiment, the dividing unit 243 divides the recognition result into a plurality of sections according to the subword section specified by the subword section specifying unit 240. Then, the correction unit 270 corrects the recognition result for each divided section divided by the dividing unit 243. Thereby, the recognition target can be shortened, and more accurate recognition processing can be performed.

  Also, in the client device 110k, the dividing unit 243 recognizes the end point of the subword section as the end point of one divided section and the start point of the subword section as the start point of the next divided section of the one divided section. Split. Then, the correction unit 270 performs correction of the recognition result for each divided section divided by the dividing unit 243 and sets the subword section as a constraint condition for correction of each divided section. As a result, the subword section is included in any of the divided sections. Therefore, when the recognition process is performed, the subword section is always included, so that the recognition process using the subword character string as a constraint condition can be performed.

In the client device 110k, the dictionary adding unit 265 adds the subword character string in the subword section designated by the subword section designating unit 242 to the dictionary holding unit 283 for recognition processing. As a result, subword character strings can be accumulated, and can be used effectively for future recognition processing, and more accurate recognition processing can be performed.

<Twelfth Embodiment>
In the eleventh embodiment, the method of dividing a subword character string into boundaries has been described. In this embodiment, a method of using a subword character string without fail when it is recognized again will be described. This embodiment assumes the same apparatus configuration as that of the eleventh embodiment.

  FIG. 29 is a conceptual diagram of a search process in speech recognition, FIG. 29 (a) is a conceptual diagram showing a search process including the subword character string “Sanyomusen”, and FIG. It is a conceptual diagram which shows the search process in a some area by making a subword character string into a constraint condition.

In general, the likelihoods of hypotheses for all paths are calculated in the speech recognition search process, the intermediate results are stored, and the results are finally generated in descending order of likelihood. In practice, considering the cost, a method of narrowing the search range within a certain range is used. In the present embodiment, when the subword section specified by the subword section specifying unit 242 is in a predetermined section (for example, between 2 seconds and 3 seconds), the correction unit 270 includes the subword described in the subword section. Using the character string, the recognition process is performed so that the rank in which the subword character string appears in the search process is ranked higher than the other paths, and finally the recognition result including the subword character string is preferentially output. For example, the following search route is obtained by the correction unit 270 and held.
Route 1: Recent / At / Entrance / At / Meeting Route 2: Yesterday / No / Meeting / Ha / World / Medium /
Path 3: Recent / At / Unit price / High / San Iomsen Path 4: Recent / At / Electric manufacturer // San Iomsen

  Since there is “Sanyomusen” in the route 3 and the route 4 among these, the correction unit 270 performs processing so that the rank of these two routes is higher than that of the route 1 and the route 2. If the range is narrowed down, the route 3 and the route 4 are left without the route 1 and the route 2 being left. Furthermore, the appearance position of “Sanyomusen” may be determined and narrowed down to a route limited to a certain range close to the appearance position of “Sanyomusen” (300 ms to 500 ms) in the original recognition result. In addition, a candidate in which “Sanyomusen” appears in preference to a candidate in which “Sanyomusen” does not appear in the final recognition result may be output preferentially.

  As described above, in the client device 110k, the correction unit 270 holds the hypothesis including the subword character string described in the subword section specified by the subword section specifying unit 242 with a higher priority as a recognition search process, and holds the hypothesis. Correction is performed by selecting a final recognition result from the hypothesis. Thereby, the recognition process using the subword character string can be performed without fail.

It is a system configuration | structure figure of the communication system containing the client apparatus 110 (110a-110k) which is the speech recognition result correction apparatus of this embodiment. 3 is a block diagram illustrating functions of a client device 110. FIG. 2 is a hardware configuration diagram of a client device 110. FIG. It is a conceptual diagram which shows the concept of the various information contained in a speech recognition result. (A) is a conceptual diagram when a context before and after an error section is designated, and (b) is a conceptual diagram showing a concept when performing recognition processing based on a constraint condition. 4 is a flowchart illustrating an operation of the client device 110. It is a flowchart which shows the detailed process of the correction process including designation | designated of an error area. It is a block diagram which shows the function of the client apparatus 110a which receives an error area by a user input. It is a flowchart which shows the process of the client apparatus 110a. It is a flowchart which shows the detailed process when an error area is designated by the user input in the client apparatus 110a. It is a block diagram which shows the function of this client apparatus 110b. It is a flowchart which shows the process of the client apparatus 110b. It is a flowchart which shows the detailed process when designating the error area in the client apparatus 110b. It is a block diagram which shows the function of the client apparatus 110c. It is a flowchart which shows the process of the client apparatus 110c. It is a block diagram which shows the function of the client apparatus 110d. It is a flowchart which shows the process of the client apparatus 110d. It is a block diagram which shows the function of the client apparatus 110f. It is a flowchart which shows the process of the client apparatus 110f. It is a block diagram which shows the function of the client apparatus 110g. It is a flowchart which shows the process of the client apparatus 110g. It is a block diagram which shows the function of the client apparatus 110h. It is a block diagram which shows the function of the client apparatus 110i. It is a conceptual diagram which shows the concept when correct | amending the part designated as word information as a constraint condition. FIG. 10 is a block diagram illustrating a modified example of the client device 110. It is a block diagram which shows the function of the client apparatus 110k. It is a flowchart which shows operation | movement of the client apparatus 110k. It is explanatory drawing explaining the correspondence of utterance content, a recognition result, and a division | segmentation area. It is a conceptual diagram of the search process in speech recognition.

Explanation of symbols

110, 110a, 110b, 110c, 110d, 110f, 110g, 110h ... client device, 120 ... server device, 210 ... feature amount calculation unit, 220 ... feature amount compression unit, 225 ... transmission unit, 226 ... first recognition unit, 227 ... Language model holding unit, 228 ... Dictionary holding unit, 229 ... Acoustic model holding unit, 230 ... Feature amount storage unit, 235 ... Reception unit, 236 ... Operation unit, 237 ... Result storage unit, 238 ... User input detection unit, 239 ... Time information calculation section, 240, 240a, 240b, 240c ... Error section designation section, 241 ... End point determination section, 242 ... Subword section designation section, 243 ... Division section, 250, 250a ... Context designation section before and after error section, 251 ... word information analysis unit, 260 ... error section feature quantity extraction unit, 265 ... dictionary addition unit, 270, 270a, 270b ... correction Department, 280 ... integrating unit, 281 ... acoustic model holding unit, 282 ... language model holding unit, 283 ... dictionary holding unit, 284 ... language DB holding unit, 285 ... constraint condition storage unit, 290 ... display unit.

Claims (32)

An input means for inputting voice;
Calculating means for calculating feature data based on the voice input by the input means;
Storage means for storing feature amount data calculated by the calculation means;
Obtaining means for obtaining a recognition result for the voice input by the input means;
In the recognition result recognized by the acquisition means, designation means for designating an error section where a recognition error has occurred;
Extracting the feature amount data corresponding to the error section specified by the specifying unit from the feature amount data stored in the storage unit, and performing re-recognition using the extracted feature amount data; A speech recognition result correction apparatus comprising: correction means for correcting the recognition result obtained by the above. The acquisition means includes
Transmitting means for transmitting the voice input by the input means to a voice recognition device;
Receiving means for receiving a recognition result recognized by the voice recognition device,
2. The speech recognition result correction apparatus according to claim 1, wherein the specifying unit specifies an error section in which a recognition error occurs in the recognition result received by the receiving unit.   The speech recognition result correction apparatus according to claim 1, wherein the specifying unit specifies an error section by receiving a user operation.   The said designation | designated means determines an error area based on the reliability of the recognition result provided to the said recognition result, The said determined error area is specified, The any one of Claim 1 to 3 characterized by the above-mentioned. The speech recognition result correction apparatus described in 1.   The said designation | designated means calculates the reliability of the said recognition result, judges an error area based on the said reliability, and designates the judged error area, The any one of Claim 1 to 3 characterized by the above-mentioned. The speech recognition result correction apparatus described in 1. A specifying means for specifying a recognition result forming either at least one word immediately before the error section specified by the specifying means, at least one word immediately after, or both the immediately preceding word and the immediately following word. Further comprising
The correction means uses the recognition result specified by the specifying means as a constraint condition, and extracts feature amount data corresponding to a section including a word immediately before and an error section from the storage means according to the constraint condition. The speech recognition result correction apparatus according to claim 1, wherein recognition processing is performed on the extracted feature amount data. A specifying means for specifying a recognition result forming either at least one word immediately before the error section specified by the specifying means, at least one word immediately after, or both the immediately preceding word and the immediately following word. Further comprising
The correction means uses the recognition result specified by the specifying means as a constraint condition, and extracts feature quantity data corresponding to an error section from the storage means according to the constraint condition, and recognizes the extracted feature quantity data. The speech recognition result correction apparatus according to claim 1, wherein: Word information which is information for specifying at least one word immediately before the error section specified by the specifying means, word information of at least one word immediately after, or word information of the word immediately before and the word immediately after Word information specifying means for specifying the word information of the word in the recognition result forming either of the word information of
The correction means uses the word information specified by the word information specifying means as a constraint condition, and according to the constraint condition, the storage unit stores feature quantity data corresponding to a word immediately before an error section and a section including a word immediately after the error section. The speech recognition result correction apparatus according to claim 1, wherein a recognition process is performed on the extracted feature amount data.   9. The speech recognition result correction apparatus according to claim 8, wherein the word information includes one or more of part-of-speech information indicating a word part-of-speech and reading information indicating a word-reading method. Based on the word information, at least one word immediately before the error interval specified by the specifying means, or at least one word immediately after, or both the immediately preceding word and the immediately following word are formed. It further comprises an unknown word determination means for determining whether or not the word of the recognition result is an unknown word,
9. The correction unit corrects the recognition result based on the word information when the unknown word determination unit determines that the word of the recognition result is an unknown word. Or the speech recognition result correction apparatus of 9. It further comprises connection probability storage means for storing the connection probability between words,
The correction means creates a connection probability with the word in the error section and the word before or after the error section by performing the correction process, and uses the connection probability to calculate the connection probability stored in the connection probability storage means. The speech recognition result correction apparatus according to claim 1, wherein the speech recognition result correction apparatus is updated. A constraint condition storage means for storing the word information specified by the word information specifying means or the word specified by the specification means as a constraint condition;
The speech recognition result correction apparatus according to claim 6, wherein the correction unit performs correction processing according to a constraint condition stored in the constraint condition storage unit. It further comprises a receiving means for receiving character information from the user,
The speech recognition result according to any one of claims 1 to 12, wherein the correction unit performs a correction process on a recognition result in an error section using the character information received by the receiving unit as a constraint condition. Correction device. Based on the recognition result received by the receiving means and the feature amount data stored in the storage means, further comprising time information calculating means for calculating an elapsed time in the recognition result,
The speech recognition result correction apparatus according to claim 1, wherein the specifying unit specifies an error section based on the time information calculated by the time information calculating unit. Further comprising display means for displaying the recognition result corrected by the correction means,
The speech recognition result correcting apparatus according to claim 1, wherein the display unit does not display the recognition result acquired by the acquiring unit.   When the recognition result obtained by re-recognition by the correcting means is the same as the recognition result obtained by the obtaining means, or when there is a deviation in the time information included in each of these recognition results The speech recognition result correcting apparatus according to claim 15, wherein a recognition error is determined and the display means does not display a recognition result.   The designation means designates the start point of an error section by a user operation, and designates the end point of the error section based on the reliability of the recognition result given to the recognition result acquired by the acquisition means. The speech recognition result correction apparatus according to claim 3.   4. The speech recognition result correcting apparatus according to claim 3, wherein the designating unit designates a start point of an error section by a user operation, and designates an end point of the error section with a predetermined number of recognition units from the start point.   The said designation | designated means designates the start point of an error area by user operation, and designates the end point of an error area based on the predetermined phonetic symbol in the recognition result acquired by the said acquisition means. Voice recognition result correction device. The acquisition means acquires a plurality of recognition candidates as a recognition result when acquiring the recognition result,
4. The speech recognition result correcting apparatus according to claim 3, wherein the specifying unit specifies a start point of an error section by a user operation and specifies an end point based on the number of recognition candidates acquired by the acquiring unit. . A calculation means for calculating an average value of the section including the error section of the feature amount data calculated by the calculation means;
The correction means subtracts the average value calculated by the calculation means from the extracted feature quantity data, and performs re-recognition processing using the data obtained by the subtraction as feature quantity data. 21. The speech recognition result correcting apparatus according to any one of 1 to 20. An input means for inputting voice;
Obtaining means for obtaining a recognition result for the voice input by the input means;
In the recognition result recognized by the acquisition means, designation means for designating an error section where a recognition error has occurred;
Notification means for requesting the external server to re-recognize the error section by notifying the external server of the error section specified by the specifying means;
Receiving means for receiving a recognition result of an error section re-recognized in the external server in response to a request by the notification means;
A speech recognition result correction apparatus comprising: An input step for inputting voice;
A calculation step of calculating feature amount data based on the voice input in the input step;
A storage step of storing the feature amount data calculated by the calculation step;
An acquisition step of acquiring a recognition result for the voice input by the input step;
In the recognition result recognized by the obtaining step, a designation step for designating an error section in which a recognition error occurs;
By extracting feature quantity data corresponding to the error section designated by the designation means from the feature quantity data stored in the storage step, and performing re-recognition using the extracted feature quantity data, A speech recognition result correction method comprising: a correction step for correcting the obtained recognition result. An input step for inputting voice;
An acquisition step of acquiring a recognition result for the voice input in the input step;
In the recognition result recognized by the obtaining step, a designation step for designating an error section in which a recognition error occurs;
A notification step of requesting the external server to re-recognize the error section by notifying the external server of the error section specified in the specifying step;
A receiving step of receiving a recognition result of an error section re-recognized in the external server in response to a request by the notification step;
A speech recognition result correction method comprising: In the recognition result obtained by the obtaining means, subword section designating means for designating a subword section, and
The correcting means further extracts feature data corresponding to the subword section specified by the subword section specifying means from the storage means in the error section specified by the specifying means, and uses the extracted feature data. 23. The speech recognition result correction apparatus according to claim 1, wherein the recognition result obtained by the acquisition unit is corrected by performing re-recognition. Further comprising a dividing means for dividing the recognition result acquired from the acquiring means into a plurality of sections according to the subword section specified by the subword section specifying means;
26. The speech recognition result correction apparatus according to claim 25, wherein the correction unit performs correction of a recognition result for each divided section divided by the dividing unit.   The dividing means divides the recognition result so that the end point of the subword section is the end point of one divided section and the start point of the subword section is the start point of the next divided section of the one divided section. The speech recognition result correction apparatus according to claim 26.   28. The correction unit according to claim 27, wherein the correction unit performs correction of a recognition result for each divided section divided by the dividing unit, and uses the subword section as a constraint condition in correction of each divided unit. Voice recognition result correction device.   The correction means retains a hypothesis including a subword character string described in the subword section designated by the subword section designation means as a recognition search process, and corrects by selecting a final recognition result from the hypothesis 26. The speech recognition result correction apparatus according to claim 25, wherein:   30. The dictionary adding means according to any one of claims 25 to 29, further comprising a dictionary adding means for adding a subword character string in the subword section designated by the subword section designating means to a dictionary database for recognition processing. Speech recognition result correction device. A dictionary database generated by the user;
31. The speech recognition result correction apparatus according to claim 25, wherein the correction unit performs correction processing using a character string obtained by converting a subword character string according to the dictionary database. The speech recognition result correction apparatus according to any one of claims 1 to 22 or 25 to 31,
A server device that performs speech recognition based on the speech transmitted from the speech recognition result correction device, and transmits the recognition result to the speech recognition result correction device;
Speech recognition result correction system consisting of

Images