JP6735392B1 - Audio text conversion device, audio text conversion method, and audio text conversion program - Google Patents

Abstract

A speech recognition result with higher accuracy is output. SOLUTION: A noise suppressing section 11 suppresses noise of an original speech waveform f1, a speech section detecting section 12 detects a speech section tj from a noise suppressing speech waveform f2, and a speech waveform cutting section 13 detects the original speech waveform f1 and noise. The suppressed voice waveform f2 is cut for each utterance section tj to obtain the section voice waveforms f1_tj, f2_tj, and the voice recognition unit 14 causes the plurality of voice recognition engines ei to detect the section voice waveforms f1_tj, f2_tj before and after noise suppression. Each of them is subjected to voice recognition, and the one having the larger number of characters is set as the voice recognition result Rij of the utterance section tj by the voice recognition engine ei, and the recognition result correction unit 15 compares the voice recognition results Rij for each utterance section tj to correct the voice recognition result. To do. [Selection diagram] Figure 1

Description

The present invention relates to a technique for improving voice recognition accuracy.

In recent years, voice recognition technology has been widely used. For example, smart speakers that have a microphone built in a speaker connected to a network and that can be operated by voice recognition have become widespread. Various companies have provided speech recognition engines, which makes it easy to convert speech into text.

Further, a noise suppression technique for improving the accuracy of voice recognition has also been studied (for example, Non-Patent Document 1).

"Voice processing technology for improving voice recognition accuracy in noisy environments", Nippon Telegraph and Telephone Corporation, [Search on April 22, 2019], Internet <URL: http://www.ntt.co. jp/svlab/activity/category_2/product2_29.html〉

The characteristics of the recognition result differ depending on the voice recognition engine, and each voice recognition engine has its strengths and weaknesses. Since the data used for learning and the voice recognition algorithm are different for each voice recognition engine, a voice recognition engine with high recognition accuracy for a speech with a neat speaking style such as a sentence, or a voice with an unnatural speech like a spoken word There is a voice recognition engine with high recognition accuracy. Depending on the voice recognition engine, there are those that output only the portion where the recognition accuracy is estimated to be high, and those that output all the recognized portions.

Further, there is a portion where the voice recognition accuracy is improved by suppressing the noise and a portion where it is not, and if the noise is suppressed, the recognition accuracy is not necessarily improved. For example, when the noise suppression technique is applied, the noise is suppressed in a noisy place, so that the speech recognition accuracy is improved. However, noise-free processing may reduce the sound quality at a noise-free location, resulting in a decrease in voice recognition accuracy.

The present invention has been made in view of the above, and an object thereof is to output a voice recognition result with higher accuracy.

A speech text forming apparatus according to the present invention, a noise suppressing unit for suppressing noise of an input speech waveform, and a first speech recognition result of speech recognition of the speech waveform by each of a plurality of speech recognition engines, and noise A second speech recognition result obtained by recognizing the suppressed noise-suppressed speech waveform is obtained, and one of the first speech recognition result and the second speech recognition result having the larger number of characters is determined by the speech recognition engine. And a recognition result correction unit that compares the voice recognition results of the plurality of voice recognition engines with each other and corrects the voice recognition result.

A voice text conversion method according to the present invention suppresses noise in an input voice waveform, first voice recognition result of voice recognition of the voice waveform by each of a plurality of voice recognition engines, and noise is suppressed. A step of obtaining a second voice recognition result obtained by voice-recognizing a noise-suppressed voice waveform, and a method having a larger number of characters of the first voice recognition result and the second voice recognition result is used as the voice recognition result of the voice recognition engine. And a step of comparing the voice recognition results of the plurality of voice recognition engines with each other to correct the voice recognition result.

According to the present invention, it is possible to output a voice recognition result with higher accuracy.

It is a functional block diagram which shows the structure of the audio-text conversion apparatus of this embodiment. It is a flowchart which shows the flow of a process of the audio-text conversion apparatus of this embodiment. It is a flowchart which shows the flow of a voice waveform cutting process. It is a figure which shows the relationship between the length of the silent area from the head of a speech waveform, and recognition accuracy. It is a flowchart which shows the flow of a voice recognition process. It is a flow chart which shows a flow of recognition result amendment processing. It is a figure which shows the example which divided the speech recognition result into morphemes, and extracted the mismatched part. It is a figure which shows an example of a correction state flag.

Embodiments of the present invention will be described below with reference to the drawings.

(Structure of voice text conversion device)
FIG. 1 is a functional block diagram showing the configuration of the voice text conversion device 1 of this embodiment. The voice text conversion device 1 inputs a voice and outputs a text which is a recognition result of voice recognition of the input voice. The voice text conversion device 1 may output a correction state indicating the correction content of the voice recognition result, in addition to the text.

The speech text forming apparatus 1 shown in FIG. 1 includes a noise suppression unit 11, a speech section detection unit 12, a speech waveform cutting unit 13, a speech recognition unit 14, and a recognition result correction unit 15. Each unit included in the voice text conversion device 1 may be configured by a computer including an arithmetic processing unit, a storage device, and the like, and the process of each unit may be executed by a program. This program is stored in a storage device included in the voice text conversion device 1, and can be recorded in a recording medium such as a magnetic disk, an optical disk, a semiconductor memory, or provided via a network.

The noise suppression unit 11 inputs the original speech waveform f1 to be speech-recognized, performs noise suppression processing, and outputs a noise-suppressed speech waveform f2. For the noise suppression processing, for example, the audio processing technology of Non-Patent Document 1 or the technology implemented in a noise canceling earphone can be used. The original speech waveform f1 and the noise-suppressed speech waveform f2 are input to the speech waveform cutting unit 13.

The utterance section detection unit 12 receives the noise-suppressed voice waveform f2 and detects a utterance section tj (j=1, 2,..., M) in which a person speaks in the voice waveform. A VAD (Voice Activity Detection) library published by Google or the like can be used for detecting the utterance section. The speech section detection unit 12 may detect the speech section from the original speech waveform f1.

The voice waveform cutting unit 13 cuts out the voice waveforms of the original voice waveform f1 and the noise-suppressed voice waveform f2 at the utterance section tj, and adds a sound waveform to the beginning of each of the cut voice waveforms for each utterance section. The voice waveform cutting unit 13 cuts out the original voice waveform f1 for each utterance section tj and adds a non-voice waveform to the section voice waveform f1_tj, and the noise-suppressed voice waveform f2 to cut a voice signal for each utterance section tj to add a non-voice waveform. The waveform f2_tj is output to the voice recognition unit 14.

The voice recognition unit 14 uses a plurality of voice recognition engines ei (i=1, 2,..., N) to voice the section voice waveform f1_tj and the section voice waveform f2_tj before and after noise suppression for each utterance section tj. recognize. The voice recognition unit 14 sets the recognition result with the larger number of characters among the recognition results of the section voice waveform f1_tj and the section voice waveform f2_tj as the voice recognition result Rij of the utterance section tj by the voice recognition engine ei. That is, the voice recognition unit 14 outputs the voice recognition result Rij by the plurality of voice recognition engines ei for each utterance section tj.

The voice recognition unit 14 may include a plurality of voice recognition engines ei, or may perform voice recognition using an external voice recognition service. The format does not matter as long as a plurality of different voice recognition engines ei are used. For a voice recognition engine that outputs multiple results, the recognition result with the highest reliability is adopted. Alternatively, a plurality of results may be output from the one having the highest reliability, and the recognition result correction unit 15 in the subsequent stage may compare them.

The recognition result correction unit 15 compares the voice recognition results Rij of the respective voice recognition engines ei for each utterance section tj to identify the non-coincidence points, and adopts the more voice recognition results of the voice recognition engine ei for the non-coincidence points. To do. When the voice input by the voice text conversion device 1 is associated with a video or a slide, the recognition result correction unit 15 compares the voice recognition result Rij with the character recognition result of the video or the slide, and finds the most suitable. Corrected to the contents The character recognition result of a video or a slide may be extracted by processing the video or the like by another device and input by the voice text conversion device 1, or the voice text conversion device 1 may input or extract the video or the like. May be.

The recognition result correction unit 15 outputs the correction state of the mismatched portion of the voice recognition result Rij in addition to the text that is the voice recognition result after correction. For example, the recognition result correction unit 15 adds information such as correction by voice recognition comparison or correction by comparison with character recognition to the corrected non-matching portion.

(Operation of voice text conversion device)
Next, the operation of the voice text conversion device 1 of this embodiment will be described.

FIG. 2 is a flowchart showing the flow of processing of the voice text conversion device 1 of this embodiment.

In step S1, the noise suppression unit 11 performs noise suppression processing on the original speech waveform f1 and outputs a noise-suppressed speech waveform f2.

In step S2, the speech section detection unit 12 detects the speech section tj from the noise-suppressed speech waveform f2.

In step S3, the voice waveform cutting unit 13 cuts out the utterance section tj from each of the original voice waveform f1 and the noise-suppressed voice waveform f2, and adds a non-voice waveform to the heads of the cut section voice waveforms f1_tj and f2_tj. Details of the voice waveform cutting processing by the voice waveform cutting unit 13 will be described later.

If the original speech waveform f1 is short, the original speech waveform f1 and the noise-suppressed speech waveform f2 may be passed to the speech recognition unit 14 without performing the processes of steps S2 and S3.

In step S4, the voice recognition unit 14 performs voice recognition on each of the section voice waveforms f1_tj and f2_tj using the plurality of voice recognition engines ei, and obtains a voice recognition result Rij. Details of the voice recognition processing by the voice recognition unit 14 will be described later.

In step S5, the recognition result correction unit 15 compares the voice recognition results Rij by the plurality of voice recognition engines ei, adopts an appropriate recognition result, and outputs the text. The recognition result correction unit 15 may correct the voice recognition result using the character recognition result related to the original voice. Details of the recognition result correction processing by the recognition result correction unit 15 will be described later.

(Voice waveform cutting process)
FIG. 3 is a flowchart showing the flow of the voice waveform cutting process. The speech waveform cutting unit 13 inputs the original speech waveform f1, the noise-suppressed speech waveform f2, and the utterance section tj, and executes speech waveform cutting processing.

In step S31, the voice waveform cutting unit 13 cuts out the original voice waveform f1 in the utterance section tj.

In step S32, the voice waveform cutting unit 13 cuts out the noise-suppressed voice waveform f2 in the utterance section tj.

In step S33, the speech waveform cutting unit 13 adds a sound waveform to the beginning of each of the speech waveforms obtained by cutting the original speech waveform f1 and the noise-suppressed speech waveform f2 in the utterance section tj. The voice waveform cutting unit 13 outputs a section voice waveform f1_tj obtained by cutting out the original voice waveform f1 in the utterance section tj and adding silence, and a section voice waveform f2_tj obtained by cutting out the noise suppression voice waveform f2 in the utterance section tj and adding silence. To do.

As shown in FIG. 4, at the time of voice recognition, the recognition accuracy is improved if the silent section before utterance has a predetermined length or more. Therefore, the voice waveform cutting unit 13 predetermines the time of the silent section where the recognition accuracy is saturated, and adds the silent section to the head of the cut section voice waveforms f1_tj and f2_tj.

In step S34, the voice waveform cutting unit 13 determines whether or not all utterance sections have been processed. If there is an unprocessed utterance section, the process returns to step S31 to process the next utterance section tj+1. When all utterance sections have been cut out, the voice waveform cutting process ends.

(Voice recognition processing)
FIG. 5 is a flowchart showing the flow of voice recognition processing. The voice recognition unit 14 inputs the section voice waveforms f1_tj and f2_tj before and after noise suppression, and obtains a voice recognition result for each utterance section using each of a plurality of voice recognition engines.

In step S41, the voice recognition unit 14 selects one voice recognition engine ei from the plurality of voice recognition engines.

In step S42, the voice recognition unit 14 performs voice recognition of the section voice waveform f1_tj cut out from the original voice waveform f1 by using the voice recognition engine ei selected in step S41.

In step S43, the voice recognition unit 14 uses the voice recognition engine ei selected in step S41 to perform voice recognition of the section voice waveform f2_tj cut out from the noise suppression voice waveform f2.

In step S44, the voice recognition unit 14 compares the numbers of characters of the voice recognition results obtained in steps S42 and S43, and the voice recognition result of the one with the larger number of characters is the voice recognition result Rij of the utterance section tj by the voice recognition engine ei. To be adopted as. By comparing the recognition results of the waveforms before and after noise suppression, it can be reflected that there is a part where the speech recognition accuracy is improved by noise suppression and a part where it is not. By comparing the number of recognized characters before and after noise suppression and adopting the recognition result with a large number of characters, it is possible to prevent omission of recognition.

In step S45, the voice recognition unit 14 determines whether or not all the utterance sections have been processed. If there is an unprocessed utterance section, the process returns to step S42 to process the next utterance section tj+1.

In step S46, the voice recognition unit 14 determines whether all the voice recognition engines have processed. If there is a voice recognition engine that has not been processed, the process returns to step S41, the next voice recognition engine ei+1 is selected, and processing is performed in order from the first speech section. The processes of steps S42 to S45 may be executed in parallel by a plurality of voice recognition engines.

(Recognition result correction process)
FIG. 6 is a flowchart showing the flow of the recognition result correction process. The recognition result correction unit 15 compares the voice recognition results Rij of the voice recognition engines ei for each utterance section tj, and corrects the voice recognition result based on the comparison result.

In step S51, the recognition result correction unit 15 compares the speech recognition results of each speech recognition engine with respect to the utterance section tj, and extracts a mismatched portion. Specifically, the recognition result correction unit 15 divides the speech recognition result Rij into morphemes using MeCab or Japanese, and compares the recognition results between the speech recognition engines for each morpheme using a library such as difflib. To extract the mismatched part.

FIG. 7 shows an example in which the voice recognition result is divided into morphemes and the mismatched portions are extracted. In the example of the figure, the recognition results of the six voice recognition engines e1 to e6 in the utterance section tj are shown divided into morphemes. In the utterance section tj, the voice recognition results by the voice recognition engines e1-e3 are "I climb a mountain", the voice recognition results by the voice recognition engines e4, e5 are "I am a mountain ride", and the voice recognition engine The result of voice recognition by e6 is "I ride a mountain". When each speech recognition result is divided into morphemes and compared, "I" and "eagle" and "climbing" and "ride" are extracted as non-matching portions.

In step S52, the recognition result correction unit 15 adopts the result output by the plurality of voice recognition engines for the mismatched portion. For example, in the example of FIG. 7, the recognition result correction unit 15 determines that the number of voice recognition engines that recognize “I” is “I” when the number of voice recognition engines that recognize “I” is different from that of the voice recognition engines that recognize “I”. Since it is more than the number, "I" is adopted. Further, in the example of FIG. 7, the recognition result correction unit 15 has the same number of voice recognition engines with respect to the disagreement between “climbing” and “riding”, so either one may be adopted.

In step S53, the recognition result correction unit 15 compares the character recognition result and the recognition result of the mismatched portion with respect to the mismatched portion, and adopts a more appropriate candidate. For example, a character recognition result is acquired from a video or a slide from a section including 10 seconds before and after the utterance section tj, and the character recognition result and the meaning vector of each recognition result of a mismatched portion are compared, and the character recognition result and the meaning are similar. The recognition result is adopted. The semantic vector can be derived using a vectorization method such as word2vec. In the example of FIG. 7, when the character “mountain climbing” can be acquired from the video, the recognition result correction unit 15 adopts “climbing” for a portion where “climbing” and “ride” do not match.

The order of step S52 and step S53 may be reversed. When the same non-matching portion is corrected in step S52 and step S53, the one with higher reliability may be adopted.

In step S54, the recognition result correction unit 15 sets the correction state flag based on the correction status in steps S52 and S53. FIG. 8 shows an example of the correction state flag. In the example of FIG. 8, when the voice recognition result is not corrected in step S52 and step S53, the correction state flag is set to 1, and when the voice recognition result is corrected based on the comparison between the voice recognition results in step S52, The correction state flag is set to 2, and the correction state flag is set to 3 when the voice recognition result is corrected based on the comparison with the character recognition result in step S53. The flag is not limited to the above.

In step S55, the recognition result correction unit 15 outputs the correction state flag fj set in step S54 together with the text Tj of the voice recognition result for the utterance section tj.

In step S56, the recognition result correction unit 15 determines whether or not all utterance sections have been processed. If there is an unprocessed utterance section, the process returns to step S51 to process the next utterance section tj+1.

As described above, according to the present embodiment, the noise suppression unit 11 suppresses the noise of the original speech waveform f1, the speech section detection unit 12 detects the speech section tj from the noise suppression speech waveform f2, and the speech waveform disconnection is performed. The unit 13 cuts the original speech waveform f1 and the noise-suppressed speech waveform f2 for each utterance section tj to obtain section speech waveforms f1_tj and f2_tj, and the speech recognition unit 14 suppresses noise by each of the plurality of speech recognition engines ei. Each of the preceding and following section speech waveforms f1_tj, f2_tj is speech-recognized, and the one having the larger number of characters is set as the speech recognition result Rij of the speech section tj by the speech recognition engine ei, and the recognition result correction unit 15 makes the speech recognition result Rij for each speech section tj. By correcting the voice recognition result by comparing the above, the accuracy of the voice recognition can be improved according to the presence or absence of the noise suppression effect and the strength and weakness of the voice recognition engine.

According to the present embodiment, the voice waveform cutting unit 13 adds the non-voice waveform to the head of the section voice waveforms f1_tj, f2_tj, so that the accuracy of voice recognition of the section voice waveforms f1_tj, f2_tj can be improved.

According to the present embodiment, the recognition result correction unit 15 corrects the voice recognition result based on the character recognition result extracted from the video associated with the original voice waveform, thereby obtaining the voice recognition result that matches the meaning of the voice. ..

According to the present embodiment, the recognition result correction unit 15 outputs the correction state flag indicating the correction content of the voice recognition result, so that the validity of the voice recognition result can be determined.

DESCRIPTION OF SYMBOLS 1... Speech text-ized apparatus 11... Noise suppression part 12... Speech section detection part 13... Speech waveform cutting part 14... Speech recognition part 15... Recognition result correction part

Claims (6)

A noise suppression unit that suppresses noise in the input speech waveform,
Each of the plurality of voice recognition engines obtains a first voice recognition result of voice recognition of the voice waveform and a second voice recognition result of voice recognition of a noise-suppressed voice waveform in which noise is suppressed, and obtains the first voice recognition result. A voice recognition unit that selects one of the voice recognition result and the second voice recognition result having the larger number of characters as the voice recognition result of the voice recognition engine;
And a recognition result correcting unit that corrects the voice recognition result by comparing the voice recognition results of the plurality of voice recognition engines with each other. An utterance section detection unit that detects an utterance section from the voice waveform,
A voice waveform cutting unit that cuts the voice waveform and the noise-suppressed voice waveform for each utterance section, and adds a silent waveform to the head of the section voice waveform cut for each utterance section,
The voice recognition device according to claim 1, wherein the voice recognition unit performs voice recognition of the section voice waveform cut out from each of the voice waveform and the noise suppression voice waveform for each utterance section. The voice recognition device according to claim 1 or 2, wherein the recognition result correction unit corrects the voice recognition result based on a character recognition result extracted from an image associated with the voice waveform. The voice text conversion device according to any one of claims 1 to 3, wherein the recognition result correction unit outputs information indicating correction contents of the voice recognition result. Suppressing the noise of the input speech waveform,
Obtaining a first voice recognition result of voice recognition of the voice waveform and a second voice recognition result of voice recognition of a noise-suppressed voice waveform with noise suppressed by each of a plurality of voice recognition engines;
Selecting one of the first voice recognition result and the second voice recognition result, which has the larger number of characters, as the voice recognition result of the voice recognition engine;
A step of comparing the speech recognition results of the plurality of speech recognition engines with each other to correct the speech recognition result. A voice text conversion program for operating a computer as each part of the voice text conversion device according to any one of claims 1 to 4.

Images