JPH10198397A - Voice recognition device and voice recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a result of voice recognition, substantially simultaneously with the completion of the voice. SOLUTION: After voice data from a voice input device 21 is taken into a voice data processing part 22, and is fed to a successive recording and processing part 23 so as to successively recorded on a recording buffer memory which is sectioned into a plurality of parts. Thus, when recording into one of the parts of the memory is completed, recording to the next memory is started while voice data with which recording is completed, are inputted in a feature extracting part 24a constituting a recognizing part 24, and thereafter the voice data obtains is subjected to frequency analysis so as to obtain a spectrum row which is then inputted a sound element recognizing part 24b constituted by a neural network so as to obtain a sound element candidate row. This candidate row is inputted to a word spot 24c and is verified with a dictionary template 24d and DTW so as to deliver a most resembling word as a result. The output results are calculated by an integrated distance calculating part 25, and thus calculated value is delivered to a back trance part 26 in order to take out a series of recognized words.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus and a speech recognition processing method using a sequential speech processing method.

[0002]

2. Description of the Related Art As a speech recognition apparatus, there is a discrete word speech recognition system shown in FIG. This system is, as shown in FIG. 12, a voice input device 1 such as a telephone or a microphone.
The voice data is input to the voice input unit 12 from 1. The voice data input to the voice input unit 12 is transmitted to the feature extraction unit 1
3 where the audio data is frequency analyzed.
A spectrum sequence is obtained from the result of the frequency analysis and input to the phoneme recognition unit 14. The phoneme recognition unit 14 is configured by a neural network with duplicated output. This neural network is composed of an input layer, a hidden layer, and an output layer. For example, a spectrum of five frames is input to the input layer at each time, and a central spectrum thereof indicates which phoneme corresponds. Send by layer unit value. Because the output unit is duplicated,
Two units are associated with each phoneme category. On the other hand, the result selects two units from the one showing the largest output value, and the corresponding phoneme is ranked first,
Obtained as the second phoneme candidate.

[0003] The similarity between the recognized phoneme candidate sequence and the template 15 in the dictionary having the phoneme pattern of the vocabulary to be recognized is determined by the first place in the phoneme candidate sequence recognized as the phoneme in the template. And the similarity with the second-ranking candidate is set as a local score, and the local score is matched by the matching unit 16 by the DTW method, and the total similarity score is obtained by accumulating the most similar words, and all the scores to be recognized are obtained. The word having the minimum similarity score in the vocabulary is output from the matching unit 16 as a recognition result.

[0004]

In the above-described discrete word speech recognition system, recognition processing is performed only after it is determined that the input speech has been completed. Therefore, in the case of recognition requiring a long calculation time such as continuous speech recognition, it takes a considerable time from the end of speech to the output of a recognition result, which is not practical. Also, even for short speech input such as word recognition, processing time is significantly longer when using a personal computer or the like that has a relatively slow calculation process compared to a high-speed workstation.
There is a problem that it is difficult to construct such a device on a personal computer.

The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a speech recognition apparatus and a speech recognition processing method capable of obtaining a speech recognition result almost simultaneously with the end of speech.

[0006]

SUMMARY OF THE INVENTION In order to achieve the above-mentioned object, a first aspect of the present invention provides a voice input device such as a telephone and the like, and phoneme recognition of voice data input from the voice input device. A recognition processing unit for analyzing the frequency of the voice data, inputting the frequency analysis to an output multiplexing neural network to perform phoneme recognition, and
The similarity between the recognized phoneme candidate sequence and the template in the dictionary having the phoneme pattern of the vocabulary to be recognized is obtained by obtaining the phoneme candidate and the second phoneme candidate, and the phoneme recognized as the phoneme in the template. The similarity between the first and second candidates in the candidate sequence is defined as a local score, and the local score is defined as DTW
The overall similarity score is obtained by accumulating by the method,
In a vocabulary to be recognized, in a voice recognition processing device configured to output a word having a minimum similarity score as a recognition result, voice data output from the voice input device is input, An audio processing unit that performs audio processing on the data, a buffer memory unit that sequentially records audio data processed by the audio processing unit into a plurality of divided memories, and a recording unit that stores the data in one of the memory units. A recording end detecting unit for detecting whether or not recording has been completed, and when the detecting unit detects the end of recording in the memory, the output of the buffer memory unit is input to the recognition processing unit. And a memory switching unit for switching to.

According to a second aspect of the present invention, when the recording of the voice data in the buffer memory unit is completed, a backtrace is performed to extract a recognized word string from the recognition processing unit, and a recognition result is obtained. It is.

According to a third aspect of the present invention, after audio data output from an audio input device is processed by an audio processing unit, the processed audio is sequentially recorded in a plurality of divided recording buffer memories. When the recording in one recording buffer memory is completed, the recording in the next memory is started, and the phoneme recognition processing is performed in parallel.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic structural explanatory view showing an embodiment of the present invention. In FIG. 1, reference numeral 21 denotes a voice input device such as a telephone or a microphone, and voice data from the voice input device 21 is input to a voice processing unit 22. Is done. The audio data captured by the audio processing unit (sound board) 22 is sequentially supplied to a recording unit 23 and sequentially recorded in a recording buffer memory (described later). The recording buffer memory is divided into a plurality of memories, and when the recording to one memory is completed (the memory is full), the recording to the next memory is started, and the recording is started. The completed voice data in the buffer memory is stored in the feature extraction unit 24 constituting the recognition processing unit 24.
a, and the recognition processing is performed in parallel.

The voice data from the buffer memory input to the feature extraction unit 24a is subjected to frequency analysis to obtain a spectrum sequence, and this spectrum sequence is input to a phoneme recognition unit 24b composed of a neural network, and a phoneme candidate is output to the phoneme recognition unit 24b. A row is obtained. This candidate column is word spot 24c
(The word spot refers to a calculation for obtaining an optimum starting point and a matching distance when a designated frame in continuous speech is assumed to be the end of the dictionary word), and is collated by the dictionary template 24d and the DTW. Output the most similar words as the result. Output result is cumulative distance calculator 2
5, when there is a voice input, the processing is again performed from the sequential recording processing unit 23, and when the voice input is completed, it is supplied to a back trace (calculation method for extracting a recognized word later) unit 26 to recognize the recognized word. Retrieve a column.

FIG. 2 is a block diagram showing details of the above-mentioned sequential recording section 23. The sequential recording section 23 includes a recording buffer memory section 23a having a plurality of divided memories.
The memory section 23a is provided with a recording end detecting section 23b for detecting the end of voice recording to one memory and a memory switching section 23c. The memory switching unit 23c includes the recording end detection unit 23b.
Is used to switch the start of recording to the next memory when a detection output is sent.

FIG. 3 is a structural view of a cyclic recording buffer memory unit 23a.
Are buffer memories 1, 2,... (N-
1), n, and audio data is input in parallel to the buffer memories 1, 2,... N, but if one memory is not full of recordings, the next memory Is controlled not to be. The control is performed as shown in FIG.

FIG. 4 is an explanatory diagram for explaining an outline of a recording operation to the recording back memory unit 23a. FIG. 4A shows an outline of an operation from an initial state to a silent idle state, A1 shows an initial state, At this time, the number of recording buffers is “0”. A2 indicates a state in which the buffer memory 1 has finished recording, and A3 indicates a state in which the buffer memories 1 and 2 have finished recording.
Reference numeral 4 denotes that the buffer memories 1, 2, and 3 have finished recording, that is, the number of recording buffers has become "4". Then, in the idle state, the uppermost buffer memory is once deregistered and then reregistered.

FIG. 4B describes the operation after the detection of the voice part. When the voiced voice continues, a voice buffer is added.

FIG. 4C illustrates the operation from the detection of a silent portion to the end of recording. When silence continues, the recording is terminated.

Next, the processing flow of the embodiment of FIG. 1 configured as described above will be described with reference to FIG. In this embodiment, sequential recognition processing and sequential recording processing are performed in parallel as processing for recognition. In FIG. 5, when a process start command is issued in step S1, a sequential recording process (S2) to a buffer memory (hereinafter referred to as a buffer) is started. In step S3, it is determined whether or not an event of ending the recording of the buffer being recorded has been issued. If (Y), the audio check of the buffer is performed in step S4. If it is determined that the buffer is a voice part (Y), a sequential recognition process is performed (S5), and the process returns to the sequential recording process of step S2. If the buffer is recognized (N) as a silent section in step S4, it is determined in step S6 whether the end condition is satisfied,
If the end condition is satisfied (Y), the recognition result is output, the recording buffer is reset (S7), and the process ends (S7).
8).

Here, the sequential recording process will be described. This processing means that the recording is performed until the buffer is full, and the control for recording to the next buffer when the buffer is full is continued until it is determined that the voice input is completed. This processing includes (1)
The process includes a recording start process, (2) a process by a sound device driver, (3) a recording process, and (4) a recording stop process. The contents of each process will be described below.

(1) Recording start processing (a) Open a sound device for recording (b) Register a recording buffer group realized as a recording buffer memory (c) Issue a recording start command.

(2) Processing in the sound device driver (a) Record audio data continuously in the order registered in each recording buffer (b) Each time recording to each recording buffer is completed, the number of recorded data And issues a recording end event.

(3) Recording process The recording process is performed like the algorithm shown in FIG. In FIG. 6, first, it is determined whether the number of buffer data in the buffer that caused the recording end event is zero (S11), and if (N), a voice check is performed to determine whether the data is a voice part or a silent part (S12). ). In the check, it is determined whether the data is voiced (S13). If (Y), it is determined in step S14 whether the data is being voiced. As a result of that judgment,
If (Y), add an audio buffer and end the process,
If (N), the number of voices is counted (S15), and if the number of voices continues more than a predetermined NR BUF START (S16) (Y), the flag in the voice is turned on (S17), and the buffer is idle. The state is updated (S18).

On the other hand, if (N) in the step S13, it is determined in a step S19 whether a voice is being played. If the result of the determination is (Y), the number of silences is counted (S20), and if the silence continues for more than a predetermined NR BUF END (S21) (Y), the recording is stopped assuming that it is a silence part. To end the recording (S22). If there is no sound (N) in the judgment of step S19, the idle state of the buffer is updated (S23).

(4) Recording stop processing (a) Issuing a recording stop command and a reset command,
Stop recording. (B) Cancel registration of recording buffer. (C) Close sound device.

Next, the sequential recognition process will be described. In the sequential recognition process, every time a recorded buffer is sent,
This is a process in which recognition calculation is performed in consideration of the calculation results up to that time. As this processing, two systems of discrete word recognition and continuous word recognition were implemented. In both cases, the recorded buffers are adapted to be sequentially input. In addition, another processing can be performed without waiting for the end of the processing of the recognition processing unit, thereby improving the efficiency and speed of the processing. Here, examples of the above two methods will be described.

Example 1: Discrete Word Recognition Method In FIG. 7, recognition is started, and in step S31, an invoic
After confirming that e flg (voice detection flag) = OFF and buff count (buffer counter) = 0, the voice detection flag invoice flg is turned on each time a voice is detected in the voice check of the recorded buffer (S32). Set (S33), otherwise set to off (S34). If the voice detection flag invoice flg (S35) is ON (Y), it is determined whether the recognition processing unit is performing the recognition processing (S36). If the recognition processing unit is processing (Y), the transfer of the recorded buffer is postponed.
Return to 32. If the processing is not being performed in step S36 (N), the recorded buffer is transferred to the recognition processing unit to perform recognition processing (S37), and the buffer count is incremented, and step S32 is performed without waiting for the end of the recognition processing unit.
Return to The recognition process differs depending on the value of the buffer count.

When buff count = 1, the recognition processing unit performs frequency analysis, phoneme recognition, and DTW. Then, the DT with each dictionary template required for the calculation of the next buffer
Store the result of W in the specified location.

When buff count = 2, the recognition processing unit extracts the stored calculation result and sets it as the initial value of the DTW. Thereafter, in the same manner as described above, the frequency analysis, phoneme recognition, and DTW of the input buffer are performed, and the result of the DTW of each dictionary template is stored in the designated location.

The voice detection flag invoice flg (S3
If 5) is not on, that is, if it is off (N), it is determined whether there is any recorded buffer that has not been recognized (S).
38), if there is any unrecognized one (Y), the transfer to the recognition processing unit and the recognition are repeated until all the unprocessed buffers complete the recognition processing (S39). After that, a recognition result is obtained, and the process is terminated. If (N) in step S38, the recognition process ends. FIG. 8 is a timing chart of the above-described recognition processing and recording.

Example 2: Continuous Word Recognition Method In FIG. 9, recognition is started, and in step S41, an invoic
After confirming that e flg (voice detection flag) = OFF and buff count (buffer counter) = 0, the voice detection flag invoice flg is turned on each time a voice is detected in the voice check of the recorded buffer (S42). Set (S43), otherwise set to off (S44). If the voice detection flag invoice flg (S45) is ON (Y), it is determined whether the recognition processing unit is performing the recognition processing (S46). If the recognition processing unit is processing (Y), the transfer of the recorded buffer is postponed.
Return to 42. If processing is not being performed in step S46 (N), the recorded buffer is transferred to the recognition processing unit to perform recognition processing (S47), and the buffer count is incremented. The recognition process differs depending on the value of the buffer count. This determination is made in step S48.

If (N) in step S48, buff count =
Since it is 1, when buff count = 1, the recognition processing unit performs frequency analysis, phoneme recognition, and word spot DTW. After that, the result of the word spot necessary for the calculation of the next buffer is stored in the specified location. The process returns to step S42 without waiting for the end of the recognition processing unit.

If (Y) in step S48, buff c
Since bunt count = 2 because ount = 2, the recognition processing unit extracts the stored calculation result and sets the word spot value as an initial value. In the same manner as above, the frequency analysis, phoneme recognition, and word spot DTW of the input buffer are performed.
And store the result needed for the next calculation. afterwards,
Without waiting for the end of the processing of the recognition processing unit, the calculation of the cumulative distance to the previous buffer is performed in step S49, and the process returns to step S42.

The voice detection flag invoice flg (S4
If 5) is not on, that is, if it is off (N), it is determined whether there is any recorded buffer that has not been recognized (S).
50) If there is any unprocessed buffer (Y), transfer and recognition to the recognition processing unit are performed until all the unprocessed buffers complete the recognition process (S51). After the recognition processing, a cumulative distance calculation is performed in step S52, a back trace (S53) is performed using the calculated cumulative distance result, and a recognition result is obtained.
The process ends. If (N) in step S50, a back trace (S53) is performed and the recognition process ends.
FIG. 10 is a timing chart of the above-described recognition processing and recording.

FIG. 11 is a system configuration diagram when the above-described embodiment is processed by a personal computer. In FIG. 11, voice from a voice input device 21 such as a telephone is transmitted to the public line network 10.
0 to the network controller 101,
The voice processing unit 22 of the personal computer 102 performs voice processing from the network controller 101 to perform continuous word recognition on the personal computer 102. 103 is a speech synthesizer.

If the recognition processing section 24 is configured to perform parallel processing, the processing can be speeded up.

[0034]

As described above, according to the present invention,
The recognition and recording processes are performed sequentially each time recording to one buffer is completed, and the recognition processing unit and the computer process are performed in parallel. At the same time, there is an advantage required at a high speed. In addition, since a personal computer is used, the system can be easily configured.

[Brief description of the drawings]

FIG. 1 is a schematic configuration explanatory view illustrating an embodiment of the present invention.

FIG. 2 is a configuration diagram of a sequential recording process of a main part of the embodiment.

FIG. 3 is a structural diagram of a cyclic recording buffer memory.

FIG. 4 is a schematic diagram of a recording operation to a cyclic recording buffer memory.

FIG. 5 is a flowchart illustrating a flow of an embodiment.

FIG. 6 is a flowchart of an algorithm of a recording process.

FIG. 7 is a flowchart of a recognition process of a discrete word recognition method.

FIG. 8 is a timing chart of recognition and recording of the discrete word recognition method.

FIG. 9 is a flowchart of a recognition process of the continuous word recognition method.

FIG. 10 is a timing chart of recognition and recording in the continuous word recognition system.

FIG. 11 is a system configuration diagram of the embodiment.

FIG. 12 is a schematic diagram of a speech recognition system.

[Explanation of symbols]

 Reference Signs List 21 voice input device 22 voice processing unit 23 sequential recording processing unit 24 recognition processing unit 25 cumulative distance calculation unit 26 backtrace unit

Claims (3)

[Claims] 1. A speech input device comprising a telephone or the like, and a recognition processing unit for recognizing phonemes of speech data input from the speech input device, wherein the recognition processing unit analyzes the frequency of the speech data, Input to the output multiplexing neural network to perform phoneme recognition to obtain the first phoneme candidate and the second phoneme candidate for the recognized phoneme, and to give the recognized phoneme candidate sequence and the phoneme pattern of the vocabulary to be recognized. The similarity with the template in the extracted dictionary is defined as the similarity between the phoneme in the template and the first and second candidates in the sequence of recognized phoneme candidates, and the local score is accumulated by the DTW method. Thus, in the speech recognition processing device configured to obtain the overall similarity score and output, as a recognition result, a word having the minimum similarity score among all the vocabularies to be recognized, A voice processing unit that receives voice data output from a voice input device and processes the data, and a buffer memory unit that sequentially records the voice data processed by the voice processing unit in a plurality of divided memories. A recording end detecting unit that detects whether recording to one memory of the memory unit is completed, and an output of the buffer memory unit to the recognition processing unit when the detecting unit detects the end of recording of the memory. And a memory switching unit for inputting and switching to a next recording start memory of the buffer memory unit. 2. The speech recognition processing device according to claim 1, wherein when the recording of the speech data in the buffer memory unit is completed, a backtrace is performed to extract a recognized word string from the recognition processing unit. A speech recognition processing device for obtaining a result. 3. An audio processing unit processes audio data output from the audio input device, and then sequentially records the processed audio in a plurality of divided recording buffer memories.
A phoneme recognition processing method characterized in that when the recording in one of the divided recording buffer memories is completed, the recording in the next memory is started and the phoneme recognition processing is performed in parallel.