JP3163109B2 - Multi-directional simultaneous voice pickup speech recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-directional simultaneous sound pickup speech recognition method for simultaneously picking up and recognizing sounds from multiple directions.

[0002]

2. Description of the Related Art Speech recognition devices have begun to be used as powerful input means for computers and various other devices. FIG. 4 is a block diagram of a typical example of a conventional recognition device as disclosed in Japanese Patent Application Laid-Open No. 62-73298.
1 is a voice input terminal, 402 is a voice analysis unit, 403 is a voice section detection unit, 404 is a comparison matching pattern memory unit, 405 is a similarity calculation unit, and 406 is a determination unit.

[0003] A voice signal input from a voice input terminal 401 is converted by a voice analysis unit 402 into a time series of feature vectors representing voice features.

[0004] The voice section detection section 403 includes a voice analysis section 40.
Section where voice is present based on the feature vector from 2,
That is, the voice section is determined. This feature vector sequence from the speech start to the speech end is called a speech matching pattern.

Next, a comparison matching pattern memory section 40
The function of No. 4 will be described. In specific speaker speech recognition for limiting a speaker, a word to be recognized (referred to as a category)
Must be uttered in advance (referred to as learning speech), and a speech matching pattern (referred to as a comparison matching pattern) obtained by performing the same speech analysis must be stored. The comparison pattern memory unit 404 stores such a comparison matching pattern. The operation of storing the comparison matching pattern is called a registration process. In the case of speech recognition that does not limit a speaker (referred to as unspecified speaker speech recognition), comparison matching patterns of various speakers are stored in a comparison pattern memory unit 40.
4 is stored in advance.

The similarity calculator 405 calculates the similarity between an input matching pattern generated from input speech to be recognized and a comparison matching pattern. Similarity calculation is performed by a well-known method such as DP matching or JP-A-62-732.
Various methods have been proposed, including simple linear matching as disclosed in Japanese Patent Publication No. 99-99, and similarity calculation is performed by any appropriate method.

Using the similarity for each category output from the similarity calculator 405, the determiner 406 outputs the category name given to the comparison matching pattern giving the maximum similarity as a recognition result.

[0008]

In the above-described conventional recognition device, since sound is collected from one microphone,
In an input form in which the distance and direction between the vocal organ and the microphone, such as a close-talking microphone, are constant, the recognition ability of the voice recognition device has always been able to be fully exhibited. However,
In an input mode in which the distance and direction between the vocal organ and the microphone change greatly, the recognition performance often decreases extremely. Furthermore, even in the case where a close-talking microphone is used, in an environment where the background background noise greatly changes, the recognition performance is still not stable, and a malfunction or erroneous recognition may occur.
SUMMARY OF THE INVENTION An object of the present invention is to provide a voice recognition method capable of obtaining a stable and high recognition performance even in an environment where the distance and direction between a vocal organ and a microphone change greatly and the background noise environment changes greatly. .

[0009]

According to a first aspect of the present invention, there is provided a voice recognition method for achieving this object, wherein voices are simultaneously collected from a plurality of microphones to obtain a plurality of input signals. It is characterized by comprising a process of obtaining a plurality of identification results by independently performing voice recognition on the obtained plurality of input signals, and a process of integrally determining the plurality of identification results. At this time, when the majority of the plurality of inputs match, the integrated identification result is determined as the final identification result. Of the plurality of inputs, the input with the highest identification accuracy is determined as the final identification result. There is a method of judging, or judging the input having the largest signal-to-noise ratio among a plurality of inputs as the final identification result.

The second aspect of the present speech recognition method is that, based on the recognition result determined by the speech recognition method, the main input system having the best speech signal / background noise ratio in each input system and the speech A process for determining a noise input system having the worst signal / background noise ratio; and a process for configuring an adaptive noise elimination filter from the main input system and the noise input system. It is characterized in that speech is identified from an input signal.

[0011]

According to the first configuration, an identification operation is performed independently on a plurality of input signals picked up simultaneously from a plurality of microphones, and the plurality of identification results and recognition auxiliary information are comprehensively determined by an integrated determination unit. The second configuration constitutes an adaptive noise elimination filter based on the recognition result determined by the recognition determination by the first configuration, and thereafter, the input signal after the filtering process is performed. According to the above configuration, recognition is performed, and speech recognition can be performed using an input signal having the best signal-to-noise ratio in consideration of the position of the noise utterance source. Therefore, by using the voice recognition method of the present invention, a voice recognition device capable of obtaining a stable and high recognition performance even in an environment in which the distance and direction between the vocal organ and the microphone greatly changes and the background noise environment greatly changes. Can be realized.

[0012]

Embodiments of the present invention will be described below. Here, the voice matching pattern means a pattern created in a generation process common to the input matching pattern and the comparison matching pattern.

FIG. 1 is a block diagram showing a first embodiment of the present invention applied to a specific speaker recognition system for performing a registration operation. Reference numerals 101, 102, and 103 denote a first channel, a second channel, and a second channel, respectively. 3 channel audio input terminal,
Reference numerals 104, 105, and 106 denote sound analysis units for the first channel, the second channel, and the third channel, respectively.
Reference numerals 108 and 109 denote voice section detectors for the first, second and third channels, respectively.
Reference numerals 1 and 112 denote comparison pattern memory sections of the first channel, the second channel and the third channel, respectively.
Reference numerals 114 and 115 denote a first-channel, second-channel, and third-channel audio discriminating units, respectively, an integrated determining unit 116, and an output terminal 117.

Although the first embodiment shows an example of three audio input signals for simplicity of explanation, it is possible to provide a larger number of audio signals. Also here 1
The functional blocks 01, 104, 107, 110, and 113 are collectively referred to as a first channel audio processing unit. Similarly, functional blocks 102, 105, 108, 111, and 114 are collectively called a second channel audio processing unit.
The function blocks 3, 106, 109, 112, and 115 are collectively referred to as a third channel audio processing unit.

First, the process of identifying an audio signal input to the first channel will be described. The audio signal input from the audio input terminal 101 is converted into a time series of feature vectors representing the features of the audio in the audio analysis unit 104. As a method for deriving the feature vector, a method using a plurality of bandpass groups having slightly different center frequencies, a method using spectral analysis by FFT (Fast Fourier Transform), and the like can be considered. Here, a bandpass filter group is used. The method is taken as an example.

After the audio signal is converted from analog to digital by the audio analyzer 104, each band-pass filter extracts only its frequency component. The sequence of data sorted by each bandpass filter in this manner is called a channel. The output of the filter for each channel is rectified to obtain an absolute value, and the average value is calculated for each frame. This calculated value becomes the magnitude of the feature vector of each channel in the frame. That is, the magnitude Aij of the feature vector in the ith frame is Aij = (Ai1, Ai2,..., Aip). Here, p is the number of channels.

In the voice section detection section 107, the voice analysis section 10
Section where voice is present based on the feature vector from
That is, the voice section is determined.

The comparison pattern memory unit 110 stores a comparison matching pattern of a recognition target category, and the operation of storing the comparison matching pattern is called a registration process.
Here, as an example of the registration processing, a case where one learning voice is uttered once per category is taken for simplification of description. When the total number of categories is N, the comparison pattern memory has N comparison matching patterns Sln (n = 1, 2, 2).
.., N) are stored.

In comparison between matching patterns of voice,
It is necessary to take a time response between the two. A typical method of calculating the similarity between the two patterns while taking the optimum correspondence is a so-called DP matching method as shown in Japanese Patent Publication No. 50-23941. The voice identification unit 113 calculates the similarity between patterns using such a DP matching method or another suitable method. That is, the similarity between the input matching pattern I1 generated from the input speech to be recognized and all the comparison matching patterns SIn in the comparison pattern memory unit 110 is obtained, and the speech feature similarity XIn is obtained.

Among the voice feature similarities Xln for each pattern, the category name C1 given to the comparison matching pattern giving the maximum value P1 is output to the integration determination section 116 as the voice identification result for the channel. Further, as the identification auxiliary information, the maximum value P1 of the audio feature similarity (referred to as identification accuracy), the start and end times VS1, VE1 of the audio, the audio level (the signal level of the audio section) and the noise level (the non-input state immediately before the audio section). Also, the ratio SN1 (referred to as signal / noise ratio) of the signal level is output to the integration determination unit 116.

The process of identifying the audio signal input to the second channel is performed in exactly the same manner as in the case of the first channel. That is, the audio signal input from the audio input terminal 102 is converted by the audio analysis unit 105 into a time series of feature vectors representing audio features. Voice section detection unit 10
In step 8, a speech section is determined based on the feature vector from the speech analysis unit 105, and a feature vector sequence of that portion, that is, an input matching pattern I2 is generated. Next, the speech identification unit 114 calculates the similarity between the input matching pattern I2 and all the comparison matching patterns S2n in the comparison pattern memory unit 111, and obtains the speech feature similarity X2n.

Among the voice feature similarities X2n for each pattern, the category name C2 given to the comparison matching pattern giving the maximum value is output to the integration determination unit 116 as a voice identification result for the channel. Further, as identification auxiliary information, the identification accuracy P2, the voice start / end time VS
2, VE2, signal / noise ratio SN2, etc.
16 is output.

The process of identifying the audio signal input to the third channel is performed in exactly the same manner as in the case of the first channel, and the integrated determination unit 116 sends the audio identification result C3 and the identification auxiliary information (identification accuracy P3, audio Start and end times VS3, V
E3, signal / noise ratio SN3, etc.).

The integrated determination unit 116 makes a comprehensive determination based on the audio identification result and the identification auxiliary information output from the audio processing unit of each channel, and sends the final recognition result from the output terminal 117 to an external host or the like. Various methods can be considered for this determination method. For a group of identification information whose voice start and end times are almost the same, only when the majority of the identification results match, the method of validating the identification result or the identification accuracy P A method of determining the highest discrimination result as valid, a method of validating the discrimination result of the channel having the highest signal / noise ratio SN, and the like are given. The final determination result may be calculated using the above-described determination method or another suitable method.

FIG. 2 is a block diagram showing a second embodiment of the present invention. In the second embodiment, three audio input signals 20
The speech analysis unit 204, the speech section detection unit 207, and the speech identification unit 213 are operated in a time-division manner with respect to 1, 202, and 203 to perform speech identification independently.
C2, C3 and identification auxiliary information (identification accuracy P1, P2, P
3. Voice start time VS1, VS2, VS3, voice end time VE1, VE2, VE3, signal / noise ratio SN
1, SN2, SN3). The comparison pattern memory unit 210 is common. The processing of the overall judgment unit 216 is the same as that of the first embodiment, and the final recognition result is output to the output terminal 21.
7 to an external host or the like.

FIG. 3 is a block diagram showing a third embodiment of the present invention. The operation of the third embodiment is divided into a noise adaptive processing mode and a non-noise adaptive processing mode. In the non-noise adaptive processing mode, the adaptive noise removing unit 318 is removed, and the operation of the recognition processing is exactly the same as in the second embodiment.
After performing the speech recognition in the non-noise adaptive processing mode, the processing shifts to the noise adaptive processing mode. In the noise adaptive processing mode, the following processing is performed prior to recognition.

The speech recognition results C1, C2,
C3 and identification auxiliary information (identification accuracy P1, P2, P3, audio start time VS1, VS2, VS3, audio end time V
E1, VE2, VE3, signal / noise ratio SN1, SN
2, SN3) and a recognition result Cr (sent from an external host) after the final confirmation determines a main input system and a noise input system. The decision algorithm is as follows.

(1) An input system that has output a speech recognition result that matches the recognition result Cr after the final confirmation is selected as a correct answer input system. If there is no such input system, the operation does not shift to the noise adaptive mode, and the next recognition processing is also performed in the non-noise adaptive mode. (2) The input system having the highest signal / noise ratio SN among the correct input systems is defined as the main input system. (3) Among the input systems other than the main input system, the input system with the lowest signal / noise ratio SN is defined as the noise input system.

Next, recognition processing in the noise adaptive processing mode will be described. First, the adaptive noise elimination unit 318 performs adaptive noise elimination filtering from the input signal of the main input system and the input signal of the noise input system to generate a noise-eliminated audio signal. Thereafter, speech recognition is performed on the noise-removed speech signal in the same manner as in the first embodiment. That is, the noise-removed voice signal is converted into a time series of feature vectors representing the features in the voice analysis unit 304. The speech section detection unit 307 determines a speech section based on the feature vector from the speech analysis unit 304, and generates a feature vector sequence of that portion, that is, an input matching pattern I. Next, the speech identification unit 313 obtains the similarity between the input matching pattern I and all the comparison matching patterns Sn in the comparison pattern memory unit 310, and obtains the speech feature similarity Xn. Among the voice feature similarities Xn for each pattern, the category name C given to the comparison matching pattern giving the maximum value is output to the integration determination unit 316 as a voice identification result. The integration determination unit 316 sends the category name C as it is as a final recognition result from the output terminal 317 to an external host or the like.

After the recognition result is confirmed, correct / incorrect information of the recognition result is received from the host. If the recognition result is incorrect at this time, it is considered that the selection of the main input system and the noise input system has become inappropriate due to a change in the noise environment or the like. Redo the selection. Conversely, if the recognition result is correct, all the settings are considered to be appropriate, and the recognition in the noise adaptive processing mode is continued. Note that the selection of the processing mode based on the recognition result is an example. For example, the user may select the processing mode. Further, a method of determining the change of the processing mode based on the frequency of erroneous recognition is also conceivable. FIG.
9 shows the operation flow of the embodiment.

The adaptive noise elimination filtering processing is evident in, for example, the lecture “Application of Digital Filters in Microphone System” in the Journal of the Acoustical Society of Japan, Vol. 45, No. 2 (1989), and references described therein. Description is omitted. However, the voice section detection unit 30
It is assumed that the adaptive operation is stopped (see FIG. 5) from the time when the voice start end is detected to the time when the voice end is detected in step 7, and the coefficient of the adaptive elimination filter of the adaptive noise elimination unit 318 is fixed.

[0032]

As described above in detail, according to the present invention, an identification operation is performed independently on input signals picked up from a plurality of microphones at the same time, and comprehensively using the plurality of recognition results. An adaptive noise elimination filter is configured based on the recognition result obtained in the non-noise adaptive processing mode for performing the recognition determination and as shown in the third embodiment. Since the recognition judgment is performed based on the signal, there is an advantage that a stable and high recognition performance can be obtained even in an environment in which the distance and direction between the vocal organ and the microphone are largely changed, and the background noise environment is largely changed. . For example, when the voice recognition device is used in a place where the position and size of a noise source change irregularly, such as in a car, an office, a factory, or a street, the recognition performance is significantly improved by performing the adaptive operation according to the present invention. Can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the present invention.

FIG. 3 is a block diagram showing a third embodiment of the present invention.

FIG. 4 is a block diagram of a conventional recognition device.

FIG. 5 is a diagram showing an operation flow of a third embodiment.

[Explanation of symbols]

 101 audio input terminal of the first channel 102 audio input terminal of the second channel 103 audio input terminal of the third channel 104 audio analysis unit of the first channel 105 audio analysis unit of the second channel 106 audio analysis unit of the third channel 107 1-channel voice section detection section 108 2nd channel voice section detection section 109 3rd channel voice section detection section 110 1st channel comparison pattern memory section 111 2nd channel comparison pattern memory section 112 3rd channel comparison pattern Memory unit 113 First-channel audio identification unit 114 Second-channel audio identification unit 115 Third-channel audio identification unit 116 Integration determination unit 117 Output terminal

Continuation of the front page (56) References JP-A-59-23397 (JP, A) JP-A-2-178699 (JP, A) JP-A-58-143396 (JP, A) JP-A-58-52696 (JP) JP-A-60-16669 (JP, A) JP-A-63-18400 (JP, A) JP-A-61-35496 (JP, A) JP-A-61-35495 (JP, A) 4-240898 (JP, A) JP-A-4-240897 (JP, A) JP-A-4-199197 (JP, A) JP-A-4-273298 (JP, A) JP-A-4-212600 (JP, A A) Japanese Utility Model Application Sho 57-116999 (JP, U) Japanese Utility Model Application Sho 57-69067 (JP, U) Japanese Utility Model Application No. 2-41680 (JP, Y2) Journal of the Acoustical Society of Japan, Vol. 45, No. 2, Yutaka Kaneda “Application of Digital Filter in Microphone System-Technology for Removing Unwanted Sound-”, p. 125-128 (Issued February 1989) (58) Fields investigated (Int. Cl. ⁷ , DB name) G10L 15/00-17/00

Claims (3)

(57) [Claims] Claims: 1. A sound is simultaneously picked up from a plurality of microphones,
A process of obtaining a plurality of input signals; a process of obtaining a plurality of identification results by independently performing voice recognition on the plurality of input signals; and a case where a majority of the plurality of identification results match, the matching identification results are And a process of determining as a final identification result. 2. A method according to claim 1, wherein
Based on the recognition result
Main input system with the best signal / background noise ratio,
Processing to determine the noise input system with the worst background noise ratio
And an adaptive noise elimination filter from the main input system and the noise input system.
And a process for forming a filter. In the subsequent recognition process, the input signal is subjected to the adaptive noise elimination.
Sound on the output filtered by the
A voice recognition method characterized by voice recognition. 3. A method for simultaneously collecting sounds from a plurality of microphones,
A process of obtaining a plurality of input signals;
Processing to obtain another result, and among the plurality of identification results, a signal
Processing to determine the one with the maximum noise-to-noise ratio as the identification result
And management, based on the determined recognition result by the determination process, the correct answer
Processing for determining the input system; and when the correct input system is determined, the correct input system
Noise in the said audio signal / background noise ratio is best primary input system in each input system, a process of determining the worst noise input system of the audio signal / background noise ratio, the main input system and the A speech recognition method comprising: constructing an adaptive noise elimination filter from an input system; and performing speech recognition on an output obtained by filtering the input signal by the adaptive noise elimination filter in a subsequent recognition process.