JPH04318900A - Multidirectional simultaneous sound collection type voice recognizing method - Google Patents

Abstract

PURPOSE:To obtain stable recognition performance even in environment wherein the distance and direction between the voicing windpipe and a microphone change and background noise environment changes. CONSTITUTION:Voices which are collected through plural microphones at the same time are inputted from input terminals 101, 102, and 103 and passed through voice analysis parts 104, 105, and 106 and voice section detection part 107, 108, and 109, and comparison pattern memory parts 110, 111, and 112 are referred to, so that voice identification parts 113, 114, and 115 recognize them independently of one another. A total decision part 116 totally decides the results of the independent recognition and identification auxiliary information (identification accuracy, start and end times of voice, and signal-to-noise ratio) and outputs the final recognition result to an output terminal 117.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multidirectional simultaneous sound recognition method for simultaneously collecting and recognizing sounds from multiple directions.

0002

2. Description of the Related Art Speech recognition devices are beginning to be used as powerful input means for computers and various other devices. FIG. 4 is a block diagram of a typical example of a conventional recognition device as shown in Japanese Unexamined Patent Publication No. 62-73298.
1 is a voice input terminal, 402 is a voice analysis section, 403 is a voice section detection section, 404 is a comparison matching pattern memory section, 405 is a similarity calculation section, and 406 is a determination section.

[0003] An audio signal input from an audio input terminal 401 is converted into a time series of feature vectors representing audio features in an audio analysis section 402 .

[0004] In the speech section detection section 403, the speech analysis section 40
2. Based on the feature vector from 2, the section where the voice exists,
That is, the voice section is determined. This feature vector sequence from the beginning of the voice to the end of the voice is called a voice matching pattern.

Next, a comparison matching pattern memory section 40
Function 4 will be explained. In speaker-specific speech recognition that limits speakers, words to be recognized (referred to as categories)
It is necessary to utter in advance (referred to as a learning voice) and store a voice matching pattern (referred to as a comparison matching pattern) obtained by performing the same voice analysis. The comparison pattern memory section 404 stores such comparison matching patterns. This comparison matching pattern storage operation is called registration processing. In the case of speech recognition that does not limit speakers (referred to as speaker-independent speech recognition), comparison matching patterns of various speakers are stored in the comparison pattern memory section 40.
4 is stored in advance.

[0006] The similarity calculation unit 405 calculates the similarity between the input matching pattern generated from the input speech to be recognized and the comparison matching pattern. Similarity calculations are performed using well-known methods such as DP matching and Japanese Patent Application Laid-Open No. 62-732.
Various methods have been proposed, including a simple linear matching method as shown in Japanese Patent No. 99, and similarity calculation is performed using any suitable method.

Using the degree of similarity for each category outputted from the similarity degree calculation section 405, the determination section 406 outputs the category name given to the comparison matching pattern that gives the maximum degree of similarity as a recognition result.

[0008]

[Problems to be Solved by the Invention] In the conventional recognition device described above, since the sound is collected from one microphone,
In an input mode such as a close-talk type microphone in which the distance and direction between the vocal organ and the microphone are constant, the recognition ability of the speech recognition device is always maximized. however,
In input formats in which the distance and direction between the vocal organ and the microphone change significantly, recognition performance often deteriorates dramatically. Furthermore, even when a close-talking microphone is used, recognition performance is still unstable in an environment where the surrounding background noise changes significantly, and malfunctions and erroneous recognition may occur. An object of the present invention is to provide a speech recognition method that can obtain stable and high recognition performance even under environments where the distance and direction between the vocal organ and the microphone change greatly, and the background noise environment changes greatly. .

[0009]

[Means for Solving the Problems] The present speech recognition method for achieving this object includes, as a first configuration, a process of simultaneously collecting sounds from a plurality of microphones to obtain a plurality of input signals; The present invention is characterized by comprising a process of independently voice-identifying a plurality of input signals to obtain a plurality of identification results, and a process of performing an integrated judgment on the plurality of identification results.

[0010] Furthermore, the present speech recognition method has a second configuration in which the main input system with the best speech signal/background noise ratio in each input system and the speech signal are selected based on the recognition results determined by the speech recognition method. It includes a process of determining the noise input system with the worst signal/background noise ratio, and a process of configuring an adaptive noise removal filter from the main input system and the noise input system. It is characterized by identifying voices from input signals.

[0011]

[Operation] The first configuration performs identification operations independently for multiple input signals picked up simultaneously from multiple microphones, and comprehensively determines the multiple identification results and recognition auxiliary information by the integrated determination unit. The second configuration configures an adaptive noise removal filter based on the recognition result determined by the recognition determination using the first configuration. According to the above configuration, speech recognition can be performed using an input signal with the best signal-to-noise ratio in consideration of the position of the noise source. Therefore, by using the speech recognition method of the present invention, it is possible to create a speech recognition device that can obtain stable and high recognition performance even in environments where the distance and direction between the vocal organ and the microphone change greatly, and where the background noise environment changes greatly. It can be achieved.

0012

[Examples] Examples of the present invention will be described below. Note that the speech matching pattern here means a pattern created in a common generation process for the input matching pattern and the comparison matching pattern.

FIG. 1 is a block diagram showing a first embodiment of the present invention applied to a specific speaker recognition system that performs a registration operation. 3 channel audio input terminal, 1
04, 105, 106 are the first channel and the second channel, respectively.
Channel, 3rd channel audio analysis section, 107,1
08 and 109 are audio section detection units for the first channel, second channel, and third channel, respectively; 110 and 111;
, 112 are the first channel, the second channel,
3rd channel comparison pattern memory section, 113, 11
4 and 115 are audio identification units for the first channel, second channel, and third channel, respectively; 116 is an integrated determination unit;
117 is an output terminal.

Although the first embodiment shows an example of three lines as the audio input signal to simplify the explanation, there is no problem in providing a larger number of lines. Again here 1
The functional blocks 01, 104, 107, 110, and 113 are collectively referred to as a first channel audio processing section. Similarly, the functional blocks 102, 105, 108, 111, and 114 are collectively called the second channel audio processing section.
The functional blocks 3, 106, 109, 112, and 115 are collectively referred to as a third channel audio processing section.

First, the identification process of the audio signal input to the first channel will be explained. An audio signal input from the audio input terminal 101 is converted into a time series of feature vectors representing the characteristics of the audio in the audio analysis unit 104. Possible methods for deriving feature vectors include using multiple bandpass groups with slightly different center frequencies, and using spectral analysis using FFT (fast Fourier transform).
Here, a method using a group of bandpass filters will be exemplified.

After the audio signal is analog-to-digital converted by the audio analysis section 104, only its frequency components are extracted by each bandpass filter. A series of data distributed by each bandpass filter in this manner is called a channel. The output of the filter for each channel is rectified, the absolute value is taken, and the average value is calculated for each frame. This calculated value becomes the size of the feature vector of each channel in that frame. That is, the size Aij of the feature vector in the i-th frame is Aij=
(Ai1, Ai2, ..., Aip). Here p is the number of channels.

In the speech section detection section 107, the speech analysis section 10
4. Based on the feature vector from 4, the section where the voice exists,
That is, the voice section is determined.

Comparison matching patterns of categories to be recognized are stored in the comparison pattern memory section 110, and the operation of storing the comparison matching patterns is called registration processing. Here, as an example of the registration process, for the purpose of simplifying the explanation, we will take up a case where the learning voice is uttered once per category. When the total number of categories is N, the comparison pattern memory contains N comparison matching patterns Sln (n=1, 2, .
..., N) are stored.

In comparing audio matching patterns,
It is necessary to accommodate both parties in terms of time. A typical method for calculating the similarity between two patterns while taking an optimal correspondence is the so-called DP matching method as shown in Japanese Patent Publication No. 50-23941. The speech identification unit 113 calculates the similarity between patterns using such a DP matching method or other suitable method. That is, the degree of similarity between the input matching pattern I1 generated from the input voice to be recognized and all the comparison matching patterns Sln in the comparison pattern memory section 110 is determined to obtain the degree of voice feature similarity Xln.

Among the voice feature similarities Xln for each pattern, the category name C1 given to the comparison matching pattern that gives the maximum value P1 is output to the integrated determination section 116 as the voice identification result for that channel. Furthermore, as identification auxiliary information, the maximum value P1 of voice feature similarity (referred to as identification accuracy), voice start and end times VS1, VE1, voice level (signal level of voice section) and noise level (no input state immediately before voice section) The ratio SN1 (referred to as signal/noise ratio) of the signal level at

The identification process for the audio signal input to the second channel is performed in exactly the same manner as in the case of the first channel described above. That is, the audio signal input from the audio input terminal 102 is converted into a time series of feature vectors representing audio features in the audio analysis section 105. Voice section detection unit 10
In step 8, a speech section is determined based on the feature vectors from the speech analysis section 105, and a feature vector series for that part, that is, an input matching pattern I2 is generated. Next, the voice identification section 114 determines the degree of similarity between the input matching pattern I2 and all comparison matching patterns S2n in the comparison pattern memory section 111, and obtains the degree of voice feature similarity X2n.

Of the voice feature similarities X2n for each pattern, the category name C2 given to the comparison matching pattern that gives the maximum value is output to the integrated determination section 116 as the voice identification result for that channel. Furthermore, as identification auxiliary information, identification accuracy P2, audio start and end time VS2,
VE2, signal/noise ratio SN2, etc. are also integrated in the judgment unit 116.
Output to.

The identification process for the audio signal input to the third channel is performed in exactly the same way as for the first channel described above, and the integrated determination unit 116 is sent the audio identification result C3 and the identification auxiliary information (identification accuracy P3, audio start and end time VS3,V
E3, signal/noise ratio SN3, etc.) are output.

[0024] The integrated determination unit 116 performs a comprehensive determination on the voice recognition results output from the voice processing units of each channel and the identification auxiliary information, and sends the final recognition result to an external host or the like from an output terminal 117. Various methods can be considered for this determination, such as a method in which the identification results are valid only when a majority of the identification results match for a group of identification information with almost the same voice start and end times, and a method in which the identification accuracy P is Examples include a method in which the highest identification result is determined to be valid, and a method in which the identification result of the channel with the highest signal/noise ratio SN is determined to be valid. The final determination result may be calculated using the determination method described above or other suitable method.

FIG. 2 is a block diagram showing a second embodiment of the present invention. In the second embodiment, three audio input signals 20
1, 202, and 203, the speech analysis section 204, speech section detection section 207, and speech identification section 213 are operated in a time-sharing manner to independently perform speech identification, and the speech identification results C1,
C2, C3 and identification auxiliary information (identification accuracy P1, P2, P3
, audio start time VS1, VS2, VS3, audio end time VE1, VE2, VE3, signal/noise ratio SN1
, SN2, SN3). Further, the comparison pattern memory section 210 is common. The processing of the comprehensive judgment unit 216 is the same as that of the first embodiment, and the final recognition result is sent to the output terminal 217.
Send it to an external host, etc.

FIG. 3 is a block diagram showing a third embodiment of the present invention. The operation of the third embodiment is divided into a noise adaptive processing mode and a non-noise adaptive processing mode. In the no-noise adaptive processing mode, the adaptive noise removal section 318 is removed, and the recognition processing operation is exactly the same as in the second embodiment. The noise adaptive processing mode is entered after speech recognition is performed in the non-noise adaptive processing mode. In the noise adaptive processing mode, the following processing is performed prior to recognition.

[0027] Speech identification results C1, C2, at the time of immediately preceding recognition,
C3 and identification auxiliary information (identification accuracy P1, P2, P3, audio start time VS1, VS2, VS3, audio end time V
E1, VE2, VE3, signal/noise ratio SN1, SN
2, SN3) and the recognition result Cr after final confirmation (sent from the external host), the main input system and noise input system are determined. The decision algorithm is as follows.

(1) The input system that outputs the voice recognition result that matches the recognition result Cr after final confirmation is selected as the correct input system. If there is no such input system, the next recognition process will be performed in the non-noise adaptive mode without shifting to the noise adaptive mode. (2) The input system with the highest signal/noise ratio SN among the correct input systems is set as the main input system. (3) Among the input systems other than the main input system, the input system with the lowest signal/noise ratio SN is set as the noise input system.

Next, recognition processing in the noise adaptive processing mode will be explained. First, the adaptive noise removal section 318 performs adaptive noise removal filtering on the input signal of the main input system and the input signal of the noise input system to generate a noise removed audio signal. After this, voice identification is performed on the noise-removed voice signal in the same manner as in the first embodiment. That is, the noise-removed audio signal is converted by the audio analysis unit 304 into a time series of feature vectors representing features. In the speech section detection section 307, the speech analysis section 3
Determine the speech interval based on the feature vector from 04,
A feature vector series for that part, ie, an input matching pattern I, is generated. Next, the voice identification section 313 determines the degree of similarity between the input matching pattern I and all comparison matching patterns Sn in the comparison pattern memory section 310 to obtain the degree of voice feature similarity Xn. Voice feature similarity for each pattern Xn
The integrated determination unit 3 uses the category name C given to the comparison matching pattern that gives the maximum value as the voice recognition result.
Output to 16. The integrated determination unit 316 sends the category name C as it is to an external host or the like from the output terminal 317 as the final recognition result.

Furthermore, after the recognition result is confirmed, information on whether the recognition result is correct or incorrect is received from the host. If the recognition result is incorrect at this time, it is considered that the selection of the main input system and noise input system has become inappropriate due to changes in the noise environment, so the main input system and noise input system are Redo the selection. On the other hand, if the recognition result is correct, all settings are considered appropriate, and recognition in the noise adaptive processing mode is continued. Note that the selection of the processing mode based on the recognition result is just one example, and the user may select the processing mode, for example. Another possible method is to determine whether to change the processing mode based on the frequency of misrecognitions. Figure 5 shows the third
This figure shows the operational flow of the embodiment.

[0031] The adaptive noise removal filtering process is clear, for example, in the course ``Applications of digital filters in microphone systems'' in the Journal of the Acoustical Society of Japan, Vol. 45, No. 2 (1989), and the references therein, so it will not be described here. The explanation will be omitted. However, the voice section detection unit 30
It is assumed that the adaptive operation is stopped from the time when the voice start point is detected in step 7 until the voice end point is detected (see FIG. 5), and the coefficients of the adaptive removal filter of the adaptive noise removal section 318 are fixed.

[0032]

[Effects of the Invention] As explained in detail above, according to the present invention, the input signals picked up simultaneously from a plurality of microphones are each independently discriminated, and the plurality of recognition results are used to comprehensively perform the discrimination operation. In order to perform recognition judgment, and as shown in the third embodiment, an adaptive noise removal filter is configured based on the recognition result obtained in the no-noise adaptive processing mode, and after shifting to the noise-adaptive processing mode, the noise-removed voice is Since recognition is determined based on signals, the advantage is that stable and high recognition performance can be obtained even in environments where the distance and direction between the vocal organs and the microphone change significantly, and where the background noise environment changes significantly. . For example, when using a speech recognition device in a place where the position and size of the noise source change irregularly, such as in a car, office, factory, or on the street, the adaptive operation according to the present invention can significantly improve recognition performance. can be improved.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the invention.

FIG. 3 is a block diagram showing a third embodiment of the present invention.

FIG. 4 is a block diagram of a conventional recognition device.

FIG. 5 is a diagram showing an operation flow of a third embodiment.

[Explanation of symbols]

101 First channel audio input terminal 102
2nd channel audio input terminal 103 3rd channel audio input terminal 104 1st channel audio analysis section 105 2nd channel audio analysis section 106 3rd channel audio analysis section 10
7 First channel audio section detection unit 108
2nd channel voice section detection section 109 3rd channel voice section detection section 110 1st channel comparison pattern memory section 111 2nd channel comparison pattern memory section 112 3rd channel comparison pattern memory section 113 1st channel voice Identification unit 114 Second channel audio identification unit 115 Third channel audio identification unit 1
16 Integrated determination unit 117 Output terminal

Claims (2)

[Claims] 1. A process of simultaneously collecting sounds from a plurality of microphones to obtain a plurality of input signals, and a process of independently identifying each of the plurality of input signals to obtain a plurality of identification results.
A speech recognition method comprising: a process of performing an integrated judgment on the plurality of identification results. 2. Based on the recognition results determined by the speech recognition method according to claim 1, the main input system with the best speech signal/background noise ratio in each input system, and the main input system with the best speech signal/background noise ratio in each input system. It includes a process of determining a bad noise input system and a process of configuring an adaptive noise removal filter from the main input system and the noise input system, and in the subsequent recognition process, speech is identified from the input signal after the adaptive noise removal filtering. A speech recognition method characterized by: