JP2012220607A - Sound recognition method and apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound recognition method and an apparatus that allow a sound interval of object sound under a noise environment to be detected.SOLUTION: There is provided a sound recognition method, which allows a sound interval of object sound having periodical stationarity under a noise environment to be detected and comprises: a first step for collecting an analog acoustic signal by using sound inputting means, and converting the analog acoustic signal to a digital waveform signal constituted by frames; a second step for analyzing the digital waveform signal in frame units to calculate an auto-correlation function and a secondary auto-correlation function; and a third step for determining a range in which a sum of difference absolute values of the secondary auto-correlation functions calculated for each frame exceeds a pre-set threshold value, to be the sound interval, and an apparatus of the method.

Description

  The present invention relates to a sound recognition method and apparatus that can detect a sound section of a target sound having periodic steadiness in a noisy environment.

  In recent years, it can be said that a speech recognition system is a familiar technology as an application interface in a place name input of a car navigation, a computer or a mobile phone. Speech recognition systems have been actively studied for the construction and application of systems on computers using software due to the need for computational complexity and real-time processing. In this situation, with the improvement of processing capability, speech recognition systems realized on computers are spreading to technologies that can be used in embedded systems and smartphones that have been spreading in recent years. In this situation, it is speculated that the development of leading speech recognition systems in basic research will continue to be implemented in software.

  On the other hand, a speech recognition system using an embedded system does not require a computer, and a speech recognition system using a statistical method can be realized. It is clear that further technological innovation will progress in the future, and large vocabulary continuous word recognition will also be realized on embedded systems. For this reason, it can be said that the speech recognition system has progressed from the conventional software to the middleware implementation. From this, the spread of technology to hardware speech recognition systems can be expected.

  A hardware speech recognition system is constructed by a DSP (Digital Signal Processor) or an FPGA (Field-Programmable Gate Array) to realize a small vocabulary specific speaker recognition system. This can be attributed to the fact that compilers and simulators for improving the speed and accuracy of microcomputers and mounting hardware have been prepared. Along with this, the implementation of speech recognition systems using statistical methods using hidden Markov models has also been reported. In a speech recognition system based on a hidden Markov model using continuous distribution, a system implemented with software and hardware was compared, and a speedup of 40 times or more was confirmed while maintaining the same precision recognition performance. (See Non-Patent Document 1), it has been confirmed that a hardware speech recognition system is superior in speed to a system based on software.

  The inventor is a speech recognition device capable of performing speech recognition at high speed in Patent Document 1, and extracts a feature vector representing a feature amount of speech by analyzing a speech waveform signal in units of frames. A feature vector storage unit that stores a plurality of frames of feature vectors in time series, a recognition candidate speech storage unit that stores a plurality of speech that is a speech recognition candidate, and a plurality of frames stored in the feature vector storage unit A first analysis unit that calculates the likelihood of each speech that is a speech recognition candidate based on the feature vector, and an average feature vector per frame unit is calculated from the feature vectors for a plurality of frames, and the speech recognition candidate is obtained from the average feature vector In the second analysis unit for calculating the likelihood of speech and the likelihood of each speech that is a speech recognition candidate calculated in the first analysis unit and in the second analysis unit Proposed a voice recognition device and a voice determination unit that determines the one voice based on the voice of the likelihood that the speech recognition candidates out.

  Patent Document 2 analyzes the input sound to detect periodicity, detects a voice section based on the power information of the input sound, and determines in advance based on these two detection results. There has been proposed a speech segment detection device that detects a speech segment according to a rule for determining a speech segment and a non-speech segment.

JP 2010-224020 JP JP-A-8-305388

S. J. Melnikoff, S. F. Quigley and M. J. Russell, “Speech Recognition on an FPGA Using and Continuous Hidden Markov Models”, Proceedings of 12th International Conference on Field Programmable Logic and Applications, pp.201-211, 2002.

As described above, the speed of speech recognition can be increased by the hardware speech recognition system, but it is necessary to detect a sound section in a noisy environment based on actual use.
Therefore, the present invention has an object to provide a sound recognition method and apparatus capable of detecting a sound section of a target sound under a noisy environment, and in particular, a sound recognition method and apparatus capable of voice recognition under a noisy environment. The purpose is to provide.

A first invention is a sound recognition method capable of detecting a sound section of a target sound having periodic steadiness in a noisy environment, wherein an analog sound signal is sampled by sound input means, and a digital waveform configured by a frame 1st step to convert to signal, 2nd step to analyze digital waveform signal by frame unit and calculate autocorrelation function and secondary autocorrelation function, and absolute difference value of secondary autocorrelation function calculated for each frame And a third step of determining a range in which the sum exceeds a preset threshold value as a sound section.
According to a second aspect, in the first aspect, the target sound is a voice.

A third invention comprises a sound input means for collecting an analog sound signal, a signal processing unit for detecting a sound section, and a sound recognition unit for determining the type of sound, and having a periodic steadiness in a noisy environment A sound recognition device capable of detecting a sound section of a sound, wherein the signal processing unit converts the digital waveform signal into a digital waveform signal composed of frames, analyzes the digital waveform signal in units of frames, and 2 Means for calculating a second autocorrelation function, and means for determining, as a sound section, a range in which a sum of absolute differences of secondary autocorrelation functions calculated for each frame exceeds a preset threshold value. It is a sound recognition device.
According to a fourth aspect of the present invention, in the third aspect, the vowel group is recognized based on the ratio of frequency band power in a plurality of channels for the sound section determined by the signal processing unit, and the vowel group is recognized based on the spectral distance for the recognized vowel group. It has a voice recognition function for judging

According to the present invention, it is possible to detect a sound section of a target sound having periodic steadiness in a noisy environment.
In addition, speech recognition in a noisy environment can be performed with high accuracy.

Conceptual diagram of hardware speech recognition system Voice signal of vowel / a / Autocorrelation function of speech Second-order autocorrelation function of speech Noise signal Autocorrelation function of noise Second-order autocorrelation function of noise Noise of vowel / a / in high SNR environment Voice segment detection of vowel / a / under high SNR environment Noise generation of vowel / a / in low SNR environment Voice interval detection of vowel / a / in low SNR environment The voiced noise of the uttered sentence “Turning the screw” Speech segment detection of uttered sentence "Screw bend"

The sound recognition apparatus of the present invention includes sound input means for collecting an analog sound signal, a signal processing unit for detecting a sound section, and a sound recognition unit for determining the type of sound. This sound recognition device can realize instantaneous processing and instantaneous recognition by performing speech section detection and speech recognition using a signal of about one frame or several frames.
The signal processing unit is a means for converting to a digital waveform signal composed of frames, a means for analyzing the digital waveform signal in units of frames to calculate an autocorrelation function and a secondary autocorrelation function, and 2 calculated for each frame. It has means to determine the range in which the sum of absolute differences of the next autocorrelation function exceeds a preset threshold as a sound section, and performs signal processing and speech recognition by hardware calculation at the moment when the waveform signal is framed It is possible.

A typical example of the sound input means is a microphone, but for example, an acceleration pickup that extracts a solid propagation signal such as a body conduction sound including a bone conduction sound may be used. The analog acoustic signal collected by the sound input means is AD converted by the signal processing unit and converted into a waveform signal in the PCM (pulse code modulation) format.
In the signal processing unit, it is possible to detect a sound section of a periodic stationary sound emitted from any sound source. A typical example is a voice uttered by a person, but the present invention is not limited to this. For example, it is also possible to detect a sound having periodic steadiness from devices such as an engine, a motor, and a bell.

  The present invention can be realized by software using a personal computer, but is preferably realized by a hardware speech recognition system in which a signal processing unit and a sound recognition unit are integrated from the viewpoint of processing speed. Furthermore, the hardware sound recognition system is preferably configured using an FPGA. The FPGA is an integrated circuit composed of logic elements such as AND, OR, and NAND, and flip-flops, and can constitute a user-specific logic circuit. As implementation methods, there are software development using a compiler and development using a GUI-based simulator. In the embodiment, however, it was necessary to confirm the behavior of the system in advance, so development was performed on the simulator.

  The present invention is particularly suitable for applications that do not require complex recognition algorithms such as vowels having abnormal characteristics and abnormal sound detection, and applications that require real-time performance. Further, since the signal processing unit and the sound recognition unit can be modularized as necessary, it can be used as a front-end unit for all sound recognition systems such as noise suppression, sound section detection, and feature extraction.

  Hereinafter, details of the present invention will be described with reference to examples, but the present invention is not limited to the examples.

[1] System Overview The embodiment relates to a hardware speech recognition system for processing speech. The system of the present embodiment is premised on being used in an environment in which successive conditions change, such as changes in ambient noise and speaker changes, and thus has a function of preprocessing the obtained signal. In other words, the system of this embodiment is an integrated system from front-end processing to recognition processing, and can provide a speech recognition system capable of changing an acoustic model and a template associated with speaker change, and interactive. Operation as a system is also possible.

  FIG. 1 shows a conceptual diagram of a hardware speech recognition system proposed in this embodiment. The voice recognition system according to the present embodiment includes a voice recognition device 1 having a signal processing unit 2 and a voice recognition unit 3, voice input means 4 including a microphone, and an output device 5. The signal processing unit 2 uses the waveform signal collected by the voice input unit 4 to perform preprocessing in voice recognition such as speaker recognition, voice segment detection, and noise suppression. The speech recognition unit 3 is a part that performs efficient speech recognition using the result obtained from the signal processing unit 2, and realizes processing in units of frames and can perform instantaneous processing (real-time processing). The determined audio information is output to the output device 5.

  In this example, the signal processing unit 2 and the speech recognition unit 3 were all developed using LabVIEW (registered trademark). LabVIEW (registered trademark) is a graphical programming environment mainly used for measurement control provided by National Instruments, and can perform intuitive programming such as flowcharts with icons and wires. It can also be integrated with many hardware devices and has a built-in library that allows advanced analysis and data visualization, allowing you to conduct necessary studies in system development such as evaluation, testing, and testing. Each study in this example is a study result by a system implemented on a simulator.

  Hereinafter, a robust speech segment detection algorithm in a noisy environment provided in the speech recognition apparatus 1 of the present embodiment will be described based on an experiment example of vowel recognition. The signal processing unit 2 of the speech recognition device 1 performs speech section detection in a noisy environment using the sum of absolute differences of the second-order autocorrelation function, and the speech recognition unit 3 compares the distance between the output of the frequency band power and the formant frequency. We confirmed the effectiveness of the two-stage speech recognition using.

[2] Signal processor 2
The signal processing unit 2 performs preprocessing for realizing robust speech recognition such as speech segment detection and noise suppression. In the following, pre-emphasis using a difference between signals and robust speech section detection under a noise environment will be described.

[2-1] Pre-emphasis Voice signals collected from microphones are framed by window functions. Here, there is a problem that it becomes difficult to detect a peak because the signal level attenuates as the formant frequency becomes higher, and pre-emphasis is performed to solve this problem. Here, pre-emphasis may not be effective for clarification or the like. However, since the speech recognition unit 3 of this embodiment is used for spectrum peak detection, the signal processing unit 2 performs pre-emphasis as preprocessing. In the present embodiment, in order to simplify the system, pre-emphasis using a time waveform difference is performed.

  In general, pre-emphasis of audio processing is performed using a coefficient of about 0.97, and this parameter is often used even though the audio sampling frequency for the system is variable. Here, if the sampling frequency changes, the differential interval of the discrete signal changes and a different frequency characteristic can be obtained. If there is no need to obtain a scaling coefficient strictly, it is sufficient to use a simple difference. it is conceivable that. Therefore, in this embodiment, the difference calculation is adopted from the viewpoint of speeding up the system.

[2-2] Voice interval detection using second-order autocorrelation function Since the ambient noise environment changes constantly, a system that does not depend on the environment, such as a quiet environment or a noise environment, is required. Therefore, in the present embodiment, speech segment detection is performed using a secondary autocorrelation function. Equation 1 for obtaining the autocorrelation function is shown below.

The autocorrelation function R (j) can be obtained by using Equation 1 for the waveform signal x (i).
Fig. 2 shows the speech signal when a male 20-year-old utters a vowel / a /, Fig. 3 shows the autocorrelation function of speech, and Fig. 4 shows the secondary autocorrelation function of speech. FIG. 5 shows a noise signal, FIG. 6 shows a noise autocorrelation function, and FIG. 7 shows a noise secondary autocorrelation function.
In this embodiment, the autocorrelation function is calculated again with respect to the autocorrelation function obtained from the speech signal, and the secondary autocorrelation function R ′ (k) is obtained. In this way, it is possible to obtain a signal that emphasizes continuity that cannot be expressed by an autocorrelation function. Then, the sum of absolute differences of the secondary correlation function is used as the utterance estimation value A (l). Equation 2 for obtaining the estimated speech value is shown below.

  In a quiet environment, the amplitude value of the autocorrelation function during no speech cannot be obtained, and in a noisy environment, the amplitude value of the autocorrelation function decreases due to non-stationary noise or the like. On the other hand, vowels are highly stationary, and the amplitude value of the autocorrelation function increases accordingly. Although the consonant has a lower autocorrelation function value than the vowel, it can be estimated that the consonant is higher than the uncorrelated signal such as white noise. Therefore, robust voice segment detection can be easily realized regardless of whether the environment is quiet or noisy.

First, the effectiveness is confirmed when a 20-year-old man utters a vowel / a / in a noisy environment.
The microphone used for signal detection is ECM-31HVC (600Ω) manufactured by SONY. The white noise generated by free software (WaveGene) was reproduced by a general microphone and used as background noise during utterance.
FIG. 8 shows the noisy speech uttered in a high SNR environment, and FIG. 9 shows the result of speech segment detection performed on FIG. From FIG. 9, it can be confirmed that the voice section can be detected in the vicinity of 0.9 to 1.8 seconds.
10 is a noisy voice uttered by the same male as in FIG. 8 under a condition that is more difficult than FIG. 8, that is, in a low SNR environment, and FIG. 11 is the result of performing voice segment detection on FIG. . From FIG. 11, it can be confirmed that the values in the vicinity of 0.5 to 1.6 seconds are relatively high.
In the speech section detection, a relative comparison between the noise section and the utterance section is performed. In FIGS. 9 and 11, it can be determined that the speech section can be detected because the utterance estimation value is higher than the noise section. The effectiveness of other Japanese vowels (/ i /, / u /, / e / and / o /) was confirmed, and similar results were obtained.

Next, the validity is confirmed when a phrase utterance is made, not a vowel produced by a single sound.
FIG. 12 shows a noisy voice when a 20-year-old male voice utters the phrase “twisted” in a noisy environment, and FIG. 13 shows the result of voice segment detection performed on FIG. From FIG. 13, it can be confirmed that an estimated speech value can be obtained although the value of the autocorrelation function is high in the vowel section and the value of the autocorrelation function is low compared to the vowel in the consonant section. This shows that the periodicity of the consonant section is relatively high compared to noise, and the speech section can be detected even in the consonant section. Irregular noise such as white noise can be detected robustly, but with steady noise, the estimated speech value becomes high, and performance degradation is expected. On the other hand, it was confirmed that detection of various desired signals such as detection of abnormal sounds and environmental sounds as well as voices can be expected.

[3] Voice recognition unit 3
In the present embodiment, as a technique for taking advantage of the hardware voice recognition system, a technique for performing successive voice recognition for each input frame is employed. By using this method, voice recognition can be performed in units of frames, and filter determination for sound quality conversion and abnormal sounds can be estimated for each frame. In this embodiment, a recognition system using two-stage vowel recognition was constructed.

  In the first pass of the speech recognition unit 3, vowel groups were recognized, and recognition was performed to narrow down the recognition target. Here, the band where the formant frequency exists was divided into three channels, and parameters were set for each band power obtained from it. By normalizing with the maximum value PLocalMax (i) and the minimum value PLocalMin (i) in each channel using a known formula, and expressing each output P (i) of channel i in frequency band power as a ratio Ratio (i) It was confirmed that there was a difference in the band of formant frequencies that existed.

  In the second pass of the speech recognition unit 3, vowels in the group were identified. Templates of formant frequencies investigated in advance were created, and the distance between the formant frequencies obtained from the template and the audio signal was compared. Here, since there is a difference in scale between the distances of the first and second formants, there is a problem that it is difficult to perform discrimination by simple distance comparison. Therefore, we decided to solve the scaling problem using a known formula and determine the final candidate.

The first pass identifies vowels / a /, / u /, / o / groups and vowels / i /, / e / groups using frequency band power, and each group uses spectral distance measures in the second pass. Identification within. The template for recognition was preliminarily estimated from three 20-year-old men, and a recognition experiment was conducted with four male 20-year-olds, one added to the three speakers. Each speaker uttered 40 sets of each Japanese vowel in a quiet environment in the laboratory. At this time, the sampling frequency was set to 44.1 kHz in consideration of environmental sound recognition.
Table 1 shows the error matrix and vowel recognition rate of recognition results for all speakers. As can be confirmed from each recognition result, it was confirmed that a recognition rate of about 75% was obtained for isolated vowels.

1 Voice recognition device
2 Signal processor
3 Voice recognition unit
4 Voice input means (microphone)
5 Output device

Claims (4)

A sound recognition method capable of detecting a sound section of a target sound having periodic steadiness in a noisy environment,
A first step of collecting an analog sound signal by sound input means and converting it into a digital waveform signal constituted by a frame; a first step of calculating an autocorrelation function and a secondary autocorrelation function by analyzing the digital waveform signal in units of frames; A sound recognition method comprising: two steps; and a third step of determining, as a sound section, a range in which a sum of absolute differences of secondary autocorrelation functions calculated for each frame exceeds a preset threshold value.   2. The sound segment recognition method according to claim 1, wherein the target sound is a voice. Equipped with a sound input means that collects analog sound signals, a signal processing unit that detects sound intervals, and a sound recognition unit that determines the type of sound, and detects the sound interval of the target sound that has periodic steadiness in a noisy environment A sound recognition device that enables
The signal processing unit converts to a digital waveform signal composed of frames, means for analyzing the digital waveform signal in units of frames to calculate an autocorrelation function and a secondary autocorrelation function, and 2 calculated for each frame And a means for determining, as a sound section, a range in which a sum of absolute differences of the next autocorrelation function exceeds a preset threshold value.   It has a voice recognition function for recognizing a vowel group based on a frequency band power ratio in a plurality of channels for a sound section determined by the signal processing unit and determining a vowel based on a spectral distance for the recognized vowel group. 5. The sound recognition apparatus according to claim 4.