JP6594278B2 - Acoustic model learning device, speech recognition device, method and program thereof - Google Patents

Description

  The present invention relates to a technique for learning an acoustic model, a technique for recognizing speech, and a technique for processing noise.

  In voice recording for learning an acoustic model or voice recognition, an intelligent microphone (see, for example, Non-Patent Document 1) is used to focus the voice focused on the speaker and the background noise. Collecting audio and is done.

  At that time, in order to prevent the deterioration of the accuracy of acoustic model learning and speech recognition due to the influence of noise, noise suppression is performed on the signal obtained by recording, and the acoustic model learning is performed using the signal from which the noise has been removed. Was sometimes done. However, in this method, since the learning of the acoustic model is performed only with speech without noise, there is a possibility that the performance of speech recognition may be reduced when noise cannot be removed in the previous stage in an actual environment.

  By the way, there are stationary noises with small time variations and non-stationary noises with different noise characteristics depending on the observation time. In particular, when non-stationary noise is included, the performance of the speech recognition system may deteriorate.

  When using speech recognition installed in a smartphone or the like, a non-stationary noise environment is the most common in the real environment, and a system that operates robustly in this non-stationary noise environment has been demanded.

NTT Advanced Technology Co., Ltd., “Sound Collection Software Intelligent Microphone Library for High Noise”, [online], [Searched on September 12, 2016], Internet <URL: http://www.ntt-at.co.jp / product / i-mic />

This invention is robust to non-stationary noise than conventional acoustic model learning device, a voice recognition device, and to provide a these methods and programs.

  An acoustic model learning device according to an aspect of the present invention generates a sound signal by collecting a target sound and generates a noise signal by collecting background noise, and a sound based on the sound signal. A feature extraction unit that generates a feature quantity and generates a noise feature quantity based on a noise signal, and whether the noise feature quantity corresponds to stationary noise or non-stationary noise in the past. Judgment is made using a noise feature amount corresponding to noise or non-stationary noise, and when it is determined that the noise feature amount corresponds to non-stationary noise, the noise feature amount corresponds to the stationary noise. A noise information processing apparatus that performs a process close to, and an acoustic model learning unit that generates an acoustic model by learning an acoustic model using the processed noise feature and speech feature.

  A speech recognition apparatus according to an aspect of the present invention includes a sound collection unit that generates a sound signal by collecting a target sound and generates a noise signal by collecting background noise, and a sound feature based on the sound signal. A feature amount extraction unit that generates a noise feature amount based on a noise signal and whether the noise feature amount corresponds to stationary noise or non-stationary noise in the past. Alternatively, when it is determined using a noise feature amount corresponding to non-stationary noise and the noise feature amount is determined to correspond to non-stationary noise, the noise feature amount is changed to a noise feature amount corresponding to stationary noise. A noise information processing apparatus that performs a process of approaching, a speech recognition unit that performs speech recognition using the processed noise feature quantity and speech feature quantity, and the acoustic model generated by the acoustic model learning apparatus of claim 1; I have.

Robust to non-stationary noise than conventional acoustic model learning device, a speech recognition device, can provide these methods and programs.

The block diagram for demonstrating the example of an acoustic model learning apparatus. The flowchart for demonstrating the example of the acoustic model learning method. The block diagram for demonstrating the example of a speech recognition acoustic model learning apparatus. The flowchart for demonstrating the example of the acoustic model learning method.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[Acoustic model learning apparatus and method]
A block diagram of the acoustic model learning apparatus is shown in FIG.

  As shown in FIG. 1, the acoustic model learning device includes, for example, a sound collection unit 101, a feature amount extraction unit 102, a noise information processing device 103, an acoustic model learning unit 104, and an acoustic model storage unit 106. In the acoustic model learning method, each unit of the acoustic model learning apparatus is realized, for example, by the processing from step S1 to step S4 described below with reference to FIG.

  Hereinafter, processing of each unit of the acoustic model learning device will be described.

<Sound Collection Unit 101>
The sound collection unit 101 collects the target sound and background noise. The sound collection unit 101 generates an audio signal by collecting the target sound, and generates a noise signal by collecting background noise (step S1).

  The sound collection unit 101 is, for example, two intelligent microphones. Hereinafter, “intelligent microphone” is also referred to as “intelligent microphone”. For details of the intelligent microphone, see Non-Patent Document 1, for example.

  One of the intelligent microphones is focused on the speaker to pick up the target sound, and the other intelligent microphone is focused on the ambient noise to pick up surrounding background noise that is not the target sound. Focusing on a desired location can be realized by changing an internal parameter of the intelligent microphone.

  As for the position of the focus on the target sound and background noise, if the target sound and noise do not move, the focus is fixed assuming that the position is known, and if the target sound moves, the position of the sound source is determined by the function of the intelligent microphone itself. Focus after estimating. Thereby, for example, a 3ch audio signal and a 3ch noise signal are obtained.

  The obtained signal is output to the feature amount extraction unit 102.

<Feature Extraction Unit 102>
The feature amount extraction unit 102 generates a speech feature amount based on the speech signal, and generates a noise feature amount based on the noise signal (step S2).

  For example, the feature quantity extraction unit 102 first performs preprocessing in the intelligent microphone for each of the audio signal and the noise signal. The pre-processing in the intelligent microphone is to extract only the focus signal from the 3ch signal and output the 1ch processed sound.

  Thereafter, the feature quantity extraction unit 102 performs feature quantity extraction on each of the signals after preprocessing in the intelligent microphone, and obtains a filter bank feature quantity.

  After that, the feature quantity extraction unit 102 takes the logarithm of the filter bank feature quantity and adds a feature quantity related to temporal change. An existing technique can be used for the calculation of the filter bank feature value and the subsequent processing. As described above, the feature amount extraction unit 102 may use the logarithmic value of the filter bank feature amount as the speech feature amount and the noise feature amount.

  The generated voice feature amount is output to the acoustic model learning unit 104. The generated noise feature amount is output to the noise information processing apparatus 103. Since the voice feature quantity and the noise feature quantity can also be expressed in a vector format, they are also expressed as a voice feature quantity vector and a noise feature quantity vector, respectively.

<Noise Information Processing Device 103>
The noise information processing apparatus 103 determines whether the noise feature amount obtained from the feature amount extraction unit 102 is stationary noise or non-stationary noise, and performs processing suitable for each noise. Specifically, the noise information processing apparatus 103 determines whether the noise feature amount corresponds to stationary noise or non-stationary noise, and the noise feature amount corresponds to past stationary noise or non-stationary noise. When it is determined that the noise feature value corresponds to the non-stationary noise, a process of bringing the noise feature value close to the noise feature value corresponding to the stationary noise is performed (step S3).

  Noise feature quantities can be divided into two types: stationary noise that does not change much with time, such as the sound of an air conditioner, and non-stationary noise, where the nature of the noise changes with time. Since these noises have completely different characteristics, it is preferable to take different measures.

  The determination of whether the noise is stationary or non-stationary can be made from the similarity between the average noise vector held by the system and the input noise feature vector.

  The average noise vector is, for example, a vector obtained from the average value of stationary noise up to the previous time. There are several methods for obtaining the initial value of the average noise vector, such as calculating from the stationary noise observed in advance or giving the initial value from the non-speech interval included in the beginning of the input speech. There is a way.

An existing technique can be used as a method for determining the similarity between vectors. For example, when using the cosine similarity, two vectors by taking the inner product of the mean noise vector n_ave the input vector N_in _t is determined an angle as shown in the following equation (1), by using it as a degree of similarity Can be determined.

  For example, the cosine similarity defined by the above equation (1) shows a value closer to 1 as the two vectors are closer.

  For example, if the cosine similarity defined by the above equation (1) is larger than a threshold value, it is determined as stationary noise, and when it is below the threshold value, it is determined as non-stationary noise. The threshold value can take any value between -1 and 1.

  As a result of obtaining the similarity, when it is determined that the noise is stationary noise, the average of the noise feature amount and the average noise vector is taken and used as a new average noise vector at time t.

If it is determined as non-stationary noise, it will adversely affect the recognition as it is, so that processing is performed to bring the noise feature quantity closer to stationary noise. Various methods can also be used for conversion to stationary noise. For example, in the case of calculating cosine similarity, the angle formed by two vectors has already been obtained, so that the noise feature vector is calculated. By applying a cosine of the angle and applying rotation, it becomes possible to bring the property closer to the average noise vector. For the rotation of the multidimensional vector, for example, the technique of Reference 1 can be used.
[Reference 1] A. Antonio; PA Ricardo, “General n-Dimensional Rotations,” WSCG 2004, pp. 1-8, 2004

  If it is determined that the noise feature value corresponds to non-stationary noise, the noise feature value that has been processed to approach the noise feature value corresponding to the stationary noise is output to the acoustic model learning unit 104. . When it is determined that the noise feature value corresponds to stationary noise, the noise wiretapping amount is output to the acoustic model learning unit 104.

<Acoustic model learning unit 104>
The acoustic model learning unit 104 generates an acoustic model by learning the acoustic model using the voice feature amount output from the feature amount extraction unit 102 and the noise feature amount output from the noise information processing apparatus 103 ( Step S4).

As the acoustic model learning method, an existing acoustic model learning method such as a learning method for the DNN acoustic model can be used. For example, the method described in Reference 2 can be used.
[Reference 2] Hinton, et al., “A fast learning algorithm for deep belief nets.”, Neural Computation, Vol18, No.7, pp.1527-1554, 2006
The generated acoustic model is stored in the acoustic model storage unit 106.

[Voice recognition apparatus and method]
A block diagram of the sound recognition apparatus is shown in FIG.

  As illustrated in FIG. 1, the acoustic recognition device includes, for example, a sound collection unit 101, a feature amount extraction unit 102, a noise information processing device 103, a speech recognition unit 105, and an acoustic model storage unit 106. The voice recognition method is realized by, for example, each unit of the voice recognition apparatus by the processing from step S1 to step S5 described below with reference to FIG.

  Hereinafter, processing of each unit of the speech recognition apparatus will be described. Note that the description will focus on the speech recognition unit 105 that is different from the acoustic model learning device. A duplicate description of the same parts as those of the acoustic model learning apparatus is omitted.

<Sound Collection Unit 101>
The sound collection unit 101 is the same as the sound collection unit 101 of the acoustic model learning device.

<Feature Extraction Unit 102>
The feature amount extraction unit 102 generates a speech feature amount based on the speech signal and generates a noise feature amount based on the noise signal in the same manner as the feature amount extraction unit 102 of the acoustic model learning device (step S2).

  The generated voice feature amount is output to the acoustic model learning unit 104.

<Noise Information Processing Device 103>
The noise information processing apparatus 103 is similar to the noise information processing apparatus 103 of the acoustic model learning apparatus. The noise information processing apparatus 103 corresponds to stationary noise or non-stationary noise. If the noise feature quantity is determined to correspond to the non-stationary noise, the noise feature quantity is determined to be the stationary noise. A process of approaching the noise feature amount corresponding to is performed (step S3).

<Acoustic model storage unit 106>
The acoustic model storage unit 106 stores an acoustic model generated by the acoustic model learning device.

<Voice recognition unit 105>
The speech recognition unit 105 performs speech recognition using the speech feature amount output from the feature amount extraction unit 102, the noise feature amount output from the noise information processing apparatus 103, and the acoustic model read from the acoustic model storage unit 106. (Step S5). At that time, an existing technique can be used for voice recognition.

[Program and recording medium]
When each process in the acoustic model learning device, the speech recognition device, or the noise information processing device is realized by a computer, the processing contents of the functions that these devices should have are described by a program. Then, by executing this program on the computer, the processing of each device is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

[Modification]
The speech recognition apparatus may further include an acoustic model learning unit 104 as indicated by a broken line in FIG. In this case, the acoustic model learning unit 104 generates an acoustic model by the same processing as the acoustic model learning unit 104 of the acoustic model learning device, and outputs the generated acoustic model to the speech recognition unit 105. Then, the speech recognition unit 105 performs speech recognition processing using the acoustic model generated by the acoustic model learning unit 104 instead of the acoustic model read from the acoustic model storage unit 106.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

Claims (6)

A sound collection unit that generates a sound signal by collecting a target sound and generates a noise signal by collecting background noise;
A feature amount extraction unit that generates a speech feature amount based on the speech signal and generates a noise feature amount based on the noise signal;
It is determined whether the noise feature quantity corresponds to stationary noise or non-stationary noise using a noise feature quantity corresponding to past stationary noise or non-stationary noise, and the noise feature quantity is A noise information processing apparatus that performs a process of bringing the noise feature amount close to a noise feature amount corresponding to stationary noise when it is determined to correspond to non-stationary noise;
An acoustic model learning unit that generates an acoustic model by learning an acoustic model using the processed noise feature and the voice feature;
An acoustic model learning device. The acoustic model learning device according to claim 1,
The voice feature amount and the noise feature amount are logarithmic values of the filter bank feature amount.
Acoustic model learning device. A sound collection unit that generates a sound signal by collecting a target sound and generates a noise signal by collecting background noise;
A feature amount extraction unit that generates a speech feature amount based on the speech signal and generates a noise feature amount based on the noise signal;
Whether the noise feature value corresponds to stationary noise or non-stationary noise is determined using a noise feature value corresponding to past stationary noise or non-stationary noise, and the noise feature value is A noise information processing apparatus that performs a process of bringing the noise feature amount close to a noise feature amount corresponding to stationary noise when it is determined to correspond to non-stationary noise;
A speech recognition unit that performs speech recognition using the noise feature value after processing and the speech feature value, and the acoustic model generated by the acoustic model learning device according to claim 1;
A speech recognition device. The acoustic recognition device according to claim 3,
The voice feature amount and the noise feature amount are logarithmic values of the filter bank feature amount.
Voice recognition device. A sound collection unit that generates a sound signal by collecting a target sound, and generates a noise signal by collecting background noise;
A feature amount extraction unit that generates a speech feature amount based on the speech signal and generates a noise feature amount based on the noise signal; and
The noise information processing apparatus determines whether the noise feature amount corresponds to stationary noise or non-stationary noise using a noise feature amount corresponding to past stationary noise or non-stationary noise. When it is determined that the noise feature amount corresponds to non-stationary noise, a noise information processing step for performing processing to bring the noise feature amount close to a noise feature amount corresponding to stationary noise;
An acoustic model learning unit that generates an acoustic model by learning an acoustic model using the processed noise feature and the voice feature;
Acoustic model learning method including According acoustic model learning device claim 1 or 2, a program for causing a computer to function as any of the various parts of the speech recognition equipment according to claim 3 or 4.

Images