JP6348427B2 - Noise removal apparatus and noise removal program - Google Patents

Description

  The present invention relates to a noise removal apparatus and a noise removal program.

  Conventionally, there is a technique for removing noise from a voice signal corresponding to a voice collected by a sound collecting device such as a microphone. As a technique for removing noise, for example, there is DAE (Denoising AutoEncoder). DAE is a kind of DNN (Deep Neural Network).

FIG. 7A is a diagram for explaining the outline of the conventional DAE learning process. As shown in FIG. 7A, in the DAE of the prior art, in the DAE learning unit, the observation signal for learning obtained under noise by one sound collector (one channel) is subjected to frequency analysis, and learning in the entire frequency domain is performed. A vector y indicating the logarithmic spectrum of the amplitude of the observation signal for use is defined as an input vector z _in . In the prior art DAE, the output h ^l (z _in ) _in each of the L hidden layers h ^l (l = 1,..., L) is a parameter θ = {W, It is expressed as h ^l (z _in ) = σ _θ (W ^l h ^(l−1) (z _in ) + b ^l ) by a predetermined nonlinear function σ _θ having b}. The output in the hidden layer h ¹ is h ¹ (z _in ) = σ _θ (W ¹ (z _in ) + b ¹ ).

In the DAE of the prior art, the output vector z _out obtained by sequentially removing noise from l = 1,..., L from the input vector z _in by h ^l (z _in ) matches the learning original speech, that is, z. _The parameter θ is learned so that _out = f _θ (z _in ) = W ^L h ^(L−1) (z _in ) + b ^L.

FIG. 7B is a diagram for explaining the outline of the conventional noise removal processing. In the DAE of the prior art, the DAE decoder performs frequency analysis on the observation signal obtained by one sound collector (one channel), and an input vector y ′ indicating a logarithmic spectrum of the amplitude of the observation signal in the entire frequency domain is input. Let z _in ′. Then, the DAE of the prior art copies the learned parameter θ as shown in FIG. 7A from the DAE learning unit to the DAE decoding unit, and uses the learned parameter θ to each of the L hidden layers h ^l (l = 1). ,..., L), an output vector z _out ′ obtained by removing noise from the input vector z _in ′ is obtained.

P. Vincent, H. Larochelle, Y. Bengio, and P. A. Manzagol, “Extracting and composing robust features with denoising autoencoders.”, In Proc. Of ICML2008, 2008, pp.1096-1103. Y. Liu, P. Zhang, and T. Hain, “Using neural network front-ends on far field multiple microphones based speech recognitions.”, Proc. Of ICASSP2014, 2014, pp.5579-5583. S. Renals and P. Swietojanski, “Neural networks for distant speech recognition.”, In Proc. Of HSCMA2014, 2014. T. Nakatani, S. Araki, T. Yoshioka, M. Delcroix, and M. Fujimoto, “Dominance based integration of spatial and spectral features for speech enhancement.”, IEEE Trans. Audio, Speech and Language Processing, vol. 21, no. 12, 2013, pp.2516-2531. M. Fujimoto, S. Watanabe, and T. Nakatani, “A robust estimation method of noise mixture model for noise suppression.”, Proc. Of Interspeech2011, 2011, pp.697-700. M. Delcroix, Y. Kubo, T. Nakatani, and A. Nakamura, “Is speech enhancement pre-processing still relevant when using deep neural networks for acoustic modeling?”, Proc. Of Interspeech2013, 2013, pp.2992-2996.

  However, the above prior art performs learning using the observation signal for learning obtained by one sound collector (one channel) as an input of DAE in the noise removal processing, and is obtained by one sound collector. It only removes noise from the observed signal. Even when observation signals from a plurality of sound collectors (multiple channels) can be used, a channel connection in which the signals of each channel are simply connected is used, or an enhanced audio signal by the beam forming technique is DAE. The noise removal performance is low.

  An example of an embodiment disclosed in the present application has been made in view of the above, and aims to improve noise removal performance.

  An example of a noise removal apparatus disclosed in the present application includes a frequency analysis unit, a logarithmic calculation unit, a multichannel feature amount calculation unit, a DAE (Denoising AutoEncoder) learning unit, and a DAE decoding unit. The frequency analysis unit performs frequency analysis on the first sound signal observed by the first sound collector and the second sound signal observed by the second sound collector disposed at a position different from the first sound collector. The logarithm calculation unit calculates a logarithmic spectrum that is a logarithm of the amplitude of the first audio signal subjected to frequency analysis by the frequency analysis unit. The multi-channel feature value calculation unit calculates a multi-channel feature value related to the first sound signal and the second sound signal from the first sound signal and the second sound signal subjected to frequency analysis by the frequency analysis unit. The DAE learning unit includes a logarithmic spectrum of the first speech signal for learning calculated by the logarithm calculation unit, the first speech signal for learning and the second speech signal for learning calculated by the multichannel feature amount calculation unit. The learning multi-channel feature quantity is used as an input, and a DAE parameter is learned using the logarithmic spectrum of the original voice for learning as an output. The DAE decoding unit calculates the logarithm spectrum of the first speech signal to be denoised calculated by the logarithm calculation unit, the first speech signal to be denoised by the multichannel feature amount calculation unit, and the first speech signal to be denoised. The logarithm of the noise-removed speech obtained by removing the noise component from the logarithmic spectrum of the first speech signal to be denoised using the parameters learned by the DAE learning unit with the multichannel feature amount for noise removal regarding the two speech signals as input. Output the spectrum.

  According to an exemplary embodiment disclosed in the present application, for example, it is possible to improve noise removal performance.

FIG. 1 is a diagram illustrating an example of a configuration of a noise removal system. FIG. 2 is a diagram illustrating an example of the configuration of the signal reconstruction device. FIG. 3 is a flowchart illustrating an example of the DAE learning process. FIG. 4 is a flowchart illustrating an example of the noise removal process. FIG. 5A is a diagram illustrating an example of the effect of the embodiment. FIG. 5B is a diagram illustrating an example of the effect of the embodiment. FIG. 5C is a diagram illustrating an example of the effect of the embodiment. FIG. 6 is a diagram illustrating an example of a computer in which a noise removal apparatus and a signal reconstruction apparatus are realized by executing a program. FIG. 7A is a diagram for explaining the outline of the conventional DAE learning process. FIG. 7B is a diagram for explaining the outline of the conventional noise removal processing.

[Embodiment]
Hereinafter, embodiments of a noise removal device and a noise removal program disclosed in the present application will be described. The following embodiments are merely examples, and do not limit the technology disclosed by the present application. Moreover, you may combine suitably embodiment shown below and other embodiment in the range with no contradiction.

  In the following embodiments, when A is a vector or a scalar, “^ A” is equivalent to the notation defined by the following equation (1), and “￣A” is 2) It is assumed that it is equivalent to the notation defined by the formula.

In the following embodiments, the sound collection device is a microphone (hereinafter referred to as a microphone). A logarithmic spectrum of an observation signal observed by an m-th (1 ≦ m ≦ M) microphone among M microphones (M is a natural number) arranged at different positions is denoted as Y _{t, f} ^(m) . Here, t is an index of time, and f is an index of mel scale frequency. A logarithmic spectrum of an observation signal observed with M microphones is _expressed as a vector Y _{t, f} = [Y _{t, f} ⁽¹⁾ ,..., Y _{t, f} ^(M) ]. Here, the vector Y _{t, f} indicating the observation signal from the microphone is defined as X _{t, f} ^(m) as the original sound collected by the m-th microphone and N _{t, f} ^(m) as the noise. A vector indicating speech is a vector X _{t, f} = [X _{t, f} ⁽¹⁾ ,..., X _{t, f} ^(M) ], and a vector indicating noise is a vector N _{t, f} = [N _{t, f} ^{( 1)} ,..., N _{t, f} ^(M) ] are given as follows (3).

The embodiment obtains each estimated value ^ X _{t, f} ^(m) of the logarithmic spectrum of the noise-removed speech from which noise is removed from the vector Y _{t, f} indicating the observed signal, and further re-synthesizes the waveform of the speech. To obtain noise-removed speech. The embodiment describes the case where the number of microphones M = 2, but the case where M ≧ 3 can be similarly discussed.

Further, in the embodiment, for the sake of simplicity, a vector y _t = [y _{t, 1} ,..., Y is a vector in which logarithmic spectra of all the frequencies of the observation signal of the first microphone are collected. _{t, f 1} ,..., y _{t, F} ] (where y _{t, f} ⁽¹⁾ ). Further, embodiments, the logarithmic spectrum at the first original audio of all frequencies in the microphone is expressed as x _t, denoted the log spectrum of the entire frequency noise rejection speech ^ x _t and.

(Configuration of noise reduction system)
FIG. 1 is a diagram illustrating an example of a configuration of a noise removal system. The noise removal system 100 includes microphones 1 and 2, a noise removal device 10, and a signal reconstruction device 20. Assume that the microphone 1 is the first microphone and the microphone 2 is the second microphone.

  The noise removal apparatus 10 includes a frequency analysis unit 11, a logarithmic calculation unit 12, a multichannel feature calculation unit 13, a DAE (Denoising AutoEncoder) learning unit 14, and a DAE decoding unit 15.

The frequency analysis unit 11 performs frequency analysis, for example, Mel frequency analysis, on the observation signals observed by the two microphones 1 and 2, respectively, and converts them into frequency domain signals. The observation signals include an observation signal for learning by the DAE learning unit 14 described later and an observation signal to be removed from noise using the learning result. The logarithm calculation unit 12 takes the logarithm of the amplitude of the observation signal converted into the frequency domain signal, and obtains a logarithmic spectrum vector Y _{t, f} = [Y _{t, f} ⁽¹⁾ , Y _{t, f} ⁽²⁾ ]. obtain. Here, for the observation signal Y _{t, f} ⁽¹⁾ at the first microphone, y _{t, f} = Y _{t, f} ⁽¹⁾ . Then, the logarithm calculation unit 12 is a vector y _t = [y _{t, 1} ,..., Y _{t, f} ,..., Y that summarizes the logarithmic spectra at all frequencies of the observation signal of the first microphone. _{t, F} ] is input to the DAE learning unit 14 when it is an observation signal for learning, and is input to the DAE decoding unit 15 when it is an observation signal to be denoised.

The multi-channel feature calculation unit 13 performs frequency analysis on the observation signals of the microphones 1 and 2, and uses a result of the frequency analysis to calculate a vector r _t = [r _{t, 1 ,. , F} ]. Multi-channel feature calculation block 13, the calculated vector _{r t,} and inputs to the DAE learning unit 14 and the DAE decoding unit 15. Details of the multi-channel feature calculation unit 13 will be described later.

The DAE learning unit 14 is a DNN (Deep Neural Network), and can obtain an original voice as an output vector z _{out with} respect to an input vector z _in which is an observation voice for learning observed under noise. Learn l hidden layers (l is a natural number). In the embodiment, the case of l = 2 is illustrated. The DAE learning unit 14 includes an input unit 14a, a hidden layer 14b, a hidden layer 14c, and an output unit 14d. That is, the two hidden layers of the hidden layer 14b and the hidden layer 14c are included. Here, the number of hidden layers l = 2 is merely an example.

The input unit 14a receives the logarithmic spectrum vector y _t = [y _{t, 1} ,..., Y _{t, f} ,. , Y _{t, F} ] are connected using a context window consisting of T frames before and after the time t.

Further, the input unit 14a connects the vector r _t input from the multi-channel feature calculation unit 13 using a context window composed of previous and subsequent T frames centered on the time t, and inputs as shown in the following equation (4). Find the vector z _in .

The output unit 14d concatenates the original speech for learning using a context window composed of previous and subsequent T frames centered at time t, and obtains an output vector z _out as shown in the following equation (5).

Then, the hidden layer 14b and the hidden layer 14c are formed as the first hidden layer (l = 1, 2) by learning DAE using the input of the DAE learning unit 14 as the input vector z _in and the output as the output vector z _out. can get. That is, the DAE learning unit 14 sets the parameter θ = {W, b} (W is a weight vector, b so that the output of the hidden layer 14b and the hidden layer 14c can be obtained by the following equation (6). Learns the bias vector. Note that σ _θ (·) in the following equation (6) is a nonlinear function, for example, a sigmoid function. Further, in the following equation (6), the output of the first layer (ie, the hidden layer 14b) is calculated by h ¹ (z _in ) = σ _θ (W ¹ z _in + b ¹ ), and the final layer (ie, the second layer) The output of the hidden layer 14c) is calculated by a vector z _out = f _θ = W ^L h ^(L-1) + b ^L by a linear function. Learning of the parameter θ = {W, b} is obtained by an existing method, for example, Stochastic Gradient Descent Method. The DAE learning unit 14 copies the learned parameter θ = {W, b} to the DAE decoding unit 15.

  The DAE decoding unit 15 includes an input unit 15a, a hidden layer 15b, a hidden layer 15c, and an output unit 15d. The input unit 15a corresponds to the input unit 14a of the DAE learning unit 14, the hidden layer 15b corresponds to the hidden layer 14b of the DAE learning unit 14, the hidden layer 15c corresponds to the hidden layer 14c of the DAE learning unit 14, and the output unit 15 d corresponds to the output unit 14 d of the DAE learning unit 14.

The input unit 15a receives the logarithmic spectrum vector y _t = [y _{t, 1} ,..., Y _{t, f} of the observed speech to be removed from the noise observed from the microphones 1 and 2 and input from the logarithmic calculation unit 12. ,..., Y _{t, F} ] and the vector r _t = [r _{t, 1} ,..., R _{t, F} ] input from the multi-channel feature calculator 13 with the time t as the center. A context window made up of preceding and following T frames is used for connection, and a vector z _in is obtained as in the above equation (4). The input unit 15a inputs the obtained vector z _in to the hidden layer 15b.

By using the parameter θ = {W, b} learned by the DAE learning unit 14 by the hidden layer 15b and the hidden layer 15c, the output vector z _out = [x _{t−T 1} ,..., x _t ,..., x _{t + T} ]. The parameter θ = {W, b} in the following equation (7) is learned by the following equation (6). Further, in the following expression (7), the output of the first layer (that is, the hidden layer 15b) is h ¹ (z _in ) = σ _θ (W ¹ z _in + b ¹ ) as in the above expression (6). The output of the final layer (that is, the second layer and the hidden layer 15c) is calculated as a vector z _out = f _θ = W ^L h ^(L−1) + b ^L by a linear function. The output unit 15d uses the output vector z _out = [x _t−T ,..., X _t ,..., X _{t + T} ] as an estimated vector ^ x of the logarithmic spectrum of the noise-removed speech, for example, a signal reconstruction device. 20 output.

(Details of multi-channel feature calculation unit 13)
The multi-channel feature calculation unit 13 calculates a vector r _t = [r _{t, 1} ,..., R _{t, F} ] indicating the multi-channel feature amount. The vector _{r t,} and the like following.

(1) Interaural Level Difference (ILD)
The microphone 1 and 2 each observed signal, and each sensing sound binaural human, which takes the logarithm spectrum ratio of each observed signal, as a vector r _t, determined by the following equation (9). For details, refer to the document “Y. Liu, P. Zhang, and T. Hain,“ Using neural network front-ends on far field multiple microphones based speech recognitions ”, Proc. Of ICASSP2014, 2014, pp.5579-5583. Based.

(2) Interaural Phase Difference (IPD)
It is well known that the phase difference of observation signals in a plurality of microphones is a feature quantity that reflects the positional relationship between the sound source and the plurality of microphones. Therefore, as a vector r _t, the phase difference of the observed signals at the microphone 1 and 2 at the center frequency f'the f-th mel filter bank (f), as a vector r _t, determined by the following equation (10). ￣Y _{t, f ′ (f)} is a short-time Fourier transform coefficient of the logarithmic spectrum vector Y _{t, f} of the observed signal at the center frequency f _{′ (f)} of the f-th mel filter bank. ∠￣Y _{t, f ′ (f)} ⁽¹⁾ is the phase of the short-time Fourier transform coefficient of the observation signal of the microphone 1 at the center frequency f ′ (f) of the f-th mel filter bank, ∠￣Y _{t, f ′ (f)} ⁽²⁾ is the phase of the short-time Fourier transform coefficient of the observation signal of the microphone 2 at the center frequency f ′ (f) of the f-th mel filter bank.

Further, instead of the above equation (10), the level of the short-time Fourier transform coefficient of the observation signal at the microphones 1 and 2 at the center frequency f ′ (f) of the f-th mel filter bank obtained by the following equation (11): cosine values _{phi f'the} retardation of _(f), or as a vector _{r t.}

(3) Time frequency mask (MASK)
When there is multi-channel information, it is possible to calculate a feature value (for example, inter-channel phase difference IPD) that reflects the positional relationship between the sound source and a plurality of microphones, and cluster the feature values to perform sound speech enhancement time A frequency mask can be calculated. For example, the time frequency mask can be calculated by clustering the IPD vector r _t obtained by the above equation (5) or (6) obtained at each time frequency. For details, see T. Nakatani, S. Araki, T. Yoshioka, M. Delcroix, and M. Fujimoto, “Dominance based integration of spatial and spectral features for speech enhancement”, IEEE Trans. Audio, Speech and Language Processing, vol. 21, no. 12, pp. 2516-2531, 2013 ”.

Specifically, the time frequency mask M _{t, f} in the mel frequency domain can be calculated, for example, by converting the time frequency mask obtained in the short time frequency domain into the mel frequency domain. As the multi-channel feature quantity, a time frequency mask M _{t, f} in the mel frequency domain is used as in the following equation (12).

  Or you may use what took the logarithm of the time frequency mask in a mel frequency area like the following (13) Formula.

  The above equation (13) has a property that M is easy to take a value near 0 and 1, and it is difficult to take a value between them. Therefore, the above equation (13) may be converted into data having a unimodal property having a peak between 1 and 0 as in the following equation (14).

(4) Emphasized speech with time frequency mask (ENHANCE)
When the logarithm of the time frequency mask in the mel frequency domain is used as in the above equation (13), the emphasized speech ^ x _{t, f} = log (M _{t, f} · exp (y _{t, f} )) = Log (M _{t, f} ) + y _{t, f} is obtained. Therefore, the multi-channel feature quantity in the emphasized speech (ENHANCE) using the time-frequency mask is obtained by the following equation (15).

(5) Noise calculated with a time-frequency mask (NOISE)
Instead of the enhanced speech using the time-frequency mask, a noise signal estimated by the mask may be used as a multi-channel feature as shown in the following equation (16).

(6) Other multi-channel feature quantities For example, instead of the IPD, the time difference of arrival (TDOA) of observation signals at a plurality of microphones at the center frequency f ′ (f) of the f-th mel filter bank It may be used. This can be calculated, for example, by the well-known GCC-PHAT (Generalized Cross-Correlation PHAse Transform) method. This is a feature quantity that does not depend on the frequency, as shown in the following equation (17). In the following equation (17), ￣Y _{t, f ′ (f)} ⁽¹⁾ is a vector Y _t, of the logarithmic spectrum of the signal observed by the microphone 1 at the center frequency f ′ (f) of the f-th mel filter bank _{. It is} a short-time Fourier transform coefficient of _f . Further, ￣Y _{t, f ′ (f) *} ⁽²⁾ is a vector of the logarithmic spectrum of the signal observed by the microphone 2 at the complex conjugate f ′ (f) ^* of the center frequency f ′ (f) of the f-th mel filter bank. Y _{t, f is} a short-time Fourier transform coefficient. Also, | · | represents the norm of •. J is an imaginary unit.

Further, (or for example 5 seconds, 1, etc. utterance) a certain time taking a histogram of the calculated TDOA information between, the histogram may be used as a vector r _t multichannel feature quantity. In this case, t replaces the index of the bin in the histogram, not the time. In this case, the input vector z _in may not be connected by the context window. A vector obtained by arbitrarily selecting and arranging the vectors of the multichannel feature values listed above may be used as the multichannel feature value vector.

When the number M of microphones is 3 or more, the right shoulder subscript (2) is replaced with the subscript (3) in the above formulas (9), (10), (11), and (17). , (4) respectively calculated similarly for ..., those arranged in the order of subscripts obtained feature amount (i.e. the order of the microphone), or as a vector r _t multichannel feature quantity.

(Configuration of signal reconstruction device)
FIG. 2 is a diagram illustrating an example of the configuration of the signal reconstruction device. The signal reconstruction device 20 includes a noise removal filter calculation unit 21, a frequency domain conversion unit 22, a noise removal speech calculation unit 23, and an inverse Fourier transform unit 24. The noise removal filter calculation unit 21 applies a noise removal filter (Wiener filter) in the mel frequency region to the logarithmic spectrum ^ x _t at all frequencies of the noise-removed speech input from the noise removal device 10 according to the following equation (18). W _{t, f} is calculated.

Next, the frequency domain conversion unit 22 converts the noise removal filter W _{t, f} calculated by the noise removal filter calculation unit 21 into a noise removal filter ￣W _{t, f ′ in the} linear frequency domain. Here, f ′ is a linear frequency. Details are based on the literature “M. Fujimoto, S. Watanabe, and T. Nakatani,“ A robust estimation method of noise mixture model for noise suppression ”, Proc. Of Interspeech 2011, 2011, pp. 697-700.

Next, the noise-removed speech calculation unit 23 calculates a noise-removed speech in the linear frequency domain using ￣x _{t, f} = ￣W _{t, f} · ￣y _{t, f ′} . The inverse Fourier transform unit 24 performs a short-time inverse Fourier transform on ￣y _{t, f ′} calculated by the noise-removed sound calculation unit 23 to return it to a time waveform, so that the final waveform y of the noise-removed speech is obtained. _{t and f} are obtained. Here, ￣y _{t, f ′} is a linear frequency domain representation of y _{t, f} and is obtained by short-time Fourier transform of the observation signal.

(DAE learning process)
FIG. 3 is a flowchart illustrating an example of the DAE learning process. The DAE learning process is a process of learning the hidden layer parameter θ = {W, b}, and is executed by the noise removal apparatus 10. Incidentally, DAE learning process shown in FIG. 3, the input signal of the microphone 1 and an audio signal in the noise removal target, the calculation target of the audio signal of the multi-channel feature quantity r _t the input signal of the microphone 1 and 2.

First, the frequency analysis unit 11 of the DAE learning unit 14 analyzes the frequency of the input signal (and the original voice for learning) of the microphone 1 for every 100 ms frame, for example (step S11). Next, the output unit 11 connects the original speech for learning subjected to frequency analysis in step S11 to the previous and next T frames through a context window (step S12). Next, the logarithmic calculator 12 calculates a logarithmic spectrum of the input signal subjected to frequency analysis in step S11 (step S13). Next, the multi-channel feature calculation unit 13 calculates the multi-channel feature quantity r _t from the input signal of the microphone 1 and 2 (step S14).

Then, the input section 14a of the DAE learning unit 14, which is linked with the calculated log spectrum and the calculated multichannel feature quantity _{r t} Sorting context window vector back and forth T frames in step S14 in step S13 (step S15) . Next, the hidden layer 14b obtains an output for receiving the vector was ligated in the context window in step S15, the learned weight vectors ^{W 1} and the bias vector ^{b 1} (step S16). Next, the hidden layer 14c obtains an output for receiving the output of the hidden layer 14b in step S16, to learn the weight vector ^{W 2} and the bias vector ^{b 2} (step S17). Note that the output of the hidden layer 14c in step S17 coincides with the original voice for learning. And the output part 14d finally outputs the output of the hidden layer 14c by step S17 (step S18). The DAE learning unit 14 copies the hidden layer parameter θ = {W, b} learned by the DAE learning process to the DAE decoding unit 15.

(Noise removal processing)
FIG. 4 is a flowchart illustrating an example of the noise removal process. The noise removal process is a process for removing noise from the microphone input signal using the hidden layer parameter θ = {W, b} learned by the DAE learning process shown in FIG. Is done. Incidentally, the noise removal processing shown in FIG. 4, DAE learning process shown in FIG. 3 is a noise removal target speech signal of the input signal of the microphone 1, the calculation target input signal of the microphone 1 and 2 of the multi-channel feature quantity r _t If the audio signal is similarly input signal of the microphone 1 and an audio signal in the noise removal target, the calculation target of the audio signal of the multi-channel feature quantity r _t the input signal of the microphone 1 and 2.

First, the frequency analysis unit 11 analyzes the frequency of the input signal of the microphone 1 for every 100 ms frame, for example (step S21). Next, the logarithmic calculator 12 calculates a logarithmic spectrum of the input signal subjected to frequency analysis in step S21 (step S22). Next, the multi-channel feature calculation unit 13 calculates the multi-channel feature quantity r _t from the input signal of the microphone 1 and 2 (step S23).

Then, the input section 15a of the DAE decoder 15 couples the vector obtained by arranging multi-channel feature value _{r t} calculated in logarithmic spectrum and S23 calculated to the context window before and after T frame in step S22 (step S24) . Next, the hidden layer 15b inputs the vector was ligated in the context window in step S24, using the weight vectors ^{W 1} and a bias vector ^{b 1} of DAE learning section 14 learns the DAE learning process, outputs of the hidden layer 15b Is obtained (step S25). Next, the hidden layer 15c receives the output of the hidden layer 15b obtained in step S25, by using the weight vector ^{W 2} and a bias vector ^{b 2} of DAE learning section 14 learns the DAE learning process, the hidden layer 15c Is obtained (step S26). And the output part 15d finally outputs the output of the hidden layer 15c by step S26 (step S27).

[Effects of the embodiment]
5A to 5C are diagrams illustrating an example of the effect of the embodiment. The observation signal used in the experiment consists of the sound uttered at a position approximately 2 m away from the front of the M = 2 microphone array installed in the UK household and the noise in the UK household recorded with the same microphone array. It is a mixed signal. Noise includes non-stationary things (child's voice, vacuum cleaner sound, TV sound, etc.). All voice signals are command voices consisting of six English words.

The learning data consists of 6 hours of observation signals obtained by adding noise to the voice clean data of 34 speakers. The evaluation data is an observation signal obtained by adding noise (noise recorded in the same room as the learning data and different from the noise used in the learning data) to the voice uttered by the same 34 speakers, and the SN ratio is −6 dB, It consists of 600 utterances under each condition of −3 dB, 0 dB, 3 dB, 6 dB, and 9 dB. The dimension of the spectrum feature vector y _t is 40 dimensions in the mel frequency domain, and the length of the context window is T = 5. The frame length and frame shift are 100 ms and 25 ms, respectively.

The vector _{r t} multichannel features of the embodiments, the IPD based on the above (10) or (11), ENHANCE based on the above (15), and three kinds of NOISE based on the equation (16). The number of hidden layers used for DAE is 1, and the number of units (parameter dimensions) used in the hidden layers is 1,024. The multi-channel feature of the comparative example includes the observation signal itself without noise removal, the result of noise removal by the time frequency mask used for ENHANCE (see the above equation (15)), and the output of the DAE using only one microphone. And I tried four types of ILD.

  FIG. 5A and FIG. 5B show the noise removal performance by each method. The evaluation amount is a cepstrum distortion CD (Cepstral Distortion) and a segmental SN ratio (SSNR). The smaller the value of CD, the higher the performance. Moreover, SSNR shows a higher performance, so that a value is large. 5A and 5B show the CD and SSNR of the noise-removed speech obtained using each multi-channel feature amount for each S / N ratio of the observation signal. As shown in FIGS. 5A and 5B, it can be seen that the IPD, NOISE, and ENHANCE of the embodiment show higher performance than the comparative example.

  Further, FIG. 5C shows a comparative example of a non-noise-removed observation signal, a result of noise removal by the time-frequency mask used for ENHANCE (see the above equation (15)), and a DAE using only one microphone. The result of command voice recognition is shown for each of the output, ILD, and seven types of voices of IPD, ENHANCE, and NOISE as embodiments. As a speech recognizer, the literature “M. Delcroix, Y. Kubo, T. Nakatani, and A. Nakamura,“ Is speech enhancement pre-processing still relevant when using deep neural networks for acoustic modeling? ”, Proc. Of Interspeech2013, 2013, pp. 2992-2996. ", A deep neural network (DNN) based technology was used. FIG. 5C shows that IPD, ENHANCE, and NOISE as an embodiment have higher performance than the comparative example.

  That is, the embodiment estimates each original signal from acoustic data in which a plurality of signals are mixed in signal processing, and does not use information on how the original signal and the plurality of signals are mixed in sound source separation. In addition, in the blind sound source separation in which each original signal is estimated only from the acoustic data in which a plurality of signals are mixed, the noise removal performance is higher.

[Other Embodiments]
Embodiment, the observed signal y _t, but using a logarithmic spectrum in the Mel frequency domain, not limited thereto, and a logarithmic spectrum of a linear frequency domain, mel-frequency cepstral coefficients; and the like (MFCC Mel Frequency Cepstral Coefficient) It may be used. The signal reconstruction device 20 uses a Wiener filter. However, the present invention is not limited to this. For example, a time frequency mask M _{t, f} as shown in the following equation (19) may be used. Further, the context window need not be used in the output vector z _out in the above equation (5).

(About the device configuration of the noise removal device and the signal reconstruction device)
Each component of the noise removal apparatus 10 shown in FIG. 1 and the signal reconstruction apparatus 20 shown in FIG. 2 is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution and integration of the functions of the noise removal device 10 is not limited to the illustrated one, and all or a part of the functions can be functionally or physically in arbitrary units according to various loads, usage conditions, and the like. Can be distributed or integrated. For example, the DAE learning unit 14 and the DAE decoding unit 15 of the noise removal apparatus 10 may be integrated, or the process may be switched between DAE learning and noise removal processing. The frequency analysis unit 11, the logarithmic calculation unit 12, and the multi-channel feature calculation unit 13 are shared by the DAE learning unit 14 and the DAE decoding unit 15, but the DAE learning unit 14 and the DAE decoding unit 15 individually perform frequency analysis. May be configured to include a unit, a logarithm calculation unit, and a multi-channel feature calculation unit. In the embodiment, the noise removal device 10 and the signal reconstruction device 20 are separate devices, but are not limited thereto, and may be integrated devices.

  In addition, each or all of the processes performed in the noise removal apparatus 10 may be realized by a CPU (Central Processing Unit) and a program that is analyzed and executed by the CPU. Moreover, each process performed in the noise removal apparatus 10 and the signal reconstruction apparatus 20 may be realized as hardware by wired logic.

  In addition, among the processes described in the embodiment, all or a part of the processes described as being automatically performed can be manually performed. Alternatively, all or some of the processes described as being manually performed among the processes described in the embodiments can be automatically performed by a known method. In addition, the above-described and illustrated processing procedures, control procedures, specific names, and information including various data and parameters can be changed as appropriate unless otherwise specified.

(About the program)
FIG. 6 is a diagram illustrating an example of a computer in which a noise removal apparatus and a signal reconstruction apparatus are realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. In the computer 1000, these units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to the display 1061, for example.

  The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the noise removal apparatus 10 and the signal reconstruction apparatus 20 is stored in, for example, the hard disk drive 1031 as a program module 1093 in which a command executed by the computer 1000 is described. For example, a program module 1093 for executing information processing similar to the functional configuration in the noise removal device 10 and the signal reconstruction device 20 is stored in the hard disk drive 1031.

  The setting data used in the processing of the embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1031 to the RAM 1012 as necessary, and executes them.

  Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1031, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 via the network interface 1070.

  The above-described embodiments and other embodiments are included in the invention disclosed in the claims and equivalents thereof as well as included in the technology disclosed in the present application.

DESCRIPTION OF SYMBOLS 1, 2 Microphone 10 Noise removal apparatus 11 Frequency analysis part 12 Logarithm calculation part 13 Multichannel feature calculation part 14 DAE learning part 14a Input part 14b, 14c Hidden layer 14d Output part 15 DAE decoding part 15a Input part 15b, 15c Hidden layer 15d Output unit 20 Signal reconstruction device 21 Noise removal filter calculation unit 22 Frequency domain calculation unit 23 Noise removal speech calculation unit 24 Inverse Fourier transform unit 100 Noise removal system 1000 Computer 1010 Memory 1020 CPU

Claims (8)

A frequency analysis unit that performs frequency analysis on the first sound signal observed by the first sound collector and the second sound signal observed by the second sound collector disposed at a position different from the first sound collector;
A logarithm calculation unit for calculating a logarithm spectrum which is a logarithm of the amplitude of the first audio signal frequency-analyzed by the frequency analysis unit;
A multi-channel feature quantity calculation unit for calculating a multi-channel feature quantity related to the first voice signal and the second voice signal from the first voice signal and the second voice signal subjected to frequency analysis by the frequency analysis unit;
The logarithm spectrum of the first speech signal for learning calculated by the logarithm calculation unit, and the learning for the first speech signal for learning and the second speech signal for learning calculated by the multichannel feature amount calculation unit. A DAE learning unit that learns parameters of a DAE (Denoising AutoEncoder) that receives the multi-channel feature quantity of the input and outputs the logarithmic spectrum of the original speech for learning;
The logarithmic spectrum of the first speech signal to be denoised calculated by the logarithm calculation unit, the first speech signal to be denoised and the second speech signal to be denoised calculated by the multichannel feature amount calculation unit. The logarithmic spectrum of the noise-removed speech in which the noise component is removed from the logarithmic spectrum of the first speech signal to be denoised using the parameters learned by the DAE learning unit. And a DAE decoding unit that outputs the noise elimination device. The multi-channel feature quantity calculation unit, as the multi-channel feature quantity related to the first audio signal and the second audio signal subjected to frequency analysis by the frequency analysis unit, the first audio signal at each time frequency in the linear frequency domain and the A time-frequency mask for speech enhancement is calculated by clustering the amount characterizing the second speech signal into a speech cluster and a noise cluster, and converting the time-frequency mask for extracting the time-frequency component of the speech cluster into the mel frequency domain. The noise removal device according to claim 1, wherein: The multi-channel feature quantity calculation unit, as the multi-channel feature quantity related to the first audio signal and the second audio signal subjected to frequency analysis by the frequency analysis unit, the first audio signal at each time frequency in the linear frequency domain and the The logarithm of the time-frequency mask for performing speech enhancement in which the amount characterizing the second speech signal is clustered into a speech cluster and a noise cluster, and the time-frequency mask for extracting the time-frequency component of the speech cluster is converted into the mel frequency domain. The noise removal device according to claim 1, wherein: The multi-channel feature quantity calculation unit, as the multi-channel feature quantity related to the first audio signal and the second audio signal subjected to frequency analysis by the frequency analysis unit, the first audio signal at each time frequency in the linear frequency domain and the The logarithm of the time-frequency mask for performing speech enhancement in which the amount characterizing the second speech signal is clustered into a speech cluster and a noise cluster, and the time-frequency mask for extracting the time-frequency component of the speech cluster is converted into the mel frequency domain. The noise removal apparatus according to claim 1, wherein the logarithmic spectrum of the first audio signal calculated by the logarithm calculation unit is added to the logarithm calculation unit. The multi-channel feature quantity calculation unit, as the multi-channel feature quantity related to the first audio signal and the second audio signal subjected to frequency analysis by the frequency analysis unit, the first audio signal at each time frequency in the linear frequency domain and the A time frequency mask for performing speech enhancement is obtained by clustering the amount characterizing the second speech signal into a speech cluster and a noise cluster, and converting the time frequency mask for extracting the time frequency component of the speech cluster into the mel frequency domain. to that given a negative sign to the logarithm of the result of subtracting from, noise removal device according to claim 1, characterized in adding the log spectrum of the calculated first audio signal by the logarithm unit. The amount of characterizing is the phase difference between the first audio signal and the second audio signal at each time frequency in a linear frequency domain with respect to the first audio signal and the second audio signal frequency-analyzed by the frequency analysis unit. The noise removal device according to any one of claims 2 to 5, wherein: The multi-channel feature quantity calculation unit, as the multi-channel feature quantity related to the first audio signal and the second audio signal frequency-analyzed by the frequency analysis unit, the first audio signal and the first audio signal at the center frequency for each mel filter bank. The noise removal apparatus according to claim 1, wherein a phase difference between two audio signals or a cosine value of the phase difference is calculated.   The noise removal program which makes a computer function as a noise removal apparatus as described in any one of Claims 1-7.

Images