JP5457999B2 - Noise suppressor, method and program thereof - Google Patents

Description

  The present invention relates to a noise suppression device that suppresses a noise signal and extracts a target signal from an acoustic signal in which the noise signal is superimposed on an audio signal that is a target signal, and a method and a program thereof.

  When using automatic speech recognition technology in an actual environment, it is necessary to remove noise from signals other than the target signal (speech signal) to be processed, that is, an acoustic signal containing noise, and extract only the desired target signal. There is. Improvement of the noise suppression performance is a problem to be solved as soon as possible.

  In Non-Patent Document 1, a probability model of an input signal is generated from a probability model of a speech signal and a noise signal estimated in advance, and a difference between the probability model and the statistics of the entire input signal is expressed by Taylor expansion. Estimate using the EM algorithm to optimize the stochastic model of the input signal. After that, a method of suppressing noise using the optimized input signal probability model and speech signal probability model parameters is disclosed.

  In Non-Patent Document 2, a noise signal is estimated by a parallel nonlinear Kalman filter, a probability model is shared by voice signal section detection and noise suppression, and information sharing is made dense. A noise suppression method with a speech signal section detection function for designing a noise suppression filter is disclosed.

P. J. Moreno, B. Raj, and R. M. Stern, “A vector Taylor series approach for environment-independent speech recognition,” in Proceedings of ICASSP '96, vol. II, pp. 733-736, May 1996. Masakiyo Fujimoto, Kentaro Ishizuka, and Tomohiro Nakatani, “Study of Integration of Statistical Model-Based Voice Activity Detection and Noise Suppression,” in Proceedings of Interspeech ’08, pp. 2008-2011, Sept. 2008.

  In the technique disclosed in Non-Patent Document 1, a stochastic model of an input signal is optimized by an EM algorithm using the entire collected input signal. However, the noise signal included in the input acoustic signal has a steady feature. Noise suppression is performed on the assumption that However, many noise signals in the real environment have non-stationary characteristics. In other words, since the statistical characteristics of the noise signal vary with time, it is not possible to cope with the time variation of noise, and sufficient noise suppression performance cannot be obtained.

  Non-Patent Document 2 discloses a method of sequentially estimating a non-stationary noise signal using a parallel nonlinear Kalman filter, but does not take into account the presence of a potential component (parameter) of noise. Even if there is a component that is not suitable for the nonlinear Kalman filter successive estimation method, the noise signal is estimated by the successive estimation method. As a result, the estimation error of the noise signal increases, and sufficient noise suppression performance may not be obtained.

  The present invention has been made in view of such a point, and noise is estimated with high accuracy by decomposing a noise signal into a stationary component (bias component) and an unsteady component (residual component). It is an object of the present invention to provide a noise suppression device that can be suppressed, a method thereof, and a program.

  The noise suppression device of the present invention includes an acoustic feature extraction unit, a noise bias component estimation unit, a noise residual component estimation unit, and a noise suppression unit. The acoustic feature extraction unit receives an acoustic signal obtained by superimposing a noise signal on the target audio signal, and extracts a complex number spectrum and a log mel spectrum as acoustic feature amounts for each frame with a certain time length of the acoustic signal as a frame. To do. The noise bias component estimator optimally estimates the bias component that is the center of gravity of the acoustic feature amount space of the noise signal with the log mel spectrum and the parameters of the silent GMM and the clean speech GMM as inputs. The noise residual component estimation unit optimally estimates a residual component that is a difference between the noise signal and the bias component by using the log mel spectrum, the bias component, and the parameters of the silent GMM and the clean speech GMM as inputs. The noise suppression unit outputs an acoustic signal in which the noise signal is suppressed with the log mel spectrum, the complex spectrum, the bias component and the residual component, and the parameters of the silence GMM and the clean speech GMM as inputs.

  The noise suppression device according to the present invention decomposes an acoustic signal on which a noise signal is superimposed into a bias component not accompanied by a time change and a residual component accompanying a time change, and applies an estimation method suitable for each component. Since noise is estimated with high accuracy, noise suppression performance can be improved.

The figure which shows notionally the two-dimensional feature-value space of a noise signal. The figure which shows the function structural example of the noise suppression apparatus 100 of this invention. The figure which shows the operation | movement flow of the noise suppression apparatus. The figure which shows the function structural example of the noise bias component estimation part 11. FIG. The figure which shows the operation | movement flow of the noise bias component estimation part 11. FIG. The figure which shows the function structural example of the noise residual component estimation part 12. FIG. The figure which shows the operation | movement flow of the noise residual component estimation part 12. FIG. The figure which shows the function structural example of the noise suppression part 14. FIG. The figure which shows the function structural example of the noise suppression filter estimation part 140. FIG. The figure which shows the operation | movement flow of the noise suppression filter estimation part 140. The figure which shows the function structural example of the noise suppression filter application part 141. The figure which shows the operation | movement flow of the noise suppression filter application part 141. It is a figure which shows the audio | voice waveform of a time domain, (a) is the acoustic signal o ( _tau ) which superimposed the airport lobby noise on the audio | voice signal which is a target signal, (b) is the acoustic signal to the noise suppression apparatus of this invention. It is a figure which shows noise suppression sound ^ s ( _tau) obtained by inputting o ( _tau ).

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. In the following description, the symbols “^”, “˜”, etc. used in the description should be described immediately above the original character, but due to restrictions on text notation, Enter immediately before. In the formula, these symbols are written in their original positions. Each variable is a vertical vector unless otherwise specified. Prior to the description of the embodiments, the basic idea of the present invention will be described.

[Basic idea of the present invention]
The noise suppression apparatus of the present invention considers a noise signal by decomposing it into a time-invariant stationary component (bias component) and a non-stationary component (residual component) with time fluctuation.

The horizontal axis in FIG. 1 represents the first-dimensional acoustic feature value, and the vertical axis represents the second-dimensional acoustic feature value. Only a two-dimensional acoustic feature space is shown due to the problem of depiction. If the noise signal is considered to be composed of two components, a bias component and a residual component, the bias component μ _N can be regarded as the center of gravity of the acoustic feature amount space of the noise N _t , and the residual component to N _t is noise. it can be regarded as the difference between to N _t and the bias component mu _N.

Considering this, if the acoustic feature of noise in a certain frame t (for example, a 24-dimensional log mel spectrum vector) is N _t , N _t is a bias component that does not change with time as shown in Equation (1). It can be decomposed into mu _N and the residual component to N _t.

Then, in this invention, it expressed using autoregressive model shown the time variation of the residual component in the equation with the prediction error U _t (2).

Here, F is a matrix having autoregressive coefficients as diagonal components. The prediction error U _t is a multidimensional white noise with an average vector 0 and a diagonal dispersion matrix Σ _U. Each diagonal elements of sigma _U shall have a very small value (e.g., 0.001).
By substituting Equation (2) into Equation (1), the log mel spectrum vector N _t can be expressed by a biased autoregressive model as shown in Equation (3).

  The present invention estimates noise based on the biased autoregressive model shown in Equation (3) and performs noise suppression processing.

  FIG. 2 shows a functional configuration example of the noise suppression device 100 of the present invention. The operation flow is shown in FIG. The noise suppression device 100 includes an acoustic feature extraction unit 10, a noise bias component estimation unit 11, a noise residual component estimation unit 12, a GMM storage unit 13, and a noise suppression unit 14. The GMM storage unit 13 includes a silent GMM 130 and a clean voice GMM 131.

  The functions of the units other than the GMM storage unit 13 are realized when a predetermined program is read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executes the program.

The noise suppression apparatus 100 uses, as an input signal, an acoustic signal o _{τ in} which a noise signal is superimposed on an audio signal that is a target signal, and an acoustic signal having a certain time length as a frame while moving the start point with a certain time width in the time axis direction. Cut out and perform noise suppression processing for each frame. The acoustic signal _oτ is a signal that is discretely converted by an A / D converter (not shown), and the subscript τ represents a sampling point of the discrete signal. For example, one frame is set to 20 ms of Frame = 320 sample points (1/16 KHz × 320) when the sampling frequency is 16 KHz.

The acoustic feature extraction unit 10 extracts the complex spectrum Spt _t and the log mel spectrum O _t as acoustic feature amounts for each frame (step S10). The noise bias component estimation unit 11 receives the log mel spectrum O _t and the parameters of the silent GMM 130 and the clean speech GMM 131 as input, and optimally estimates the bias component μ _N that is the center of gravity of the acoustic feature space of the noise signal (step S11). ).

The noise residual component estimation unit 12 receives the log mel spectrum O _t , the bias component μ _N, and the parameters of the silence GMM 130 and the clean speech GMM 131 as inputs, and the residual component that is the difference between the noise signal and the bias component μ _N ~ N _t and the square error variance matrix ~ Σ _{N, t} are optimally estimated (step S12). The noise suppression unit 14 includes a log mel spectrum O _t , a complex spectrum Sp _t , a bias component μ _N , a residual component ˜N _t , a square error variance matrix ˜Σ _{N, t} , parameters of the silent GMM 130 and the clean speech GMM 131, , And an acoustic signal { _circumflex over (s) _{} τ in} which the noise signal is suppressed is output (step S14).

Thus noise suppression apparatus 100 estimates how the sound signal the noise signal is superimposed, the bias component without time variation, decomposed into residual component to N _t with time variation, suitable for the respective component Therefore, noise suppression performance can be improved. Hereinafter, the operation of each functional component of the noise suppression device will be described in detail.

The acoustic feature extraction unit 10 cuts out the acoustic signal ot _{, n} while moving the start point by _, for example, Shift = 160 sample points. At that time, for example, cut out by multiplying a window function w _n, such as a Hamming window as shown in equation (4).

Here, t represents the frame number, and n represents the nth sample point in the frame. A complex spectrum Spc _t = {Spc _{t, 0} ,... Is applied to the cut-out acoustic signal o _{t, n} by applying a fast Fourier transform process of M points (for example, 512) that is a power of 2 and a value equal to or greater than the frame. , Spc _{t, m} ,..., Spc _{t, M−1} }. m is a frequency bin number.

Next, a mel filter bank analysis process and a logarithmization process are applied to the absolute value of the complex spectrum Spt _{t, m} , and a vector O _t = {O having an L-dimensional (eg, L = 24) log mel spectrum as an element. _{t, 0} , ..., _{Ot, l} , ..., _{Ot, L-1} } are calculated. l is the element number of the vector.

Acoustic feature extraction unit 10 outputs the complex spectrum Spc _t noise suppressor 14, a logarithmic Mel spectrum O _t to the noise bias component estimator 11 and the noise residual component estimator 12 and the noise suppressor 14.

[Noise bias component estimation unit]
FIG. 4 shows a functional configuration example of the noise bias component estimation unit 11. The operation flow is shown in FIG. The noise bias component estimation unit 11 includes a bias component initial value estimation unit 110, a probability model generation unit 111, an expected value calculation processing unit 112, a parameter update processing unit 113, and a convergence determination processing unit 114.

The bias component initial value estimation means 110 receives the log mel spectrum O _t as input, and bias component initial value ^ μ _N ^{(i = 0) obtained} by averaging the log mel spectrum O _t for each predetermined number of frames, and its bias component initial value ^ μ _N ^{(i = 0)} to estimate the diagonal covariance matrix sigma _N in (step S110).

The bias component initial value ^ μ _N ^{(i = 0)} is calculated by equation (5) after initializing the repetition index i (step S110a) (step S110b).

Here, A is the number of frames required for initial value estimation (for example, A = 10). i indicates the number of repetitions of the i-th time. Diagonal covariance matrix sigma _N of bias component is estimated by equation (6) (step S110b).

Diagonal covariance matrix sigma _N is a parameter independent of the index i of the repeat.
Probabilistic model generation means 111, the bias component initial value ^{_{^ μ N (i = 0)}} , and sigma _N, constitute a probabilistic model of the logarithmic Mel spectrum _{O t} in GMM using the parameters of the silence GMM130 and clean speech GMM131 (step S111). The probability model of the log mel spectrum O _t is composed of a GMM as shown in the equation (7).

b _j ^{Bias (i)} (O _t ) is a probability model of the log mel spectrum O _t generated by the probability model generation unit 111, j = 0 is a probability model generated from the parameters of the silent GMM 130, and j = 1 is The probability model generated from the parameters of the clean speech GMM 131 is shown. The function N (•) is a probability distribution function of a normal distribution given by Expression (8).

Here, k is the number of the normal distribution included in the GMM, and K is the total number of normal distributions (for example, K = 256). Further, w _{j, k} is the mixing weight of the silent GMM 130 or the clean speech GMM 131, and μ _{O, j, k} ⁽ⁱ⁾ and Σ _{O, j, k} ⁽ⁱ⁾ are the bias component ^ μ _N ⁽ⁱ⁾ and the silent GMM 130 or clean. It is the average vector and diagonal dispersion matrix of the probability model of the log mel spectrum O _t generated from the parameters of the speech GMM 131.
The probability model μ _{O, j, k} ^{(i) of the} log mel spectrum O _t and the diagonal dispersion matrix Σ _{O, j, k} ⁽ⁱ⁾ are given by the following equations.

Here, μ _{S, j, k} and ΣS _{, j, k} are an average vector and a diagonal dispersion matrix of the silent GMM 130 or the clean speech GMM 131, respectively. The functions log (•) and exp (•) perform an operation for each element of the vector. “1” is a vertical vector with all elements being 1, I is a unit matrix, and H _{j, k} ⁽ⁱ⁾ is a Jacobian matrix of a function h (•).

Expectation value calculation processing unit 112 calculates the expected value of the cost function Q probabilistic model of the logarithmic spectrum Spc _t (·) in the repeating estimation of every predetermined number of frames (step S112). The expected value of the cost function Q (•) is calculated by the equation (12). This calculation corresponds to E-step in the EM algorithm.

Here, O _{0: T−1} = {O ₀ ,..., O _t ,..., O _T−1 }, where T is the total number of frames of the log mel spectrum O _t , P _{t, j} ⁽ⁱ⁾ and P _{t, j, k} ⁽ⁱ⁾ are posterior probabilities for GMM type j or normal distribution k in frame t given by the following equations, respectively. In particular, P _{t, j = 0} ⁽ⁱ⁾ is defined as a speech non-existence probability, and P _{t, j = 1} ⁽ⁱ⁾ is defined as a speech existence probability.

The parameter update processing unit 113 optimizes and updates the bias component ^ μ _N ⁽ⁱ⁾ that maximizes the expected value of the cost function Q (•) by the Newton method (step S113). This update step corresponds to M-step in the EM algorithm.

Updating bias component ^ mu _N ⁽ⁱ⁾ is carried out by obtaining a ^ mu _N ⁽ⁱ⁾ that maximizes the cost function Q a (·) in equation (12). This method is usually obtained by setting the partial derivative of the cost function Q (•) with respect to the bias component ^ μ _N ⁽ⁱ⁾ to zero. However, since the cost function Q (·) in Expression (12) is given by a nonlinear function, it is difficult to obtain an analytical solution of the bias component ^ μ _N ⁽ⁱ⁾ .
Therefore, the parameter update processing unit 113 optimizes the bias component ^ μ _N ⁽ⁱ⁾ by the following Newton method.

Here, ∇ Q ⁽ⁱ⁾ and ∇ ² Q ⁽ⁱ⁾ are the gradient vector and Hessian of the cost function Q (•) in the i-th iterative estimation, respectively.
The convergence determination processing unit 114 repeats the operations of the probability model generation unit 111, the expected value calculation processing unit 112, and the parameter update processing unit 113 until the bias component ^ _N ⁽ⁱ⁾ converges (step S114).
An example of the convergence condition is shown in the following equation. It is assumed that η = 0.0001.

When the convergence condition of Expression (16) is satisfied, μ _N = ^ μ _N ⁽ⁱ⁾ is set, and the processing of the noise bias component estimation unit 11 is terminated (Yes in Step S114a). If not, the repetitive index i is incremented (step S114b), and the processing after the probability model generation step S111 is repeated.

[Noise residual component estimation unit]
FIG. 6 shows a functional configuration example of the noise residual component estimation unit 12. The operation flow is shown in FIG. The noise residual component estimation unit 12 includes a residual component initial value estimation unit 120, a residual component prediction processing unit 121, a residual component estimation processing unit 122, a probability model generation processing unit 123, and a weighted average processing unit 124. And an expected value calculation processing means 125, a parameter update processing means 126, and a convergence determination processing means 127.

The residual component initial value estimating means 120 averages the residual component, which is the difference between the log mel spectrum O _t and the bias component μ _N output from the noise bias component estimating unit 11, for each predetermined number of frames. An initial value is estimated (step S120). The initial value of the residual component is estimated by the following equation as a parameter independent of the iteration index i, and is used as the initial value for all iteration estimation.

  Residual component initial value estimating means 120 sets the initial value of autoregressive matrix F as follows. For each element, the dimension of the autoregressive coefficient is, for example, one dimension.

  The residual component prediction processing unit 121 multiplies the residual component estimated value of the previous frame by the autoregressive matrix and predicts the residual component predicted value of the current frame by the autoregressive model (step S121). The parameters of the current frame are predicted by an autoregressive model as shown in the following equation.

In Expressions (20) and (21), ˜N _{t | t−1} ⁽ⁱ⁾ , ˜Σ _{N, t | t−1} ⁽ⁱ⁾ are the i-th iterative estimation and the residual component in frame t a predicted value of _t, in the case of t = 0 the prediction processing carried out as shown in equation (22) (23) using the initial value.

The residual component estimation processing unit 122 includes a log mel spectrum O _t , a bias component μ _N output from the noise bias component estimation unit 11, and a residual component predicted value predicted by the residual component prediction processing unit 121 to N _{t | t. −1} ⁽ⁱ⁾ , ˜N _{, t | t−1} ⁽ⁱ⁾ , parameters μ _{S, j, k} and Σ _{S, j, k} of the silent GMM 130 and the clean speech GMM 131 are input and are included in each GMM. The same number of residual component estimation value candidates as the total number of normal distributions are calculated (step S122).
Each GMM is estimated by the following equation.

In the above equation, ˜N _{t, j, k} ⁽ⁱ⁾ , ˜Σ _{N, t, j, k} ⁽ⁱ⁾ are the i-th iterative estimation and residual value candidates for the residual component ˜N _t in frame t. .
The probability model generation processing unit 123 includes residual component estimation value candidates ~ N _{t, j, k} ⁽ⁱ⁾ , ~ Σ _{N, t, j, k} ⁽ⁱ⁾ calculated by the residual component estimation processing unit 122; Using the bias component μ _N output from the noise bias component estimator 11 and the parameters μ _{S, j, k} and Σ _{S, j, k} of the silent GMM 130 and the clean speech GMM 131 as inputs, the GMM of the log mel spectrum at the current frame t Parameters ~ μ 0 _{, t, j, k} ⁽ⁱ⁾ , ~ Σ 0 _{, t, j, k} ⁽ⁱ⁾ are generated (step S123).
The parameters of GMM in the frame t of the log mel spectrum O _t are generated as shown in the following equation.

The weighted average processing means 124 receives the log mel spectrum O _t and the GMM parameters of the log mel spectrum in the current frame as input, calculates the speech non-existence probability / existence probability and the posterior probability, and weights residual component estimation value candidates. An estimated value of the residual component is calculated by averaging (step S124). By performing weighted averaging as shown in Expression (31), an i-th iterative estimation and an estimated value of the residual component in frame t are obtained.

The expected value calculation processing means 125 calculates the expected value of the cost function Q (•) of the logarithmic mel spectrum probability model in the iterative estimation for each predetermined number of frames using the probability model of the parallel nonlinear Kalman filter (step S125). This calculation corresponds to E-step in the EM algorithm.
The probabilistic model and likelihood b _j ^MNKF (O _t ) of the parallel nonlinear Kalman filter in the frame t is configured as shown in Expression (35).

  That is, the expected value of the cost function Q (•) of the probability model of the parallel nonlinear Kalman filter is obtained from the following equation.

  In Equation (36), the parallel nonlinear Kalman filter changes the probability model at each frame t, so that the expected value of the cost function Q (•) is recursively calculated as shown below in order to improve the calculation efficiency.

  When the expected value of the cost function Q (•) is calculated in the frame t, the process proceeds to the next frame t + 1 (step S125b). If the frame is t ≧ T, the residual component estimation by the parallel nonlinear Kalman filter in the i-th iterative estimation is terminated (Yes in step S125c).

The parameter update processing means 126 updates the autoregressive matrix ^ F ⁽ⁱ⁾ so as to maximize the expected value of the cost function Q (•) (step S126). The autoregressive matrix {circumflex over ⁽ F ⁾ } ⁽ⁱ⁾ that maximizes the expected value of the cost function Q (•) is obtained by setting the partial differentiation of the cost function Q (•) with respect to {circumflex over ⁽ F)} ⁽ⁱ⁾ to zero. That is, the autoregressive matrix ^ F ⁽ⁱ⁾ is given by the following equation.

Convergence determination processing means 127 includes residual component prediction processing means 121, residual component estimation processing means 122, probability model generation processing means 123, weighted average processing means 124, and expected value until autoregressive matrix ^ F ⁽ⁱ⁾ converges. The operations of the calculation processing unit 125 and the parameter update processing unit 126 are repeated (No in step S127a).
An example of the convergence condition is shown in the following equation. It is assumed that η = 0.0001.

When the convergence condition of Expression (39) is satisfied, F = ^ F ⁽ⁱ⁾ is set, and the process of the parameter update processing unit 126 is ended (Yes in Step S127a). If not satisfied, the repetitive index i is incremented, t = 0 is set (step S127b), and the processing after the residual component prediction processing step S121 is repeated.

(Noise suppression part)
FIG. 8 shows a functional configuration example of the noise suppression unit 14. The noise suppression unit 14 includes a noise suppression filter estimation unit 140 and a noise suppression filter application unit 141. The noise suppression filter estimator 140 includes the log mel spectrum O _t , the bias component μ _N , the residual components ˜N _t , ˜Σ _{N, t} , the parameters W _{j, k} , μ _{S of the} silence GMM 130 and the clean speech GMM 131. _{, J, k} , Σ _{S, j, k} are input, and the noise suppression filter W _{t, m} ^Lin is estimated.

Noise suppression filter applying unit 141 outputs the complex spectrum Spc _t, the noise suppression filter _W ^t, the noise suppression signal ^ s _tau that suppresses noise of ^{m Lin} as input. The operations of the noise suppression filter estimation unit 140 and the noise suppression filter application unit 141 will be described in detail.

[Reverberation suppression filter estimation unit]
FIG. 9 shows a functional configuration example of the noise suppression filter estimation unit 140. The operation flow is shown in FIG. The noise suppression filter estimation unit 140 includes a probability model generation processing unit 1400, a probability calculation processing unit 1401, a noise suppression filter estimation processing unit 1402, and a noise suppression filter conversion processing unit 1403.

The probabilistic model generation processing means 1400 includes a bias component μ _N output from the noise bias estimation unit 11, residual components output from the noise residual component estimation unit 12 ˜N _t , ˜Σ _{N, t} , a silent GMM 130, and a clean speech GMM parameters _{μ S, j, k, Σ} S, j, as inputs and _k, a, is generated as follows the parameters of the GMM in the frame t of the logarithmic Mel spectrum _{O t} (step S1400).

The probability calculation processing means 1401 receives the log mel spectrum O _t , the GMM parameters output from the probability model generation processing means 140, and the silence w / GMM 130 and clean voice GMM parameters w _{j, k} as inputs, and the voice non-existence probability / existence Probability P _{t, j} and posterior probability P _{t, j, k} are calculated.
The voice non-existence probability / presence probability P _{t, j} is calculated by equation (43), and the posterior probability P _{t, j, k} is calculated by equation (44) (step S1401).

The noise suppression filter estimation processing means 1402 includes a bias component μ _N , residual components ˜N _t , ˜Σ _{N, t} , posterior probability P _{t, j, k} , speech non-existence probability / existence probability P _{t, j} , As an input, the noise suppression filter W _{t, l} ^Mel on the mel frequency axis is estimated by the following equation (step S1402). The following expression is a notation for each vector element.

The noise suppression filter conversion processing unit 1403 converts the noise suppression filter W _{t, l} ^Mel on the mel frequency axis into the noise suppression filter W _{t, m} ^Lin on the linear frequency axis by cubic spline interpolation (step S1403). .

[Noise suppression filter application unit]
FIG. 11 shows a functional configuration example of the noise suppression filter application unit 141. The operation flow is shown in FIG. The noise suppression filter application unit 141 includes filtering processing means 1410, inverse fast Fourier transform processing means 1411, and waveform connection processing means 1412.
The filtering processing means 1410 outputs the complex spectrum ＳS _{t, m} (Equation (46)) noise-suppressed by multiplying the complex spectrum Spc _t by the noise suppression filter W _{t, l} ^Mel (step S1410). Expression (46) is a notation for each vector element.

The inverse fast Fourier transform processing unit 1411 obtains the noise-suppressed speech ｓ _{t t, n} in the frame t by applying the inverse fast Fourier transform to the complex spectrum ＳS _{t, m} (step S1411).
Waveform connection process unit 1412, a noise reduced speech ^ s t of each _{frame, n,} the noise reduced speech linked while releasing the window function w _n as shown in the following equation was continuously ^ s _t, obtaining _n ( Step S1412).

[Results of evaluation experiment]
In order to confirm the effect of the present invention, an experiment was conducted to evaluate the noise suppression performance of the noise suppression device of the present invention. First, experimental conditions will be described.

  The evaluation data uses 100 sentences of 23 voices from IPA (Information-technology promotion agency, Japan) -98-TestSet. These voice data are used for airport lobby and station platform. The noise recorded separately on the street was superimposed on the computer with S / N ratios of 0 dB, 5 dB, and 10 dB, respectively. That is, nine types of evaluation data of three types of noise × three types of S / N ratios were created.

  Each audio data is a monaural signal discretely sampled at a sampling frequency of 16 KHz and a quantization bit number of 16 bits. The acoustic feature extraction unit 10 was applied to this acoustic signal by setting the time length of one frame to 20 ms (1 frame = 320 sample points) and moving the start point of the frame every 10 ms.

  As the silent GMM 130 and the clean speech GMM 131, GMMs having a mixed distribution number K = 256 having an L = 24-dimensional logarithmic mel spectrum as acoustic features are used, and learning is performed using the silent signal and the clean speech signal, respectively.

The dimension of the autoregressive coefficient of the residual component initial value estimating means 120 is one dimension. The number of frames required for initial value estimation is A = 10. The parameter of the convergence condition of the convergence determination processing means 114 and 127 is η = 0.0001. In residual component prediction processing step S121, the respective diagonal components of the sigma _U gave 0.001.
The performance was evaluated by speech recognition, and the evaluation scale was the word error rate WER of the following equation.

  Here, N is the total number of words, D is the number of dropped error words, S is the number of replacement error words, and I is the number of insertion error words. The smaller the WER value, the higher the speech recognition performance.

  Speech recognition is based on a finite state transducer based recognizer (T.hori, et al., “Efficient WFST-based one-pass decoding with on-the fly hypothesis rescoring in extremely large vocabulary continuous speech recognition,” IEEE Trans. On ALSP. , vol. 15, no. 4. pp.1352-1365, May 2007.) The speaker model is a triphone HMM independent of speakers, and each HMM has a three-state left-to-right structure. Each state has 16 normal distributions. The total number of HMM states is 3,000.

The acoustic feature quantity for speech recognition is a 12-dimensional MFCC (Mel-frequency cepstral coefficient), logarithmic power value, each primary and A total 39-dimensional vector including a quadratic regression coefficient. The language model is Tri-gram and the vocabulary is 20,000 words.
Table 1 shows the evaluation results.

Thus, it has been confirmed that the noise suppression device of the present invention exhibits a noise suppression performance superior to that of the prior art. FIG. 13 shows an audio waveform in the time domain. FIG. 13A shows an acoustic signal o _τ in which airport lobby noise is superimposed on a voice signal that is a target signal. Figure 13 (b) is a noise reduced speech ^ s _tau obtained by inputting the sound signal o _tau in the noise suppressing device of the present invention. You can see how the noise is effectively suppressed.

  As described above, the noise suppression apparatus of the present invention decomposes an acoustic signal on which a noise signal is superimposed into a bias component that does not change with time and a residual component that changes with time, and each component is highly accurate. Since the estimation is performed, the noise suppression performance can be improved.

In the embodiment described, has been described with reference to hamming window to the window function w _n, rectangular window, Hanning window, may be used other window functions, such as Blackman windows. Further, instead of the silent GMM 130 and the clean speech GMM 131, another probability model such as an HMM (Hidden Markov Model) may be used as the probability model of the speech signal. In addition to the two GMMs, the silent GMM 130 and the clean voice GMM 131, more GMMs may be used. The dimension of the autoregressive coefficient may be set to 2 or more. By doing so, it is expected that the estimation performance of the residual component is improved according to the order of the autoregressive coefficient. Further, the weighted average processing means 124 may use the estimation result having the maximum weight instead of the weighted average.

  When the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  Further, the processes described in the above method and apparatus are not only executed in time series according to the order of description, but also may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording media, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (11)

An acoustic feature extraction unit that extracts an acoustic signal in which a noise signal is superimposed on an audio signal that is a target signal, and extracts a complex spectrum and a log mel spectrum as acoustic features for each frame with a certain time length of the acoustic signal as a frame; ,
A noise bias component estimator that optimally estimates a bias component that is the center of gravity of the acoustic feature amount space of the noise signal, using the log mel spectrum, silence GMM, and clean speech GMM parameters as inputs;
A noise residual component estimator that optimally estimates a residual component that is a difference between the noise signal and the bias component by using the log mel spectrum, the bias component, and the parameters of the silent GMM and the clean speech GMM as inputs. When,
A noise suppression unit that outputs an acoustic signal in which the noise signal is suppressed with the log mel spectrum, the complex spectrum, the bias component and the residual component, the silent GMM, and the parameters of the clean speech GMM as inputs; ,
A noise suppression device comprising: The noise suppression device according to claim 1,
A noise suppression apparatus, characterized in that the noise signal is represented by a sum of the bias component and the residual component expressed by an autoregressive model, and a time series of the noise signal is estimated by a biased autoregressive model. In the noise suppression device according to claim 1 or 2,
The noise bias component estimator is
With the log mel spectrum as an input, a bias component initial value obtained by averaging the log mel spectrum every predetermined number of frames, and bias component initial value estimating means for estimating a diagonal dispersion matrix of the bias component initial value;
Probability model generation means for constructing a logarithmic mel spectrum probability model by GMM using the bias component initial value and parameters of silent GMM and clean speech GMM;
An expected value calculation processing means for calculating an expected value of the cost function of the probability model of the complex spectrum in the repeated estimation for each predetermined number of frames;
Parameter update processing means for optimizing and updating the bias component that maximizes the expected value of the cost function by the Newton method;
A convergence determination processing unit that repeats the operations of the probability model generation unit, the expected value calculation processing unit, and the parameter update processing unit until the bias component converges;
A noise suppression device comprising: The noise suppression device according to any one of claims 1 to 3,
The noise residual component estimator is
A residual component initial value estimating means for averaging a residual component, which is a difference between the log mel spectrum and the bias component, every predetermined number of frames and estimating an initial value of the residual component;
A residual component prediction processing means for multiplying the residual component estimation value of the previous frame by the autoregressive matrix and predicting the residual component prediction value of the current frame by the autoregressive model;
The logarithmic mel spectrum, the bias component, the residual component prediction value, the silent GMM and the clean speech GMM parameters are input, and the residual component estimation is the same as the total number of normal distributions included in the respective GMMs. Residual component estimation processing means for calculating value candidates;
Probability model generation processing means for generating GMM parameters of the log mel spectrum in the current frame by using the residual component estimation value candidates, the silence GMM and the clean speech GMM parameters as inputs,
Using the log mel spectrum and the GMM parameters of the log mel spectrum in the current frame as inputs, the speech non-existence probability / presence probability and posterior probability are calculated, and the residual component estimation value candidates are weighted and averaged. A weighted average processing means for calculating an estimated value;
Expected value calculation processing means for calculating an expected value of the cost function of the logarithmic mel spectrum probability model in the iterative estimation for each predetermined number of frames using a parallel nonlinear Kalman filter probability model;
Parameter update processing means for updating the autoregressive matrix so as to maximize the expected value of the cost function;
Until the autoregressive matrix converges, the residual component prediction processing means, the residual component estimation processing means, the probability model generation processing means, the weighted average processing means, the expected value calculation processing means, and the parameter update processing means Convergence determination processing means for repeating the operation;
A noise suppression device comprising: The noise suppression device according to claim 4,
The parameter update processing means includes
A noise suppression apparatus, wherein the autoregressive matrix is optimized using a time series of the residual components and an EM algorithm. An acoustic feature extraction process in which an acoustic signal in which a noise signal is superimposed on an audio signal, which is a target signal, is input, and a complex spectrum and a log mel spectrum are extracted as acoustic features for each frame with a certain time length of the acoustic signal as a frame; ,
A noise bias component estimation process for optimally estimating a bias component, which is the center of gravity of the acoustic feature amount space of the noise signal, using the log mel spectrum, parameters of the silent GMM and the clean speech GMM as inputs;
A noise residual component estimation process for optimally estimating a residual component that is a difference between the noise signal and the bias component by using the log mel spectrum, the bias component, and the parameters of the silent GMM and the clean speech GMM as inputs. When,
A noise suppression process for outputting an acoustic signal in which the noise signal is suppressed with the log mel spectrum, the complex spectrum, the bias component, the residual component, the silence GMM, and the parameters of the clean speech GMM as inputs. ,
A noise suppression method comprising: The noise suppression method according to claim 6,
A noise suppression method, wherein the noise signal is represented by a sum of the bias component and the residual component expressed by an autoregressive model, and a time series of the noise signal is estimated by a biased autoregressive model. The noise suppression method according to claim 6 or 7,
The noise bias component estimation process is as follows:
With the log mel spectrum as an input, a bias component initial value obtained by averaging the log mel spectrum every predetermined number of frames, and a bias component initial value estimating step for estimating a diagonal dispersion matrix of the bias component initial value;
A probability model generation step of configuring a logarithmic mel spectrum probability model with GMM using the bias component initial value, and parameters of silent GMM and clean speech GMM;
An expected value calculation processing step of calculating an expected value of the cost function of the probability model of the probability model of the complex spectrum in the repeated estimation for each predetermined number of frames;
A parameter update processing step for optimizing and updating the bias component that maximizes the expected value of the cost function by the Newton method;
A convergence determination processing step that repeats the operations of the probability model generation step , the expected value calculation processing step, and the parameter update processing step until the bias component converges;
Including a noise suppression method. The noise suppression method according to any one of claims 6 to 8,
The noise residual component estimation process is as follows:
A residual component initial value estimating step of averaging a residual component, which is a difference between the log mel spectrum and the bias component, every predetermined number of frames to estimate an initial value of the residual component;
A residual component prediction processing step of multiplying a residual component estimation value of one frame before by an autoregressive matrix and predicting a residual component prediction value of the current frame by an autoregressive model;
The logarithmic mel spectrum, the bias component, the residual component prediction value, the silent GMM and the clean speech GMM parameters are input, and the residual component estimation is the same as the total number of normal distributions included in the respective GMMs. A residual component estimation processing step for calculating value candidates;
Probability model generation processing step of generating logarithmic spectrum GMM parameters in the current frame by using the residual component estimation value candidates, the silent GMM and the clean speech GMM parameters as inputs,
Using the log mel spectrum and the GMM parameters of the log mel spectrum in the current frame as inputs, the speech non-existence probability / presence probability and posterior probability are calculated, and the residual component estimation value candidates are weighted and averaged. A weighted average processing step to calculate an estimate;
An expected value calculation processing step for calculating an expected value of the cost function of the probability model of the log mel spectrum in the iterative estimation for each predetermined number of frames with a probability model of a parallel nonlinear Kalman filter;
A parameter update processing step for updating the autoregressive matrix so as to maximize the expected value of the cost function;
Until the autoregressive matrix converges, the residual component prediction processing step , the residual component estimation processing step , the probability model generation processing step , the weighted average processing step , the expected value calculation processing step, and the parameter update processing step Convergence determination processing step that repeats the operation;
Including a noise suppression method. The noise suppression method according to claim 9, wherein
The parameter update process step
A noise suppression method comprising the step of optimizing the autoregressive matrix using a time series of the residual components and an EM algorithm.   A program for causing a computer to execute the noise suppression method according to any one of claims 6 to 10.

Images