CN108172231A - A method and system for removing reverberation based on Kalman filter - Google Patents

Abstract

The invention discloses a method and system for removing reverberation based on Kalman filtering. The method includes: preprocessing the original signals collected by each microphone to obtain corresponding frequency domain signals, and forming input signals after delay; using Kalman The filter algorithm and the time-varying multi-channel autoregressive model estimate the reverberation signal, and the original signal collected by each microphone at the current moment is used as a reference signal, and the reverberation signal is subtracted to obtain the error signal; the Kalman gain matrix and the error signal are used to update the Kalman The coefficients of the Mann filter; the target signal is obtained by using the original signal collected by each microphone at the current moment, the input signal and the updated Kalman filter coefficient; finally, the target signal in the frequency domain is converted to the time domain by using the inverse Fourier transform. The method of the invention reduces the complexity of the adaptive multi-channel linear prediction reverberation algorithm by diagonalizing the Kalman filter state vector error covariance matrix.

Description

A method and system for removing reverberation based on Kalman filter

technical field

The invention relates to the field of speech reverberation, in particular to a method and system for reverberation based on Kalman filtering.

Background technique

As shown in Figure 1, due to the reflection of sound waves by room boundaries and objects in the room, the microphone receives reflected sounds from all directions in addition to the direct sound from the sound source. Generally, the sound signal arriving 30-50 ms after the direct sound is called the early reflection sound, and the sound signal arriving after that is called the late reflection sound, that is, the reverberation tail. Psychoacoustic research has found that early reflections can enhance the intensity of direct sound and improve speech intelligibility. The reverberation signal will mask the subsequent arrival of the direct sound signal, resulting in blurred speech. In addition, the reverberation signal will also reduce the voice quality of the signal received by the microphone and the accurate recognition rate of the voice recognition system. In application scenarios such as conference calls and smart speakers in a closed room, the microphone is often in the far field of the sound source. As the distance between the sound source and the microphone increases, the reverberation damages the signal received by the microphone more seriously. In addition, in the voice communication system, the environmental noise is small, and the signal received by the microphone is mainly affected by the reverberation of the room, resulting in a decrease in the accuracy and intelligibility of the voice signal, which seriously affects the communication quality. Therefore, it is a very necessary work to reverberate the signal received by the microphone.

Speech dereverberation is a hot research topic. The current solutions mainly include:

(1) Linear prediction residual enhancement algorithm. The speech model used by the linear prediction residual enhancement algorithm is the sound source filter model. In this model, speech is regarded as a series of excitation sequences passing through a time-varying all-pole filter. The estimated value of the all-pole filter coefficients can be obtained by performing linear prediction analysis on the reverberation speech signal, that is, the linear prediction coefficient. Then inverse filtering is performed on the signal received by the microphone to obtain the corresponding excitation signal, that is, the residual signal. The reverberation can be realized by enhancing the residual signal, and the speech signal can be reconstructed by the estimated linear prediction coefficient.

(2) Spectrum enhancement method. Spectral enhancement methods are a class of classic dereverberation algorithms. This method achieves the purpose of enhancing the speech signal by modifying the noisy or reverberant signal in the short-time Fourier transform domain. Literature [1] (K.Kinoshita, M.Delcroix, T.Nakatani, and M.Miyoshi, "Suppression of late reverberation effect on speech signal using long-term multiple-step linear prediction," IEEETrans.Audio,Speech,Lang.Process. , vol.17, no.4, pp.534–545, May 2009.) Estimation of late reverberation by delayed linear prediction followed by dereverberation with subsequent spectral subtraction. Literature [2] (F.Xiong, N.Moritz, R.Rehr, J.Anemuller, B.Meyer, T.G.G.Doclo, and S.Goetze, "Robust ASR inreverberant environments using temporal cepstrum smoothing for speechenhancement and an amplitude modulation filterbank for feature extraction, "inProc.REVERB Challenge Workshop, Florence, Italy, 2014.) Using the least mean square error method to estimate the magnitude spectrum of the clean speech signal as a preprocessing stage of automatic speech recognition, from the power of late reverberation and stationary background noise spectral density estimates the power spectral density of a clean speech signal. Generally, the spectral enhancement method needs to estimate the reverberation time in order to determine the spectral attenuation level. However, blind reverberation estimation is still a very difficult problem, especially in noisy environments, and the research on this problem is still in progress.

(3) Inverse filtering method. The blind dereverberation algorithm means that during the dereverberation process, the prior knowledge of the room impulse response between the sound source and the microphone is unknown. The multi-channel linear prediction algorithm based on microphone array is a classic blind dereverberation algorithm. According to the multiple input/output inverse theory (MINT), the multi-channel method can perfectly balance the time-invariant room impulse response under the condition that the transfer function of each channel does not contain a common zero. However, the MINT algorithm is very sensitive to system identification errors, and the impulse response of the actual room often contains similar zeros, so the MINT algorithm is difficult to apply in practice.

Because the time-domain linear prediction algorithm often requires a very long filter length, and there is a problem of whitening the target signal. Recently, some scholars proposed to apply multi-channel linear prediction algorithm in short-time Fourier transform domain to process signals independently in each sub-band. In the STFT domain, the reverberant speech signal is described by an autoregressive model in each frequency band, thereby reducing the filter length of each subband. Since the room impulse response is actually time-varying, it needs to be modeled with time-varying predictive model coefficients. Recently, some scholars have proposed a multichannel autoregressive (MAR) signal model in the STFT domain, using a Kalman filter to estimate MAR coefficients. This algorithm can be regarded as a generalized recursive least squares (Recursive least squares, RLS) algorithm.

The computational complexity of multi-channel linear prediction algorithm based on STFT domain is quadratic with the order of each sub-band filter. This complexity limits the application of the algorithm on many system platforms with limited resources. Literature [3] (Dietzen T, Doclo S, Spriet A, et al.Low-complexity Kalmanfilterformulti-channel linear-prediction-based blindspeechdereverberarion[C].IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.IEEE,2017.) for STFT domain adaptive multi-channel linear prediction dereverberation algorithm, a simplified Kalman filter solution method is proposed, which reduces the computational complexity to a linear relationship with the filter order. However, this simplified approach will result in a certain degree of voice quality degradation. In addition, the algorithm only estimates one channel signal, and multiple channels need to be calculated in practice.

Contents of the invention

The purpose of the present invention is to overcome the above-mentioned defects existing in the current de-reverberation method, and propose a low-complexity de-reverberation method based on Kalman filtering, which further reduces the STFT domain adaptive multiplicity while ensuring no loss of voice quality. Channel linear predicts the complexity of the dereverberation algorithm.

In order to realize the foregoing invention object, the present invention proposes a kind of de-reverberation method based on Kalman filter, and this method comprises:

The original signal collected by each microphone is preprocessed to obtain the corresponding frequency domain signal, and the input signal is formed after delay;

Using the Kalman filter algorithm and the time-varying multi-channel autoregressive model to estimate the reverberation signal, the original signal collected by each microphone at the current moment is used as a reference signal, and the reverberation signal is subtracted to obtain the error signal;

update the coefficients of the Kalman filter using the Kalman gain matrix and the error signal;

The target signal is obtained by using the original signal collected by each microphone at the current moment, the input signal and the updated Kalman filter coefficient;

Finally, the target signal in frequency domain is converted to time domain by inverse Fourier transform.

As an improvement of the above method, the method specifically includes:

Step 1) The signal y _m (n) collected by M microphones, 1≤m≤M is divided into frames, windowed and Fourier transformed to obtain the corresponding frequency domain signal Y _m (n),

The frequency domain signal Y _m (n) is:

Wherein, k is a frequency subscript, N is the number of points of Fourier transform; n is a time frame subscript, _wSTFT (1) is a short-time Fourier transform analysis window function, and R represents a frame shift;

Step 2) Constitute the input signal matrix Y(n-D) from the frequency domain signals of M microphones at time n-D to n-L, and use the Kalman weight vector to estimate the reverberation signal vector r(n), where D is the delay and L is the linear prediction length ;

y(n)=[Y ₁ (n),...,Y _M (n)] ^T (2)

In formula (3), I _M is the unit matrix of M×M, Represents the Kronecker product, Y(nD) is a sparse matrix of size M×L _c composed of microphone observation signals, L _c =M ² (L-D+1);

Calculate the reverberation signal vector r(n) according to formula (4);

In formula (4), The matrix C _p (n-1) of M×M is the time-varying Kalman weight vector coefficient, p=[D,D+1,...,L], and Vec{ } is the matrix column stacking operation factor;

Step 3) subtracting the reverberation signal vector r (n) obtained by the step 2) from the signal y (n) collected by each microphone at the current moment to obtain the error signal vector e (n);

e(n)=y(n)-r(n) (5)

Step 4) calculate Kalman gain matrix K (n);

Step 5) Update Kalman filter coefficients by Kalman gain matrix K(n) and error signal vector e(n)

Step 6) Use the signal y(n) collected by the microphone at the current moment, the input signal matrix Y(nD) and the updated Kalman filter coefficient Calculate the target signal vector x(n);

Step 7) Carry out inverse Fourier transform to the target signal vector x(n) in the frequency domain to obtain the target signal vector x _t (l) in the time domain:

As an improvement of the above method, the step 4) specifically includes:

Step 401) is calculated according to formula (6) using a first-order smoothing method

in, is the target signal variance at time n-1, is the target signal variance at time n-2, x(n-1) is the target signal vector at time n-1; α is the smoothing factor, the value is 0.2;

Step 402) First calculate the variance of the disturbance noise w(n) according to formula (7) Then calculate the prior misalignment variance according to formula (8)

In formula (7), L _c =M ² (L-D+1), η is usually 10 ^-5 ; is the posterior misalignment variance at time n-1;

Step 403) according to formula (9) by target signal variance and the prior misalignment variance Calculate the regularization factor δ(n);

Step 404) calculate the covariance matrix S _Y (nD) according to the signal collected by the microphone according to formula (10);

S _Y (nD) = Y (nD) Y ^H (nD) (10)

Step 405) calculate the Kalman gain matrix K (n) according to formula (11);

K(n)= ^YH (nD)[ _SY (nD)+δ(n) _IM ] ^-1 (11).

As an improvement of the above method, after the step 7) also includes:

Updated posterior misalignment variance

A de-reverberation system based on Kalman filtering, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements the above method when executing the program A step of.

The advantages of the present invention are:

1. The method of the present invention reduces the complexity of the adaptive multi-channel linear prediction de-reverberation algorithm by diagonalizing the Kalman filter state vector error covariance matrix;

2. The simplified Kalman filter algorithm of the present invention can be regarded as a normalized least mean square (Normalized Least Mean Square, NLMS) algorithm of variable regularization factors. In addition, the error signal vector e(n) and the target signal vector x(n) of the simplified Kalman filter algorithm proposed by the present invention are both M×1 vectors, which provides convenience for subsequent cascading of other multi-channel algorithms. In addition, to calculate the variance of the target signal Provides more available information.

Description of drawings

Figure 1 is a schematic diagram of room reverberation;

Fig. 2 is the block diagram of Kalman filtering reverberation of the present invention;

Fig. 3 is the block diagram of Kalman weight vector update of the present invention;

Fig. 4 is the block diagram of calculating Kalman gain matrix module of the present invention;

Fig. 5 is a block diagram of estimating a prior misalignment variance of the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

A low-complexity de-reverberation method based on Kalman filtering, the method comprising:

Step 1) The signal y _m (n) collected by M microphones, 1≤m≤M, is divided into frames, windowed and Fourier transformed to obtain the corresponding frequency domain signal Y _m (k,n), which is simplified representation , the frequency subscript k will be omitted below;

The calculation of the frequency domain signal Y _m (k,n) is calculated according to formula (1):

Wherein, k is a frequency subscript, N is the number of points of Fourier transform; n is a time frame subscript, _wSTFT (1) is a short-time Fourier transform analysis window function, and R represents a frame shift;

Step 2) Constitute the input signal matrix Y(n-D) from the frequency domain signals of M microphones at time n-D to n-L, and use the Kalman weight vector to estimate the reverberation signal vector r(n), where D is the delay and L is the linear prediction length ;

Y(nD) is a sparse matrix with a size of M×L _c composed of microphone observation signals, where L _c =M ² (L-D+1). r(n) stands for late reverberation.

Obtain input signal matrix Y(n-D) according to formula (2) and (3);

y(k,n)=[Y ₁ (k,n),...,Y _M (k,n)] ^T (2)

In formula (3), Represents the Kronecker product.

Calculate the reverberation signal vector r(n) according to formula (4);

In formula (4), represents the estimated value of a certain signal, The M×M matrix C _p (n-1) is the time-varying Kalman weight vector coefficient coefficient, p=[D, D+1,...,L]. L is the linear prediction length, and the choice of delay D>1 is related to the frame overlap parameter of STFT (Short-time Fourier transform, STFT). The value should ensure that the correlation between x(n) and r(n) can be ignored. Vec{·} is the matrix column stacking operation factor.

Step 3) subtracting the reverberation signal vector r (n) obtained by the step 2) from the signal y (n) collected by each microphone at the current moment to obtain the error signal vector e (n);

e(n)=y(n)-r(n) (5)

Step 4) By input signal matrix Y(nD), target signal variance and the prior misalignment variance Calculate the Kalman gain matrix K(n); specifically include:

Step 401) Calculate the variance of the target signal at time n by means of first-order smoothing according to formula (6)

in, is the target signal variance at time n-1, is the target signal variance at time n-2, x(n-1) is the target signal vector at time n-1; α is the smoothing factor, the value is 0.2;

Step 402) First calculate the variance of the disturbance noise w(n) according to formula (7) Then calculate the prior misalignment variance according to formula (8)

In the formula (7), L _c =M ² (L-D+1), η is a small normal constant, generally recommended to be 10 ^-5 .

Step 403) according to formula (9) by target signal variance and the prior misalignment variance Calculate the regularization factor δ(n);

Step 404) calculate the covariance matrix S _Y (nD) according to the signal collected by the microphone according to formula (10);

S _Y (nD) = Y (nD) Y ^H (nD) (10)

Step 405) calculate the Kalman gain matrix K (n) according to formula (11);

K(n)=Y ^H (nD)[S _Y (nD)+δ(n)I _M ] ^-1 (11)

Step 5) Update Kalman filter coefficients by Kalman gain matrix K(n) and error signal vector e(n)

Step 6) Use the signal y(n) collected by the microphone at the current moment, the input signal matrix Y(nD) and the updated Kalman filter coefficient Calculate the target signal vector x(n);

Step 7) seek the inverse Fourier transform of frequency domain signal vector x (n), obtain time domain target signal vector x _t (l);

Step 8) Update the posterior misalignment variance

In formula (15), I _M is an M×M unit matrix, L _c =M ² (L-D+1), and L is the linear prediction length. tr[·] means to find the trace of the matrix.

As shown in FIG. 2 , FIG. 2 is a system block diagram of the Kalman filter-based low-complexity de-reverberation algorithm of the present invention. Among them, Y(nD) is the input signal matrix composed of the frequency domain signals of M microphones from nD to nL, r(n) is the reverberation signal vector estimated by the Kalman filter algorithm, y(n) is The reference signal vector formed by the signal collected by the microphone at the current moment, x(n) is the final output target signal vector. The Fourier transform module 201 means to perform Fourier transform on the signal collected by the microphone, and the Fourier transform of the mth microphone signal is represented by Y _m (n). The delay module 202 represents performing a delay operation on the signal collected by the microphone. The choice of delay D>1 is related to the frame overlap parameter of STFT, and the value should ensure that the correlation between x(n) and r(n) can be ignored. The Kalman filter module 203 indicates that the input signal is filtered by a Kalman filter to estimate a reverberation signal. The target signal vector x(n) is calculated by the summation module 204 . The inverse Fourier transform module 205 transforms the frequency domain signal into the time domain.

FIG. 3 is a functional block diagram of updating Kalman weight coefficients, which includes a Kalman gain calculation module 303 . The update amount of the weight vector is obtained from the error signal vector and the Kalman gain matrix, and the final output target signal vector x(n) can be calculated from the updated weight vector.

FIG. 4 is a functional block diagram for calculating the Kalman gain matrix, which includes a priori offset variance estimation module 403 . The product module 401 realizes the multiplication of two input variables, and the inversion module 402 represents performing an inversion operation on the input signal. Using the variance of the target signal Input signal matrix Y(nD) and prior offset error Computes the Kalman gain matrix. It is calculated by the prior misadjustment variance estimation module 403 . The Kalman gain is crucial for updating the filter weight coefficients and estimating the variance of the prior misalignment. First calculate R _e (n), and then calculate the Kalman gain matrix K (n).

The prior misalignment variance estimation module shown in Fig. 5 also reflects the posterior misalignment variance calculation method. The transpose module 501 represents performing a transpose operation on a matrix. Block 503 represents finding the trace of the matrix.

Through the above analysis and Figure 2, Figure 3 and Figure 4, the following conclusions can be drawn:

First of all, after adopting the technology of the present invention, the computational complexity of the adaptive multi-channel linear prediction de-reverberation algorithm in the STFT domain is greatly reduced;

Secondly, after adopting the technology of the present invention, not only the computational complexity is reduced, but also the output voice quality is guaranteed;

Finally, after adopting the technology of the present invention, a good compromise can be obtained between the tracking performance and the convergence performance of the Kalman filter.

The above fully shows that the present invention provides an effective de-reverberation technology, which can well remove the reverberation interference caused by room acoustic reflection, and improve the speech intelligibility and the accurate recognition rate of the automatic speech recognition system.

It should be noted that the simplified Kalman filter algorithm described in the present invention can be regarded as a variable regularization factor NLMS algorithm, where δ(n) can be regarded as a variable regularization factor. variance The estimation of the filter coefficient c(n) plays an important role, and the smaller Value characterizes good offset performance and poor tracking performance, larger The value characterizes good tracking performance and poor offset performance. in other words, The value highly determines the tracking performance and convergence performance of the Kalman filter. When the algorithm has not yet converged, and The difference is large, according to formula (7), A larger value is also taken at this time, thus providing fast convergence performance and tracking performance. When the algorithm starts to converge to a steady state, and The difference decreases, resulting in a smaller That is, lower misalignment.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than limit them. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all of them should be included in the scope of the present invention. within the scope of the claims.

Claims (5)

1. A method for removing reverberation based on Kalman filter, said method comprising: The original signal collected by each microphone is preprocessed to obtain the corresponding frequency domain signal, and the input signal is formed after delay; Using the Kalman filter algorithm and the time-varying multi-channel autoregressive model to estimate the reverberation signal, the original signal collected by each microphone at the current moment is used as a reference signal, and the reverberation signal is subtracted to obtain the error signal; update the coefficients of the Kalman filter using the Kalman gain matrix and the error signal; The target signal is obtained by using the original signal collected by each microphone at the current moment, the input signal and the updated Kalman filter coefficient; Finally, the target signal in frequency domain is converted to time domain by inverse Fourier transform. 2. the de-reverberation method based on Kalman filter according to claim 1, is characterized in that, described method specifically comprises: Step 1) The signal y _m (n) collected by M microphones, 1≤m≤M is divided into frames, windowed and Fourier transformed to obtain the corresponding frequency domain signal Y _m (n), The frequency domain signal Y _m (n) is: Wherein, k is a frequency subscript, N is the number of points of Fourier transform; n is a time frame subscript, _wSTFT (1) is a short-time Fourier transform analysis window function, and R represents a frame shift; Step 2) Constitute the input signal matrix Y(n-D) from the frequency domain signals of M microphones at time n-D to n-L, and use the Kalman weight vector to estimate the reverberation signal vector r(n), where D is the delay and L is the linear prediction length ; y(n)=[Y ₁ (n),...,Y _M (n)] ^T (2) In formula (3), I _M is the unit matrix of M×M, Represents the Kronecker product, Y(nD) is a sparse matrix of size M×L _c composed of microphone observation signals, L _c =M ² (L-D+1); Calculate the reverberation signal vector r(n) according to formula (4); In formula (4), The matrix C _p (n-1) of M×M is the time-varying Kalman weight vector coefficient, p=[D,D+1,...,L], and Vec{ } is the matrix column stacking operation factor; Step 3) subtracting the reverberation signal vector r (n) obtained by the step 2) from the signal y (n) collected by each microphone at the current moment to obtain the error signal vector e (n); e(n)=y(n)-r(n) (5) Step 4) calculate Kalman gain matrix K (n); Step 5) Update Kalman filter coefficients by Kalman gain matrix K(n) and error signal vector e(n) Step 6) Use the signal y(n) collected by the microphone at the current moment, the input signal matrix Y(nD) and the updated Kalman filter coefficient Calculate the target signal vector x(n); Step 7) Carry out inverse Fourier transform to the target signal vector x(n) in the frequency domain to obtain the target signal vector x _t (l) in the time domain: 3. the de-reverberation method based on Kalman filter according to claim 2, is characterized in that, described step 4) specifically comprises: Step 401) is calculated according to formula (6) using a first-order smoothing method in, is the variance of the target signal at time n-1, is the target signal variance at time n-2, x(n-1) is the target signal vector at time n-1; α is the smoothing factor, the value is 0.2; Step 402) First calculate the variance of the disturbance noise w(n) according to formula (7) Then calculate the prior misalignment variance according to formula (8) In formula (7), L _c =M ² (L-D+1), η is usually 10 ^-5 ; is the posterior misalignment variance at time n-1; Step 403) according to formula (9) by target signal variance and the prior misalignment variance Calculate the regularization factor δ(n); Step 404) calculate the covariance matrix S _Y (nD) according to the signal collected by the microphone according to formula (10); S _Y (nD) = Y (nD) Y ^H (nD) (10) Step 405) calculate the Kalman gain matrix K (n) according to formula (11); K(n)= ^YH (nD)[ _SY (nD)+δ(n) _IM ] ^-1 (11). 4. the de-reverberation method based on Kalman filter according to claim 3, is characterized in that, described step 7) also comprises after: Updated posterior misalignment variance 5. A de-reverberation system based on Kalman filtering, comprising a memory, a processor and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the program, it realizes The steps of the method described in any one of claims 1-4.