CN108322858B - Multi-microphone Speech Enhancement Method Based on Tensor Decomposition - Google Patents

Abstract

The invention discloses a kind of multi-microphone sound enhancement method based on tensor resolution, comprising: the multicenter voice signal that multiple microphones observe is indicated using 3D tensor, and will be on a series of tensor projection to orthogonal basis；Using statistical risk criterion is minimized, noise segment is collected in real time, and tracking noise covariance calculates optimal thresholding according to tensor block size.Received multi-channel data is expressed as a three rank tensors by the present invention, to retain original spatial information and temporal information, it is thus possible to be removed ambient noise and weak directionality noise more obviously, and be reduced voice distortion as much as possible.

Description

Multi-microphone Speech Enhancement Method Based on Tensor Decomposition

technical field

The invention relates to the field of voice noise reduction, in particular to a multi-microphone voice enhancement method based on tensor decomposition.

Background technique

In the field of speech enhancement, although the classic single-channel algorithm can remove more background noise, it is easy to cause speech distortion, and even bring "music" noise, resulting in speech quality damage. By using a microphone array and using a beamforming algorithm, better suppression of directional interference can be achieved.

Traditional speech enhancement algorithms based on microphone arrays can be divided into time-domain noise reduction algorithms and frequency-domain noise reduction algorithms. The time-domain algorithm usually stitches the speech frames output by each microphone, and performs optimal linear filtering on this extended frame. The frequency domain algorithm performs Fourier transform on each microphone frame, extracts the time-frequency units corresponding to these frames and forms a snapshot vector, and performs optimal linear filtering on the snapshot vector. However, vector-based representation methods cannot fully utilize the space, time, and frequency information carried in multi-channel data, so there is room for improvement.

In addition, in many cases, the noise received by the microphone array is not completely directional interference, which makes the beamforming technology based on the principle of spatial filtering extremely vulnerable to performance loss. For the background noise with no obvious directionality or even no directionality, the spatial filtering effect of the beamforming algorithm is not good, which will bring more noise residues.

Contents of the invention

The purpose of the present invention is to provide a multi-microphone speech enhancement method based on tensor decomposition. Compared with the traditional beamforming method, the present invention represents the received multi-channel data as a third-order tensor, thereby retaining the original spatial information and time information, so that background noise and weak directional noise can be removed more obviously, and Speech distortion is minimized as much as possible.

The purpose of the present invention is achieved through the following technical solutions:

A multi-microphone speech enhancement method based on tensor decomposition, comprising:

Step (1), express the observed multi-channel voice data as a third-order tensor, and use the third-order tensor as the observation tensor; and use three orthogonal square matrices to perform three dimensions of the observation tensor Sparse reconstruction to obtain a core tensor containing projection coefficients, including: using a preset or pre-selected orthogonal base as the projection matrix, projecting the three dimensions of the observation tensor onto three orthogonal square matrices, to obtain A core tensor containing the projection coefficients; the input to this step is the observation tensor, and the output is the core tensor containing the projection coefficients.

Step (2), pre-setting a non-negative threshold value, zeroing the projection coefficients in the core tensor whose amplitude is lower than the threshold value, realizing the suppression of noise and the reconstruction of clean speech; including: adopting the criterion of minimizing statistical risk Design the optimal threshold value, the size of the threshold value is determined by the standard deviation of the noise and the size of the observation tensor. The input in this step is the core tensor, and the output is a tensor representing clean speech.

Further, in the above-mentioned multi-microphone speech enhancement method based on tensor decomposition, step (1) includes:

Step (11), the observed multi-channel voice data is represented as a third-order tensor by the signal receiving model of the microphone array, and the third-order tensor is used as the observation tensor;

The signal reception model of the microphone array is expressed as follows:

Y (l, k, n) = X (l, k, n) + N (l, k, n)∈RL ^×K×N ;

Among them, Y represents the observation tensor, X represents the tensor to be estimated representing clean speech, N represents the noise tensor, Y (l,k,n), X (l,k,n) and N (l,k,n ) respectively represent the observation tensor, the tensor to be estimated, the nth receiving channel in the noise tensor, and the lth element of the kth frame, L, K, and N respectively represent the frame length, the number of frames, and the number of microphones;

Step (12), using three orthogonal square matrices to perform sparse reconstruction on the three dimensions of the observation tensor respectively, to obtain a core tensor containing projection coefficients;

The decomposition of the observation tensor usually has the following form:

Y ＝ Σ × ₁ U ₁ × ₂ U ₂ × ₃ U ₃ , Σ ∈ R ^{L′×K′×N′} , U ₁ ∈ R ^L×L′ ,

U ₂ ∈ R ^K×K′ , U ₃ ∈ R ^N×N′ , L′≤L, K′≤K, N′≤N

Where {U ₁ , U ₂ , U ₃ } represents the basis matrix, and Σ represents the core tensor. Specifically, U ₁ represents the basis matrix of the observation tensor mode-1 fiber Y (:,k,n), U ₂ represents the basis matrix of the observation tensor mode-2 fiber Y (l,:,n), and U ₃ represents The base matrix of the observation tensor mode-3 fiber Y (l, k,:), Σ includes the projection coefficients of the observation tensor on the base matrix {U ₁ , U ₂ , U ₃ }, L′, K’, N ' indicates the truncated dimension of the core tensor.

Through canonical polymorphic decomposition, we can decompose the observation tensor into the most basic rank-1 tensor summation form, and the canonical polymorphic decomposition of tensor can be obtained by solving the following formula:

stL'=K'=N'=R, Σ is diagonal

Obtain superdiagonal core tensors and non-orthogonal basis matrices. Here R denotes the rank of the clean speech tensor.

Through the orthogonal decomposition of tensor, we can decompose the observation tensor into the form of the product of three orthogonal basis matrices and the core tensor, and the orthogonal decomposition of tensor can be obtained by solving the following formula:

Obtain the off-diagonal core tensor and the orthonormal basis matrix.

Note that if L'≤L, K'≤K, N'≤N, pass directly The clean speech tensor can be approximately reconstructed to restore the original speech signal. In the present invention, we choose L'=L, K'=K, N' = N to obtain an orthogonal square matrix {U ₁ , U ₂ , U ₃ } as the base matrix; Projection coefficients whose absolute values are smaller than λ are set to zero, thereby achieving noise suppression. Usually, too large a threshold will bring more speech distortion, while a smaller threshold will bring more noise residue; the selection of the optimal threshold will be described in detail below:

For the following linear observation model:

y(i)=x(i)+n(i), i=1,2,...,Q

Here n(i) obeys a univariate Gaussian distribution, n(i) at different times is independent of each other, and x(i) obeys a Gaussian distribution, then the minimum statistical risk estimate for x(i) can be expressed as H _λ (y(i )), where H _λ (·) is a hard threshold operator, and its function is to set the components of y(i) lower than the threshold λ to 0; according to the criterion of minimizing statistical risk, the optimal threshold is

For the multi-channel data receiving model, the following relationship is satisfied:

Y = X + N = Σ × ₁ U ₁ × ₂ U ₂ × ₃ U ₃

Here the elements of the noise tensor N satisfy the assumption of mutual independence and Gaussian distribution, the above formula is equivalent to:

X can be recovered by restoring X '; since U ₁ , U ₂ , and U ₃ are all orthogonal matrices, and the orthogonal matrix corresponds to rotation transformation, the nature of N'independent identical distribution and Gaussian distribution will not be changed; the above formula The tensor in is expanded to a vector, then it can be rewritten as vector here The lengths are the same, both are NLK (that is, N×L×K, N, L, and K respectively represent the number of microphones, the frame length, and the number of frames); can be estimated as According to the definition of tensor X ', can be reconstructed into X ', and then X can be restored; and the optimal threshold is Here δ represents the standard deviation of the noise; N, L, and K represent the number of microphones, frame length, and frame number respectively; log represents the logarithm with base 2.

It can be seen from the technical solution provided by the present invention that, on the one hand, compared with the traditional multi-channel speech enhancement algorithm, the present invention represents the received data as a third-order tensor, which can effectively retain the spatiotemporal correlation characteristics in the original signal; On the other hand, compared with the traditional multi-channel enhancement algorithm, the calculation amount of the solution of the present invention is small, and effective noise reduction can be realized only by determining the optimal threshold coefficient.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings on the premise of not paying creative efforts.

Fig. 1 is the multi-channel speech enhancement algorithm flowchart that the embodiment of the present invention provides;

FIG. 2 is a schematic diagram of optimal threshold calculation provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The embodiment of the present invention provides a multi-microphone speech enhancement method based on tensor decomposition, as shown in Figure 1, which mainly includes the following steps:

Step 11. Select what kind of orthogonal basis matrix (3DDCT orthogonal basis, supervised orthogonal basis, unsupervised orthogonal basis).

Step 12: Project the tensor on the selected basis matrix; select an optimal threshold to truncate the projection coefficients.

The embodiment of the present invention provides a flowchart for calculating the optimal threshold operator according to the criterion of minimizing the statistical risk, as shown in FIG. 2 , which mainly includes the following steps:

Step 21, real-time tracking and extraction of silent/noise segments, and calculation of noise variance.

Step 22. Select the tensor block size according to the signal sampling rate, etc., and calculate the optimal truncation threshold.

The above solution of the present invention, compared with the traditional multi-channel speech enhancement algorithm, realizes the enhancement of the multi-channel speech signal by using high-order tensor representation, which can effectively retain the time-space correlation characteristics of the signal; in addition, compared to the traditional multi-channel Wiener filtering algorithm, the calculation amount of the scheme of the present invention is small, and the enhancement can be realized only by determining the optimal threshold.

For ease of understanding, the following two steps are described in detail below.

1. Use three orthogonal square matrices to sparsely reconstruct the three dimensions of the tensor

The signal reception model of the microphone array is expressed as follows:

Y (l, k, n) = X (l, k, n) + N (l, k, n)∈RL ^×K×N ;

Wherein, Y (l,k,n) is the lth element of the kth frame in the nth receiving channel.

The decomposition of the observation tensor usually has the following form:

Y ＝ Σ × ₁ U ₁ × ₂ U ₂ × ₃ U ₃ , Σ ∈ R ^{L′×K′×N′} , U ₁ ∈ R ^L×L′ ,

U ₂ ∈ R ^K×K′ , U ₃ ∈ R ^N×N′ , L′≤L, K′≤K, N′≤N

Where {U ₁ , U ₂ , U ₃ } represents the basis matrix, and Σ represents the core tensor. Specifically, U ₁ represents the basis matrix of the observation tensor mode-1 fiber Y (:,k,n), U ₂ represents the basis matrix of the observation tensor mode-2 fiber Y (l,:,n), and U ₃ represents The basis matrix of the observation tensor mode-3 fiber Y (l,k,:), Σ includes the projection coefficients of the observation tensor on these basis matrices, and L', K', N' represent the dimensions of the core tensor.

Canonical polymorphic decomposition solves the following problems by:

stL'=K'=N'=R, Σ is diagonal

Obtain superdiagonal core tensors and non-orthogonal basis matrices. Here R denotes the rank of the clean speech tensor.

Orthogonal tensor decomposition solves the following problems:

Obtain the off-diagonal core tensor and the orthonormal basis matrix.

Note that if L'≤L, K'≤K, N'≤N, pass directly The clean speech tensor can be approximately reconstructed to restore the original speech signal. In the present invention, we choose L'=L, K'=K, N'=N to obtain an orthogonal square matrix {U ₁ , U ₂ , U ₃ } as the base matrix; then design a threshold λ to convert the absolute Projection coefficients with values smaller than λ are set to zero, thereby achieving noise suppression. Usually, a too large threshold will bring more speech distortion, while a smaller threshold will bring more noise residue; the selection of the optimal threshold will be demonstrated later.

The embodiment of the present invention considers four basic matrices, the basic matrix {U ₁ , U ₂ , U ₃ } can be a 3-dimensional discrete cosine transform (3D-DCT) basic matrix, supervised basic matrix, (unsupervised) approximation basis matrix, the (unsupervised) exact basis matrix. The 3D-DCT basis matrix is defined by the following formula:

The 3D-DCT basis matrix is a data-independent general basis matrix.

In the actual process, we can collect clean multi-channel speech data as training data for a specific problem, so as to obtain the optimal orthogonal basis matrix for this problem. For example, the present invention obtains a supervised basis matrix by solving the following optimization problem

Here, X _i ∈ R ^L×K×N , i=1, 2, . . . , T represents a training block composed of clean speech. Due to the existence of hidden variables Σ _i , the above problem has no explicit optimal solution. We use a loop iterative method to obtain a local optimal solution. As a first step, we will Initialized as a 3D-DCT matrix; the second step, given We use a soft-threshold or hard-threshold operator to obtain the sparse Σ _i ; in the third step, given Σ _i and renew The fourth step, given Σ _i and renew The fifth step, given Σ _i and renew Steps 2 to 5 are repeated continuously until the whole process converges.

For example, in step 3, we need to solve the following optimization problem:

The question can be transformed into:

Here Xi ₍₁₎ denotes the mode-1 expansion matrix of the clean speech block. The above problem can be simplified to:

The problem is further equivalent to:

suppose The SVD decomposition of can be simplified as The above question further translates into:

Due to the orthogonal matrix It is impossible for the diagonal elements of to exceed 1, we have The equal sign is only in was established. That is, at this step, The best value of Similarly, we can update and The whole process can converge after 20-30 cycles.

The above-mentioned supervised base matrix achieves the best results on all training data, but in practical problems, the test data we face usually does not strictly match the training data, which will lead to a certain performance degradation of the supervised base matrix. Therefore, the present invention proposes to use unsupervised learning to automatically deduce from the test data the most suitable orthogonal basis matrix for the test data. The specific optimization problem is as follows:

here Denotes the unsupervised basis matrix, Represents a sparsified tensor containing projection coefficients. Based on the above problems, the present invention provides two kinds of unsupervised basis matrices, namely approximate basis matrix and exact basis matrix.

The approximate basis matrix can be obtained by a high-order singular value decomposition algorithm; by performing SVD decomposition on Y ₍₁₎ , Y ₍₂₎ and Y _(3), it can be obtained respectively The exact basis matrix needs to be further optimized on this basis. fixed first renew then fixed and renew And so on, update The whole process is iterated in a loop until convergence. For example, for An update can be transformed into:

hypothesis matrix The singular value decomposition of can be expressed as Then the solution to the above problem can be directly written as Here M', N' are singular vector matrices, and Σ' is a diagonal matrix composed of non-negative singular values.

2. Select a non-negative threshold value, and set the coefficients in the core tensor below the threshold to zero

For the following linear observation model:

y(i)=x(i)+n(i), i=1,2,...,Q

Here n(i) obeys a univariate Gaussian distribution, n(i) at different times is independent of each other, and x(i) obeys a Gaussian distribution, then the minimum statistical risk estimate for x(i) can be expressed as H _λ (y(i )), where H _λ (·) is a hard threshold operator, and its function is to set the components in y(i) lower than the threshold λ to 0. According to the criterion of minimizing statistical risk, the optimal threshold is

For the multi-channel data receiving model, the following relationship is satisfied:

Y = X + N = Σ × ₁ U ₁ × ₂ U ₂ × ₃ U ₃

Here the elements of the noise tensor N satisfy the assumption of mutual independence and Gaussian distribution. The above formula is equivalent to:

Here, we only need to restore X ' to restore X. Since U ₁ , U ₂ , and U ₃ are all orthogonal matrices, and the orthogonal matrix corresponds to rotation transformation, the properties of N ' independent and identical distribution and Gaussian distribution are not changed. Expanding the tensor in the above formula into a vector, it can be rewritten as vector here The lengths are the same, both are NLK (that is, N×L×K, N, L, and K respectively represent the number of microphones, the frame length, and the number of frames). So can be estimated as By definition, can be refactored to X ', which in turn can be reduced to X. Therefore, the optimal threshold is

Through the above description of the implementation manners, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the above-mentioned embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.), including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present invention.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technical field can easily conceive of changes or changes within the technical scope disclosed in the present invention. Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (3)

1. A multi-microphone speech enhancement method based on tensor decomposition is characterized by comprising the following steps:

step (1), expressing the observed multi-channel voice data as a third-order tensor, and taking the third-order tensor as an observation tensor; and three dimensions of the observation tensor are respectively subjected to sparse reconstruction by using three orthogonal square matrixes to obtain a core tensor containing a projection coefficient, and the method comprises the following steps: respectively projecting three dimensions of the observation tensor onto the three orthogonal matrixes by using a preset or preselected orthogonal basis as a projection matrix to obtain a core tensor containing a projection coefficient;

step (2), a non-negative threshold value is preset, and the projection coefficient with the amplitude lower than the threshold value in the core tensor is set to be zero, so that the suppression of noise and the reconstruction of clean voice are realized; the method comprises the following steps: and designing an optimal threshold value by adopting a minimum statistical risk criterion, wherein the size of the threshold value is determined by the standard deviation of the noise and the size of the observation tensor.

2. The tensor decomposition-based multi-microphone speech enhancement method of claim 1, wherein the step (1) comprises:

step (11), representing observed multi-channel voice data as a third-order tensor through a signal receiving model of a microphone array, and taking the third-order tensor as an observation tensor;

the signal reception model of the microphone array is represented as follows:

Y(l,k,n)＝X(l,k,n)+N(l,k,n)∈R^L×K×N；

wherein,Ya tensor is represented which is a tensor of observation,Xrepresenting the tensor to be estimated that represents the clean speech,Nthe tensor of the noise is represented as,Y(l,k,n)，X(l, k, n) andN(L, K, N) respectively represent an observation tensor, a tensor to be estimated, an nth receiving channel in a noise tensor and an L element of a kth frame, and L, K and N respectively represent the frame length, the number of frames and the number of microphones;

respectively carrying out sparse reconstruction on three dimensions of the observation tensor by using three orthogonal square matrixes to obtain a core tensor containing a projection coefficient;

the decomposition of the observed tensor takes the form:

Y＝Σ×₁U₁×₂U₂×₃U₃,Σ∈R^{L′×K′×N′},U₁∈R^L×L′,

U₂∈R^K×K′,U₃∈R^N×N′,L′≤L,K′≤K,N′≤N

wherein { U₁,U₂,U₃Denotes a base matrix of the image data set,Σrepresenting a core tensor; specifically, U₁Fiber expressing observation tensor mode-1YBase matrix of (: k, n), U₂Fiber expressing observation tensor mode-2YBase matrix of (l,: n), U₃Representing observed tensor mode-3 fibersYA base matrix of (l, k,: in the following order,Σincludes the projection coefficients of the observation tensor on the basis matrixes₁、×₂、×₃Respectively representΣ、U₁、U₂、U₃Sequentially multiplying the mode 1, the mode 2 and the mode 3, wherein L ', K ' and N ' represent the size of the core tensor;

by canonicalizing polymorphic decomposition, we can decompose the observation tensor into a form of the sum of a finite number of rank-1 tensors, and canonicalizing polymorphic decomposition of the tensor can be realized by the following formula:

s.t.L′＝K′＝N′＝R,

obtaining a core tensor of the over-diagonal and a non-orthogonal basis matrix, where R represents the rank of the clean speech tensor;

by orthogonal decomposition, we can decompose the observation tensor into the form of the product of three orthogonal basis matrices and one core tensor, and the orthogonal decomposition of the tensor can be realized by the following formula:

a non-diagonal core tensor and orthogonal basis matrices are obtained.

3. The multi-microphone speech enhancement method based on tensor decomposition as recited in claim 2, wherein in the step (2):

for the following linear observation model:

y(i)＝x(i)+n(i),i＝1,2,...,Q

where Q represents the number of total samples, where n (i) obeys a univariate Gaussian distribution, n (i) at different times are independent of each other, and x (i) obeys a Gaussian distribution, then the minimum statistical risk estimate for x (i) can be represented as H_λ(y (i)) wherein H_λ(. h) is a hard threshold operator whose effect is to set the components in y (i) below threshold λ to 0; based on the minimum statistical risk criterion, the optimal threshold is

For a multi-channel data reception model, the following relationship is satisfied:

Y＝X+N＝Σ×₁U₁×₂U₂×₃U₃

here the tensor of noiseNSatisfies the assumption of mutually independent, gaussian distributions, the above formula being equivalent to:

by recoveryX' immediate recoveryX(ii) a Due to U₁,U₂,U₃Are all orthogonal matrices, and the orthogonal matrices correspond to rotational transformations and thus do not changeNThe nature of the independent co-distribution, gaussian distribution; by unfolding the tensor in the above formula into a vector, it can be rewritten asHere vectorAll the lengths of the two groups are NLK;can be estimated asAccording to the pair tensorXIn the definition of' in the present specification,can be reconstructed intoX', which in turn can reduceX(ii) a And the optimal threshold isWhere δ represents the standard deviation of the noise; log represents base 2 logarithm and NLK represents N x L x K.