CN111145768A - Speech enhancement method based on WSHRRPCA algorithm - Google Patents
Speech enhancement method based on WSHRRPCA algorithm Download PDFInfo
- Publication number
- CN111145768A CN111145768A CN201911290388.2A CN201911290388A CN111145768A CN 111145768 A CN111145768 A CN 111145768A CN 201911290388 A CN201911290388 A CN 201911290388A CN 111145768 A CN111145768 A CN 111145768A
- Authority
- CN
- China
- Prior art keywords
- time
- spectrum
- speech
- enhanced
- fourier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000001228 spectrum Methods 0.000 claims abstract description 175
- 230000002087 whitening effect Effects 0.000 claims abstract description 64
- 238000000513 principal component analysis Methods 0.000 claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000013507 mapping Methods 0.000 claims abstract description 5
- 239000011159 matrix material Substances 0.000 claims description 30
- 230000003595 spectral effect Effects 0.000 claims description 30
- 239000013598 vector Substances 0.000 claims description 24
- 230000008707 rearrangement Effects 0.000 claims description 22
- 238000005070 sampling Methods 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 15
- 238000004088 simulation Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 9
- 230000008447 perception Effects 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 238000000354 decomposition reaction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 239000010432 diamond Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0224—Processing in the time domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
Abstract
The invention discloses a voice enhancement method based on a WSHRRPCA algorithm, which mainly solves the problem of poor voice enhancement effect of the existing algorithm in a colored noise environment, and specifically comprises the following steps: establishing a whitening model by using a noisy speech sample, whitening the noisy speech by using the model in a time domain, and then acquiring a time-frequency amplitude spectrum and a time-frequency phase spectrum of the noisy speech by using short-time Fourier transform; the method comprises the steps of disordering and rearranging the arrangement sequence of the spectrum elements in each column of the time-frequency amplitude spectrum by a Hash function mapping method, decomposing the spectrum elements by using a robust principal component analysis algorithm to obtain an enhanced time-frequency amplitude spectrum and restoring the arrangement sequence; and forming an enhanced time frequency spectrum by using the enhanced time frequency amplitude spectrum and the time frequency phase spectrum, reconstructing a complete time domain whitening enhanced voice signal, and performing inverse whitening processing on the signal by using a whitening model to obtain enhanced voice. The invention can effectively eliminate various noises in the noisy speech to achieve the aim of speech enhancement, and can be applied to a speech receiving system, a speech coding system and a speech recognition system.
Description
Technical Field
The invention belongs to the technical field of signal processing, and further relates to voice signal processing, in particular to a voice enhancement method based on a Whitened short-time Fourier spectrum Hash rearrangement Robust Principal Component Analysis WSHRRPCA (Whitened-short-time-Fourier-spectral-hash-reordered Robust Component Analysis) algorithm, which can be used for a voice receiving system and a voice recognition system, realizes voice enhancement and noise reduction in the voice receiving system, and improves the signal-to-noise ratio of an input signal in a front-end preprocessing part of the voice recognition system, thereby improving the anti-interference capability and the recognition rate of the system.
Background
The speech enhancement technology is widely applied to the fields of voice call, teleconference, scene recording, military eavesdropping, hearing aid equipment, voice recognition equipment and the like, and a plurality of preprocessing modules of the speech coding and speech recognition system are all related to the technology. Traditional speech enhancement algorithms are mainly classified into three main categories: spectral subtraction, statistical model-based algorithms, and subspace algorithms. However, these conventional speech enhancement algorithms have their limitations in application. Spectral subtraction often performs signal enhancement processing based on estimation of a noise spectrum, when some non-stationary noise occurs, the estimation of the noise spectrum is inaccurate, the signal enhancement effect is affected, and the algorithm is easy to generate 'unnaturalness' music noise; statistical model-based algorithms generally require the assumption that speech and noise signals are statistically independent and obey a gaussian distribution; the subspace algorithm needs to assume that the clean speech signal subspace and the noise subspace are orthogonal, but this assumption of subspace orthogonality is very unreasonable in practical situations. In order to break through the limitations of the conventional algorithms, people begin to find new theories. In recent years, the convex optimization-based compressed sensing and matrix rank minimization and low-rank matrix recovery theory derived from the convex optimization-based compressed sensing become one of research hotspots in the field of digital signal processing, and a matrix low-rank sparse decomposition algorithm, namely robust principal component analysis, serving as the low-rank matrix recovery theory has also been applied to the field of speech enhancement and achieves a better effect. However, the speech enhancement method based on robust principal component analysis has the following disadvantages: first, the method has good performance in white noise environment, but the energy distribution characteristics of colored noise and white noise are different, which makes the method have insufficient performance in eliminating colored noise; secondly, when noise is eliminated, a part of low-rank speech components are also removed, so that the speech components are lost, and the speech enhancement effect is influenced.
Sun et al, in their published paper "A novel Speech enhancement method based on constrained low-rank and sparse matrix decomposition" (Speech Communication,60: 44-55,2014), propose a Speech enhancement method based on a matrix decomposition algorithm with low rank and sparse constraints. The method comprises the following implementation steps: the method comprises the steps that firstly, a short-time Fourier transform is used for obtaining a time-frequency amplitude spectrum and a time-frequency phase spectrum of noise-containing voice, and a three-point median filter is used for smoothing the time-frequency amplitude spectrum; secondly, decomposing a time-frequency amplitude spectrum of the noisy speech by using a constraint low-rank and sparse matrix decomposition algorithm to obtain a low-rank matrix and a sparse matrix, and performing binary time-frequency masking processing on the sparse matrix; and thirdly, reconstructing a time spectrum of the enhanced voice by using the sparse matrix and the noisy voice phase spectrum, and reconstructing the enhanced voice in a time domain form by using inverse short-time Fourier transform. The main problem of this method is to reduce the possibility that the low rank speech component is erroneously eliminated only by limiting the size of the low rank matrix rank, which is not fundamentally solved, and thus, there is still a part of the low rank speech that is removed as noise. Meanwhile, the method increases the limitation on the sparsity of the sparse matrix, so that under the condition of strong background noise, a large number of voice components are eliminated, and the voice quality is reduced.
Disclosure of Invention
The invention aims to provide a speech enhancement method based on a whitening short-time Fourier spectrum Hash rearrangement robust principal component analysis algorithm aiming at the defects of the prior art, obtains high-quality enhanced speech in a noise environment, and is mainly applied to a speech receiving system, a speech coding system and a speech recognition system.
The specific idea for realizing the purpose of the invention is that firstly a whitening model is established by utilizing a part of samples of the noisy speech, the whitening model is used for whitening the noisy speech in the time domain, short-time Fourier transform is used for carrying out time-frequency analysis on the whitened noisy speech to obtain the time-frequency amplitude spectrum and the time-frequency phase spectrum thereof, then, the arrangement sequence of the spectrum elements in each column of the time-frequency amplitude spectrum is disordered and rearranged by using a Hash function mapping method to obtain a rearranged time-frequency amplitude spectrum, then the rearranged time-frequency amplitude spectrum is decomposed by using a robust principal component analysis algorithm to obtain an enhanced time-frequency amplitude spectrum, the arrangement sequence of each column of the spectrum elements is recovered, then, an enhanced time frequency spectrum is formed by utilizing the enhanced time frequency amplitude spectrum and the time frequency phase spectrum, a complete time domain whitening enhanced voice signal is reconstructed, and finally, the voice signal is subjected to inverse whitening processing by using the established whitening model to obtain enhanced voice. The invention can be used for speech enhancement in various speech processing systems, recover the quality and intelligibility of speech seriously polluted by noise and achieve the aim of enhancing noise-containing speech.
The method specifically comprises the following steps:
(1) generating whitened noisy speech xw(n)：
(1a) Selecting an integer value as a sample point number N within the range of [1000,1500], and taking the first N sampling points of a noisy speech x (N) to establish a whitening filter;
(1b) carrying out whitening treatment on the noise-containing voice x (n) by using the whitening filter obtained in the step (1a) to obtain whitened noise-containing voice xw(n)；
(2) Generating whitened noisy speech xw(n) time-frequency amplitude spectrum | DwI andtime-frequency phase spectrum ∠ Dw：
(2a) In [20,40 ]]The duration of each frame of speech signal is optionally selected from a range of milliseconds, and the length of each frame is 25%, 75%]Selecting a value in the range of (1) as the displacement of the next frame speech relative to the previous frame speech, and whitening the noise-containing speech xw(n) dividing into a plurality of short-time speech frames;
(2b) selecting an unprocessed frame of short-time speech in sequence according to a time sequence from all the short-time speech frames as a frame to be processed currently;
(2c) performing Fourier transform on a short-time speech frame to be processed currently to obtain a Fourier spectrum of the frame, calculating the amplitude and the phase of the Fourier spectrum, and obtaining a Fourier magnitude spectrum and a Fourier phase spectrum;
(2d) judging whether all short-time speech frames are processed or not, if so, executing the step (2e), otherwise, returning to the step (2 b);
(2e) taking the Fourier magnitude spectrum of each frame as a column vector, arranging the column vector according to the time sequence to form a time-frequency magnitude spectrum | D of the whitened noisy speechwTaking the Fourier phase spectrum of each frame as a column vector, arranging the column vector in time sequence to form a time-frequency phase spectrum ∠ D of the whitened noisy speechw；
(3) Generating a rearranged time-frequency amplitude spectrum | Dw|r：
(3a) Amplitude spectrum | D in timewIn all column vectors of l, sequentially selecting an unprocessed column according to a time sequence to serve as a Fourier magnitude spectrum to be processed currently;
(3b) generating a new arrangement sequence for the spectral elements in the current Fourier magnitude spectrum by utilizing a Hash function, and rearranging the spectral elements according to the sequence to obtain a rearranged Fourier magnitude spectrum;
(3c) judging whether the | D is processedwIf yes, executing the step (3d), otherwise, returning to the step (3 a);
(3d) all the rearranged Fourier magnitude spectrums are used as column vectors and are arranged according to time sequence to form a rearranged time frequency magnitude spectrum | Dw|r；
(4) Generating an enhanced time-frequency magnitude spectrum | Sw|：
(4a) In [6,10 ]]Is selected as an integer Q as the estimated rearrangement time-frequency amplitude spectrum | Dw|rThe number of columns used for medium noise intensity, using | Dw|rFront Q-column rebinned Fourier magnitude spectral estimation | Dw|rThe intensity of the noise in (1);
(4b) utilizing robust principal component analysis algorithm to rearrange time-frequency amplitude spectrum | D according to the noise intensity estimated in (4a)w|rEnhancing to generate sparse rearrangement time-frequency amplitude spectrum | Sw|r；
(4c) Restoring | S according to the arrangement order generated in (3b)w|rThe arrangement sequence of the Fourier magnitude spectrum elements in all the columns to obtain an enhanced time frequency magnitude spectrum | Sw|；
(5) Composing an enhanced time spectrum Sw：
By enhancing the time-frequency amplitude spectrum | SwI and time-frequency phase spectrum ∠ DwComposing an enhanced time spectrum Sw；
(6) Reconstructed whitened enhanced speech yw(n)：
(6a) Spectrum S at enhancementwSequentially selecting an unprocessed column from all the column vectors according to a time sequence to serve as an enhanced Fourier spectrum to be processed currently;
(6b) performing inverse Fourier transform on the enhanced Fourier spectrum to be processed currently to obtain a frame of whitened short-time enhanced voice;
(6c) judging whether the processing is finished SwIf yes, executing the step (6d), otherwise, returning to the step (6 a);
(6d) reconstructing all whitened short-time enhanced speech frames into complete whitened enhanced speech y using overlap-Add Overlapped Add methodw(n)；
(7) Generating enhanced speech y (n):
whitening enhanced speech y using the whitening filter obtained in (1a)w(n) performing inverse whitening processing to obtain enhanced speech y (n).
Compared with the prior art, the invention has the following advantages:
firstly, the processing procedure of whitening is added in the invention, when the background noise is colored noise, the colored noise can be converted into white noise, and the capability of eliminating the colored noise is improved; moreover, the whitening processing does not influence the noise reduction capability of the invention in a white noise environment;
secondly, the invention uses the Hash function mapping method to carry out disordering rearrangement on the arrangement sequence of each column of spectrum elements of the original time-frequency amplitude spectrum before generating the enhanced time-frequency amplitude spectrum, so that the low-rank voice components in the invention become close to full rank and no longer have the characteristic of low rank, the low-rank voice components are effectively retained in the enhanced voice, and the quality of the enhanced voice is improved.
Drawings
FIG. 1 is a flow chart of an implementation of the method of the present invention;
fig. 2 is a diagram showing a comparison result between a top view of a time-frequency amplitude spectrum of colored noise F16 noise and a top view of a time-frequency amplitude spectrum of a whitening signal thereof in simulation experiment 1 according to the present invention.
FIG. 3 is a time-frequency amplitude spectrum visual comparison graph of the speech enhancement effect of the method of the present invention and the speech enhancement method based on the robust principal component analysis algorithm under the condition of colored noise F16 noise in simulation experiment 2 of the present invention;
FIG. 4 is a comparison graph of objective indexes of average speech enhancement effect of the method of the present invention and a speech enhancement method based on a robust principal component analysis algorithm under six different types of colored noise in simulation experiment 3.
Detailed Description
The implementation steps of the method of the invention are described in further detail below with reference to fig. 1.
(1.1) selecting an integer value as the number N of sample points within the range of [1000,1500], and taking the first N sampling points of noisy speech x (N) to establish a whitening filter; the specific steps for establishing the whitening filter are as follows:
(1.2) carrying out whitening treatment on the noise-containing voice x (n) by using the whitening filter obtained in (1.1) to obtain whitened noise-containing voice xw(n) of (a). The whitening processing of the noisy speech x (n) means: and (3) filtering the noisy speech x (n) by using the p-order whitening filter established in the step (1.1) of the step.
(2.1) in [20,40 ]]The duration of each frame of speech signal is optionally selected from a range of milliseconds, and the length of each frame is 25%, 75%]Selecting a value in the range of (1) as the displacement of the next frame speech relative to the previous frame speech, and whitening the noise-containing speech xw(n) are divided into a plurality of short-time speech frames.
And (2.2) sequentially selecting an unprocessed frame of short-time speech as a frame to be processed currently in time sequence from all the short-time speech frames.
And (2.3) carrying out Fourier transform on the short-time speech frame to be processed currently to obtain a Fourier spectrum of the frame, and calculating the amplitude and the phase of the Fourier spectrum to obtain a Fourier amplitude spectrum and a Fourier phase spectrum.
And (2.4) judging whether all short-time speech frames are processed, if so, executing the step (2.5) of the step, otherwise, executing the step (2.2) of the step.
(2.5) arranging the Fourier magnitude spectrum of each frame as a column vector in time sequence to form whiteTime-frequency amplitude spectrum | D for noise-containing voicewTaking the Fourier phase spectrum of each frame as a column vector, arranging the column vector in time sequence to form a time-frequency phase spectrum ∠ D of the whitened noisy speechwHere, the time-frequency magnitude spectrum | DwI and time-frequency phase spectrum ∠ DwAre all matrices and | Dw|∈Rm×n，∠Dw∈Rm×nWhere e denotes that the element belongs to the set, R denotes the matrix | DwI sum matrix ∠ DwWherein the elements are real numbers, and m is a matrix | DwI sum matrix ∠ DwN is the matrix | DwI sum matrix ∠ DwThe number of columns.
(3.1) amplitude Spectrum in time DwAnd in all column vectors of l, sequentially selecting an unprocessed column according to a time sequence to serve as a Fourier magnitude spectrum to be processed currently.
(3.2) generating a new arrangement sequence for the spectral elements in the current Fourier magnitude spectrum by utilizing a hash function, and rearranging the spectral elements according to the sequence to obtain a rearranged Fourier magnitude spectrum, wherein the method comprises the following specific steps:
let the current Fourier magnitude spectrum be X ═ X1,x2,…,xm]T∈Rm×1Wherein X is a column vector, X1,x2,…,xmM spectral elements are represented, subscripts 1,2, … of the spectral elements indicate arrangement sequence of the spectral elements in a Fourier magnitude spectrum, T indicates a vector transposition operation, e indicates that the elements belong to a set, and R indicates that the spectral elements are real numbers;
(3.2.1) selecting an integer a which is coprime to m in the range of [2, m), selecting an integer b in the range of [0, m), and constructing a hash function f (k) ═ ak + b)m+1, wherein (.)mDenotes a modulo operation, k denotes the subscript of the spectral element, and k ═ 1,2, …, m;
(3.2.2) mapping the original sequence of indices 1,2, …, m of the spectral elements to a new sequence of indices f (1), f (2), …, f (m) using a hash function f (k);
(3.2.3) according to the novel sequence of subscripts f (1)F (2), …, f (m) rearranging the spectral elements to obtain a rearranged Fourier magnitude spectrum Xr＝[xf(1),xf(2),…,xf(m)]T∈Rm×1。
(3.3) judging whether | D is processedwIf yes, executing the (3.4) step of the step, otherwise, executing the (3.1) step of the step.
(3.4) all the rearranged Fourier magnitude spectrums are used as column vectors and are arranged according to time sequence to form a rearranged time frequency magnitude spectrum | Dw|r。
(4.1) at [6,10]Is selected as an integer Q as the estimated rearrangement time-frequency amplitude spectrum | Dw|rThe number of columns used for medium noise intensity, using | Dw|rFront Q-column rebinned Fourier magnitude spectral estimation | Dw|rThe intensity of the noise in (2).
(4.2) according to the noise intensity estimated in the step (4.1) of the step, utilizing a robust principal component analysis algorithm to carry out rearrangement on the time-frequency amplitude spectrum | Dw|rEnhancing to generate sparse rearrangement time-frequency amplitude spectrum | Sw|r。
The robust principal component analysis algorithm is as follows: rearrangement time-frequency amplitude spectrum | D by using augmented Lagrange multiplier ALM methodw|rThe robust principal component analysis algorithm model carries out optimization solution, and a rearrangement time-frequency amplitude spectrum | D is decomposedw|rObtaining a sparse rearrangement time-frequency amplitude spectrum | Sw|rAnd low rank rearrangement time-frequency amplitude spectrum | Lw|r. The specific optimization process comprises the following steps:
at | Dw|r＝|Lw|′r+|Sw|′rUnder the condition, finding a sparse rearrangement time-frequency amplitude spectrum | Sw|rMatrix and low rank rearrangement time-frequency amplitude spectrum | Lw|rMatrix, such that Lw|r||*+λ|||Sw|r||1Has the smallest value, i.e.
Wherein, | Sw|′rRepresenting low rank rearranged time-frequency amplitude spectrum, | L, containing noise informationw|′rRepresenting a sparsely rearranged time-frequency amplitude spectrum containing speech information, | · | | luminance*Representing kernel norm operation, λ representing weight, | · | | | luminance1Representing a 1-norm operation.
(4.3) restoring | S according to the arrangement order generated in the (3.2) th step of the step 3w|rThe arrangement sequence of the Fourier magnitude spectrum elements in all the columns to obtain an enhanced time frequency magnitude spectrum | Sw|。
Using the enhanced time-frequency amplitude spectrum | S generated in step 4wAnd the time-frequency phase spectrum ∠ D obtained in the (2.5) step of the step 2wComposing an enhanced time spectrum Sw。
Step 6, reconstructing the whitened enhanced speech yw(n)。
(6.1) Spectrum S at enhancementwAnd sequentially selecting an unprocessed column in the all column vectors according to the time sequence to be used as the enhanced Fourier spectrum to be processed currently.
And (6.2) carrying out inverse Fourier transform on the enhanced Fourier spectrum to be processed currently to obtain a frame of whitened short-time enhanced voice.
(6.3) judging whether S has been processedwIf yes, executing the (6.4) th step of the step, otherwise, executing the (6.1) th step of the step.
(6.4) reconstructing all whitened short-time enhanced speech frames into complete whitened enhanced speech y using overlap-and-Add Overlapped Addw(n)。
And 7, generating enhanced voice y (n).
Using the whitening filter obtained in step 1 (1.1) to whiten the enhanced speech y obtained in step 6w(n) performing inverse whitening processing to obtain enhanced speech y (n).
Said whitening using the whitening obtained in step (1) of step 1The filter pair whitens the enhanced speech y obtained in step 6w(n) inverse whitening processing is performed to obtain enhanced speech y (n) as follows.
(7a) Using the whitening filter obtained in step 1 (1.1) to build an inverse whitening filter having a transfer function of WI(z)＝1/W(z)。
(7b) Speech enhancement using an inverse whitening filterw(n) filtering to obtain enhanced speech y (n).
The application effect of the invention is further explained by combining the following simulation:
1. simulation conditions
The simulation experiment of the invention is realized by MATLAB simulation software, the sampling rate of the voice is set to be 8000 Hz, the time length of each frame of short-time voice is 32 milliseconds, and the displacement of the next frame of voice relative to the previous frame of voice is 16 milliseconds. And taking the first 1024 sampling points of the noisy speech to establish a 40-order whitening filter. The method solves the robust principal component analysis algorithm by using an Exact ALM (Exact augmented Lagrange Multiplier) method in simulation, wherein the weight parameters of the robust principal component analysis algorithm and the rearranged Fourier magnitude spectrum | Dw|rThe noise intensity relation in (2) can be determined adaptively by the following formula:
λ＝-0.004×ζ+0.1181
wherein λ represents a weight parameter of the robust principal component analysis algorithm, and ζ represents a rearranged Fourier magnitude spectrum | Dw|rAn estimate of the signal-to-noise ratio in (1). Specifically, ζ can be determined by the following formula:
where ζ represents the rearranged Fourier magnitude spectrum | Dw|rEstimate of the signal-to-noise ratio in (log)10(. to.) denotes a base-10 logarithm operation,. sigma.,
representation matrix | Dw|rRow (i) and column (j)Squared spectral element of position, m being the matrix | Dw|rN is the matrix | Dw|rThe number of columns of (1), Q represents | Dw|rThe number of columns of the voice spectrum is set to 8 in the simulation experiment of the present invention.
2. Emulated content
The simulation experiments of the invention are three. Simulation experiment 1 is a whitening experiment of colored noise to illustrate the effectiveness of the whitening process in the present invention. Fig. 2 is a diagram showing the comparison result between the top view of the time-frequency amplitude spectrum of the colored noise F16 noise and the top view of the time-frequency amplitude spectrum of the whitened signal obtained in simulation experiment 1. Fig. 2(a) shows a top view of a time-frequency amplitude spectrum of a colored noise F16 noise, and fig. 2(a) shows a top view of a time-frequency amplitude spectrum of a signal obtained by whitening a colored noise F16. The horizontal axis in each time-frequency amplitude spectrum in fig. 2 represents the time axis in seconds, the vertical axis represents the frequency axis in kilohertz, and each time-frequency amplitude spectrum is represented in the form of a logarithmic spectrum with the spectral values in decibels.
The simulation experiment 2 is to visually compare the voice enhancement effect of the method of the present invention with the voice enhancement method based on the robust principal component analysis algorithm to obtain the time-frequency amplitude spectrum visual comparison graph of fig. 3. In simulation experiment 2, a clean speech segment is polluted by colored noise F16, the signal-to-noise ratio is 5dB, and speech enhancement is respectively carried out by using the method and the existing speech enhancement method based on the robust principal component analysis algorithm. Fig. 3(a) shows a top view of a time-frequency amplitude spectrum of clean speech, fig. 3(b) shows a top view of a time-frequency amplitude spectrum of colored noise F16, fig. 3(c) shows a top view of a time-frequency amplitude spectrum of a speech component obtained by a speech enhancement method based on a robust principal component analysis algorithm, fig. 3(d) shows a top view of a time-frequency amplitude spectrum of a noise component obtained by a speech enhancement method based on a robust principal component analysis algorithm, fig. 3(e) shows a top view of a time-frequency amplitude spectrum of a speech component obtained by the method of the present invention, and fig. 3(F) shows a top view of a time-frequency amplitude spectrum of a noise component obtained by the method of the present invention. The horizontal axis in each time-frequency amplitude spectrum in fig. 3 represents the time axis in seconds, the vertical axis represents the frequency axis in kilohertz, and each time-frequency amplitude spectrum is represented in the form of a log spectrum with spectral values in decibels.
3. And (3) simulation result analysis:
as can be seen from fig. 2, the colored noise F16 is mainly concentrated in the frequency bands of 0 hz-1.5 khz and 2.5 khz-3 khz, and after whitening, the signal energy is substantially uniformly distributed in the entire frequency band of 0 hz-4 khz, which is similar to white noise. Therefore, the whitening processing step in the present invention is effective in converting color noise into white noise.
FIG. 3 shows that the noise component time-frequency amplitude spectrum obtained by the speech enhancement method based on the robust principal component analysis algorithm in FIG. 3(d) has a large amount of speech components remaining, while the noise component time-frequency amplitude spectrum obtained by the method of the present invention in FIG. 3(f) has very few speech components remaining; meanwhile, the energy of the voice component in fig. 3(e) is larger than that in fig. 3(c), which intuitively shows that the algorithm of the present invention has a better voice enhancement effect. The method has the advantages that the arrangement sequence of each frame of Fourier spectrum elements is disturbed, so that the similarity between the time-frequency amplitude spectrum frame and the frame of the low-rank voice component is reduced, the condition that the low-rank voice component is wrongly decomposed is relieved, and the voice enhancement effect is improved.
As can be seen from fig. 4, the source distortion ratio curve obtained by the method of the present invention is above the source distortion ratio curve obtained by the speech enhancement method based on the robust principal component analysis algorithm. For the voice quality perception evaluation index, under the condition that the signal-to-noise ratio of the noise-containing voice is 0dB and-5 dB, the score of the method is slightly lower than that of a voice enhancement method based on a robust principal component analysis algorithm. By combining the two indexes, the method has better noise elimination capability in various colored noise environments, and simultaneously, more voice components are kept in the enhanced voice as much as possible, so that the method has good voice enhancement effect.
The foregoing description is only an example of the present invention and is not intended to limit the invention, so that it will be apparent to those skilled in the art that various changes and modifications in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (7)
1. A speech enhancement method based on a whitening short-time Fourier spectrum Hash rearrangement robust principal component analysis WSHRRPCA algorithm is characterized by comprising the following steps:
(1) generating whitened noisy speech xw(n)：
(1a) Selecting an integer value as a sample point number N within the range of [1000,1500], and taking the first N sampling points of a noisy speech x (N) to establish a whitening filter;
(1b) carrying out whitening treatment on the noise-containing voice x (n) by using the whitening filter obtained in the step (1a) to obtain whitened noise-containing voice xw(n)；
(2) Generating whitened noisy speech xw(n) time-frequency amplitude spectrum | DwI and time-frequency phase spectrum ∠ Dw：
(2a) In [20,40 ]]The duration of each frame of speech signal is optionally selected from a range of milliseconds, and the length of each frame is 25%, 75%]Selecting a value in the range of (1) as the displacement of the next frame speech relative to the previous frame speech, and whitening the noisy speech xw(n) dividing into a plurality of short-time speech frames;
(2b) selecting an unprocessed frame of short-time speech in sequence according to a time sequence from all the short-time speech frames as a frame to be processed currently;
(2c) performing Fourier transform on a short-time speech frame to be processed currently to obtain a Fourier spectrum of the frame, calculating the amplitude and the phase of the Fourier spectrum, and obtaining a Fourier magnitude spectrum and a Fourier phase spectrum;
(2d) judging whether all short-time speech frames are processed or not, if so, executing the step (2e), otherwise, returning to the step (2 b);
(2e) taking the Fourier magnitude spectrum of each frame as a column vector, arranging the column vector according to the time sequence to form a time-frequency magnitude spectrum | D of the whitened noisy speechwTaking the Fourier phase spectrum of each frame as a column vector, arranging the column vector in time sequence to form a time-frequency phase spectrum ∠ D of the whitened noisy speechw；
(3) Generating a rearranged time-frequency amplitude spectrum | Dw|r：
(3a) Amplitude spectrum | D in timewIn all column vectors of l, sequentially selecting an unprocessed column according to a time sequence to serve as a Fourier magnitude spectrum to be processed currently;
(3b) generating a new arrangement sequence for the spectral elements in the current Fourier magnitude spectrum by utilizing a Hash function, and rearranging the spectral elements according to the sequence to obtain a rearranged Fourier magnitude spectrum;
(3c) judging whether the | D is processedwIf yes, executing the step (3d), otherwise, returning to the step (3 a);
(3d) all the rearranged Fourier magnitude spectrums are used as column vectors and are arranged according to time sequence to form a rearranged time frequency magnitude spectrum | Dw|r；
(4) Generating an enhanced time-frequency magnitude spectrum | Sw|：
(4a) In [6,10 ]]Is selected as an integer Q as the estimated rearrangement time-frequency amplitude spectrum | Dw|rThe number of columns used for medium noise intensity, using | Dw|rFront Q-column rebinned Fourier magnitude spectral estimation | Dw|rThe intensity of the noise in (1);
(4b) utilizing robust principal component analysis algorithm to rearrange time-frequency amplitude spectrum | D according to the noise intensity estimated in (4a)w|rEnhancing to generate sparse rearrangement time-frequency amplitude spectrum | Sw|r；
(4c) Restoring | S according to the arrangement order generated in (3b)w|rThe arrangement sequence of the Fourier magnitude spectrum elements in all the columns to obtain an enhanced time frequency magnitude spectrum | Sw|；
(5) Composing an enhanced time spectrum Sw：
By enhancing the time-frequency amplitude spectrum | SwI and time-frequency phase spectrum ∠ DwComposing an enhanced time spectrum Sw；
(6) Reconstructed whitened enhanced speech yw(n)：
(6a) Spectrum S at enhancementwSequentially selecting an unprocessed column from all the column vectors according to a time sequence to serve as an enhanced Fourier spectrum to be processed currently;
(6b) performing inverse Fourier transform on the enhanced Fourier spectrum to be processed currently to obtain a frame of whitened short-time enhanced voice;
(6c) judging whether the processing is finished SwIf yes, executing the step (6d), otherwise, returning to the step (6 a);
(6d) reconstructing all whitened short-time enhanced speech frames into complete whitened enhanced speech y using overlap-Add Overlapped Add methodw(n)；
(7) Generating enhanced speech y (n):
whitening enhanced speech y using the whitening filter obtained in (1a)w(n) performing inverse whitening processing to obtain enhanced speech y (n).
2. The method of claim 1, further comprising: the step (1a) of establishing the whitening filter specifically comprises the following steps:
(1a1) in [30,50 ]]Selecting an integer p as the order of the whitening filter, and establishing a p-order linear predictor by using x (N) first sampling points of the noisy speech, wherein the transfer function of the linear predictor is
And solving the coefficient a of the linear predictor by using an autocorrelation methodi(i＝1,2,…,p)；
(1a2) Using a p-order linear predictor to build a p-order whitening filter having a transfer function of
3. The method of claim 1, further comprising: the whitening processing on the noisy speech x (n) in the step (1b) specifically includes: and (3) filtering the noisy speech x (n) by using the p-order whitening filter established in (1 a).
4. The method of claim 1, further comprising: the time-frequency amplitude spectrum | D in step (2e)wI and time-frequency phase spectrum ∠ DwAre all matrices and | Dw|∈Rm×n、∠Dw∈Rm×nWhere e denotes that the element belongs to the set, R denotes the matrix | DwI sum matrix ∠ DwWherein the elements are real numbers, and m is a matrix | DwI sum matrix ∠ DwN is the matrix | DwI sum matrix ∠ DwThe number of columns.
5. The method of claim 1, further comprising: in the step (3b), a new arrangement order is generated for the spectral elements in the current fourier magnitude spectrum by using a hash function, and the spectral elements are rearranged according to the new arrangement order to obtain a rearranged fourier magnitude spectrum, and the steps are as follows:
let the current Fourier magnitude spectrum be X ═ X1,x2,…,xm]T∈Rm×1Wherein X is a column vector, X1,x2,…,xmM spectral elements are represented, subscripts 1,2, … of the spectral elements indicate arrangement sequence of the spectral elements in a Fourier magnitude spectrum, T indicates a vector transposition operation, e indicates that the elements belong to a set, and R indicates that the spectral elements are real numbers;
(3b1) selecting an integer a which is coprime to m in the range of [2, m), selecting an integer b in the range of [0, m), and constructing a hash function f (k) ═ ak + b)m+1, wherein (.)mDenotes a modulo operation, k denotes the subscript of the spectral element, and k ═ 1,2, …, m;
(3b2) mapping the original index sequence 1,2, …, m of the spectral elements to a new index sequence f (1), f (2), …, f (m) using a hash function f (k);
(3b3) rearranging the spectral elements according to the new sequence of indices f (1), f (2), …, f (m) to obtain a rearranged Fourier magnitude spectrum Xr＝[xf(1),xf(2),…,xf(m)]T∈Rm×1。
6. The method of claim 1, further comprising: the robust principal component analysis algorithm in the step (4b) refers to: rearrangement time frequency amplitude by using augmented Lagrange multiplier ALM methodSpectrum | Dw|rThe robust principal component analysis algorithm model carries out optimization solution, and a rearrangement time-frequency amplitude spectrum | D is decomposedw|rObtaining a sparse rearrangement time-frequency amplitude spectrum | Sw|rAnd low rank rearrangement time-frequency amplitude spectrum | Lw|r。
7. The method of claim 1, further comprising: step (7) of using the whitening filter obtained in step (1a) to whiten the enhanced speech yw(n) performing inverse whitening processing to obtain enhanced speech y (n) as follows:
(7a) using the whitening filter obtained in (1a) to build an inverse whitening filter having a transfer function of WI(z)＝1/W(z)；
(7b) Speech enhancement using an inverse whitening filterw(n) filtering to obtain enhanced speech y (n).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911290388.2A CN111145768B (en) | 2019-12-16 | 2019-12-16 | Speech enhancement method based on WSHRRPCA algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911290388.2A CN111145768B (en) | 2019-12-16 | 2019-12-16 | Speech enhancement method based on WSHRRPCA algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111145768A true CN111145768A (en) | 2020-05-12 |
| CN111145768B CN111145768B (en) | 2022-05-17 |
Family
ID=70518298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911290388.2A Active CN111145768B (en) | 2019-12-16 | 2019-12-16 | Speech enhancement method based on WSHRRPCA algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111145768B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030014248A1 (en) * | 2001-04-27 | 2003-01-16 | Csem, Centre Suisse D'electronique Et De Microtechnique Sa | Method and system for enhancing speech in a noisy environment |
| US20100076756A1 (en) * | 2008-03-28 | 2010-03-25 | Southern Methodist University | Spatio-temporal speech enhancement technique based on generalized eigenvalue decomposition |
| CN109215671A (en) * | 2018-11-08 | 2019-01-15 | 西安电子科技大学 | Speech-enhancement system and method based on MFrSRRPCA algorithm |
-
2019
- 2019-12-16 CN CN201911290388.2A patent/CN111145768B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030014248A1 (en) * | 2001-04-27 | 2003-01-16 | Csem, Centre Suisse D'electronique Et De Microtechnique Sa | Method and system for enhancing speech in a noisy environment |
| US20100076756A1 (en) * | 2008-03-28 | 2010-03-25 | Southern Methodist University | Spatio-temporal speech enhancement technique based on generalized eigenvalue decomposition |
| CN109215671A (en) * | 2018-11-08 | 2019-01-15 | 西安电子科技大学 | Speech-enhancement system and method based on MFrSRRPCA algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111145768B (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107845389B (en) | Speech enhancement method based on multi-resolution auditory cepstrum coefficient and deep convolutional neural network | |
| CN110085249B (en) | Single-channel speech enhancement method of recurrent neural network based on attention gating | |
| JP7094340B2 (en) | A method for enhancing telephone audio signals based on convolutional neural networks | |
| CN109215671B (en) | Voice enhancement system and method based on MFrSRRPCA algorithm | |
| Soon et al. | Speech enhancement using 2-D Fourier transform | |
| CN111508518B (en) | Single-channel speech enhancement method based on joint dictionary learning and sparse representation | |
| CN111899750B (en) | Speech enhancement algorithm combining cochlear speech features and hopping deep neural network | |
| CN110808057A (en) | Voice enhancement method for generating confrontation network based on constraint naive | |
| Wang et al. | Joint noise and mask aware training for DNN-based speech enhancement with sub-band features | |
| CN114566176A (en) | Residual echo cancellation method and system based on deep neural network | |
| Do et al. | Speech Separation in the Frequency Domain with Autoencoder. | |
| Min et al. | Mask estimate through Itakura-Saito nonnegative RPCA for speech enhancement | |
| CN115574922A (en) | Hydroelectric generating set vibration signal noise reduction method and system based on cross entropy | |
| Vanjari et al. | Comparative Analysis of Speech Enhancement Techniques in Perceptive of Hearing Aid Design | |
| JP2023548707A (en) | Speech enhancement methods, devices, equipment and computer programs | |
| CN111145768B (en) | Speech enhancement method based on WSHRRPCA algorithm | |
| KR102033469B1 (en) | Adaptive noise canceller and method of cancelling noise | |
| CN105869649A (en) | Perceptual filtering method and perceptual filter | |
| CN111968651A (en) | WT (WT) -based voiceprint recognition method and system | |
| Hao et al. | An improved multivariate wavelet denoising method using subspace projection | |
| Nossier et al. | Two-stage deep learning approach for speech enhancement and reconstruction in the frequency and time domains | |
| Rao et al. | Speech enhancement using sub-band cross-correlation compensated Wiener filter combined with harmonic regeneration | |
| CN113066483B (en) | Sparse continuous constraint-based method for generating countermeasure network voice enhancement | |
| Sulong et al. | Speech enhancement based on wiener filter and compressive sensing | |
| CN106997766B (en) | Homomorphic filtering speech enhancement method based on broadband noise |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230424 Address after: 710065 5th Floor, Block B, Productivity Building, No. 3, Dianzi West Street, Electronic Industrial Park, Hi-tech Zone, Xi'an City, Shaanxi Province Patentee after: Xi'an Shengxin Technology Co.,Ltd. Address before: 710071 No. 2 Taibai South Road, Shaanxi, Xi'an Patentee before: XIDIAN University |
|
| TR01 | Transfer of patent right | ||
| CP02 | Change in the address of a patent holder |
Address after: Room 810, Building C, 8th Floor, Chuangye Building, No. 16 Gaoxin 1st Road, Xi'an City, Shaanxi Province, 710065 Patentee after: Xi'an Shengxin Technology Co.,Ltd. Address before: 710065 5th Floor, Block B, Productivity Building, No. 3, Dianzi West Street, Electronic Industrial Park, Hi-tech Zone, Xi'an City, Shaanxi Province Patentee before: Xi'an Shengxin Technology Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |