KR101290997B1 - A codebook-based speech enhancement method using adaptive codevector and apparatus thereof - Google Patents

Abstract

PURPOSE: A codebook-based speech enhancement method using an adaptive codevector and an apparatus thereof are provided to generate a super codebook by combining an existing noise codebook with adaptive codevectors that are generated through a long-term noise estimation, thereby guaranteeing performance on outside noises as well as on inside noises. CONSTITUTION: A speech enhancement apparatus estimates noise signals included in inputted speech signals using a long-term noise estimation algorithm and calculates a linear predictor parameter of the estimated noise signal (S360,S380). The speech enhancement apparatus generates a super codebook by saving N adaptive codevectors in a selected noise codebook (S400,S410). The speech enhancement apparatus outputs compensates speech signals using an integrated codevector, which is a combination of noise codevectors in the super codebook and clean codevectors in a clean codebook, and a spectral envelop value of the inputted speech signal (S420). [Reference numerals] (AA) Start; (BB) End; (S310) Input voice signals (y (n)); (S320) Generating a voice spectrum ¦ (y (k))¦^2 by Fourier conversion; (S330,S370) Fourier conversion; (S340) Extracting a linear estimation coefficient (a_y) of input voice signals (y (n)); (S350) Obtaining spectrum evelope value of input voice signals (y (n)); (S360) Estimating noise signals (d (n)); (S380) Extracting a linear estimation coefficient (a_d) of estimated noise signals (d (n)); (S390) Obtaining spectrum evelope value of estimated noise signals (d (n)); (S400) Selecting noise codebook having similar noise codevectors; (S410) Generating a super codebook by saving N adaptive codevectors in a selected noise codebook; (S420) Outputs compensated speech signals

Description

Codebook-based speech enhancement method and apparatus therefor using adaptive codevector

The present invention relates to a codebook-based speech enhancement method and apparatus using an adaptive codevector. Specifically, the present invention relates to a codebook-based speech enhancement method and apparatus capable of guaranteeing performance in all cases of noise and non-learned noise using adaptive codevectors.

The voice enhancement technique is required in various fields such as communication and music information processing using mobile devices, as well as voice recognition for robot control, and in particular, a technique for removing noise is important for improving the performance of the voice enhancement technique. Do.

Noise removal techniques use spectral subtraction (SS) methods that begin with the assumption that noise-contaminated speech is the sum of clean speech and background noise, and improves performance using the assumption that noise changes are independent of each frequency band. One multi-band spectral subtraction (MBSS) method is available. In addition, various algorithms have been studied, such as a method using voice activity detection (VAD) or a method using minimum statistics (MS).

In addition, methods such as Codebook Driven Short-Term Predictor Parameter Estimation (CDSTP) use a linear prediction coefficient of speech and noise as a database, and use the Maximum Likelihood (ML) or Minimum Mean Square Error (MMSE) estimation removes noise and obtains an improved speech signal. Because this technique uses various linear prediction coefficients included in the codebook, it shows excellent performance in both stationary and non-stationary background noise environments.

As described above, the existing codebook-based speech enhancement method uses a pre-trained linear prediction coefficient of speech and noise as a database and obtains an improved speech signal through ML or MMSE estimation with an input signal. However, since this technique uses various linear prediction coefficients included in the codebook, it shows excellent performance when the noise is in existing fixed codebooks, but does not guarantee performance with respect to untrained noise. There is this.

An object of the present invention is to provide a codebook-based speech enhancement method and apparatus using an adaptive codebook capable of guaranteeing performance against learned and unlearned noise.

Codebook-based speech enhancement method using an adaptive codevector according to an embodiment of the present invention for achieving the above object, the step of receiving an input speech signal mixed with a clean speech signal and a noise signal in units of frames, Generating an input speech power spectrum by Fourier transforming an input speech signal, obtaining a spectral envelope value of the input speech signal, estimating the noise signal from the input speech power spectrum, and spectra of the estimated noise signal Obtaining an envelope value, selecting a noise codebook having a noise code vector most similar to a spectral envelope value of the estimated noise signal among a plurality of stored noise codebooks, and spectral envelope of the estimated noise signal Set any N of values to adaptive codevector Storing the N adaptive codevectors in the selected noise codebook to generate a super codebook, and combining the noise codevectors in the super codebook and the clean codevectors in the clean codebook and the input speech. Extracting the corrected speech signal using the spectral envelope value of the signal.

Obtaining a spectral envelope value of the input speech signal, inversely Fourier transform the input speech power spectrum to generate an autocorrelation value of the input speech signal, the autocorrelation value of the input speech signal Levinson algorithm extracting a linear predictive coefficient for the input speech signal, and obtaining a spectral envelope value of the input speech signal using a frequency response value of the linear predictive coefficient for the input speech signal. It may include.

Estimating a noise signal from the input speech signal, and obtaining a spectral envelope value of the estimated noise signal, estimating the noise signal from the input speech power spectrum using a long-term noise estimation algorithm, the estimation Selecting one noise codebook from among the plurality of stored noise codebooks by using a noise type of the estimated noise signal, and inversely transforming a noise power spectrum of the estimated noise signal to obtain a magnetic signal of the estimated noise signal. Generating a correlation value, applying the autocorrelation value of the estimated noise signal to a Levinson algorithm, extracting a linear predictive coefficient for the estimated noise signal, and obtaining a linear predictive coefficient for the estimated noise signal The spectral envelope value of the input speech signal is determined using the frequency response value. Obtaining may include.

Selecting a noise codebook having a noise code vector most similar to the spectral envelope value of the estimated noise signal, selects the noise codebook using an Itakura-Saito Distortion (IS-D) algorithm Can be.

는 추정된 잡음신호, N_d는 코드북 데이터베이스 내 잡음 코드북의 개수, v_n은 n번째 코드북 내 코드벡터의 개수이고, a_d ^n,j은 n번째 잡음 코드북의 j번째 코드벡터의 선형 예측 계수이다. Where n 'is the selected noise codebook,

Is the estimated noise signal, N _d is the number of noise codebooks in the codebook database, v _n is the number of codevectors in the nth codebook, and a _d ^{n, j} is the linear prediction coefficient of the jth codevector of the nth noise codebook .

The generating of the super codebook comprises: setting any N of the spectral envelope values of the estimated noise signal as the adaptive codevector, storing the selected codeword in the selected noise codebook, and storing the selected noise codebook in the selected noise codebook. Integrating the noise codevector with the N adaptive codevectors to change the selected noise codebook into the super codebook.

The extracting of the corrected speech signal may include generating the integrated code vector by combining a noise code vector included in the super codebook and a clean code vector included in the clean codebook. Generating gain combination values by applying a gain value of a signal and a gain value of the estimated noise signal, obtaining respective similarities between the spectral envelope values of the input speech signal and the gain combination values, and The method may include obtaining a spectral envelope value of the clean speech signal and a spectral envelope value of the noise signal by applying a similarity and gain combination value to a maximum likelihood (ML) or Bayesian estimation algorithm (MMSE).

The generating of the corrected speech signal may include generating a corrected speech power spectrum from a spectral envelope value of the clean speech signal and a spectral envelope value of the noise signal through a Wiener filter, and The method may further include extracting the corrected speech signal by inverse Fourier transforming the corrected speech power spectrum.

The gain value of the clean speech signal and the gain value of the estimated noise signal may be obtained using a speech absence probability (SAP) algorithm.

The gain value c _g of the clean speech signal and the gain value n _g of the estimated noise signal may be calculated through the following equation.

이며, Y(w)는 입력 음성 신호의 주파수 응답 값을 나타내며, Aⁿ _d(w)은 슈퍼 코드북에 저장된 n번째 코드벡터를 나타내고, A^m _x(w)은 클린 코드북에 저장된 m번째 코드벡터를 나타낸다. here,

Where Y (w) represents the frequency response of the input speech signal, A ⁿ _d (w) represents the nth codevector stored in the super codebook, and A ^m _x (w) represents the mth codevector stored in the clean codebook Indicates.

Estimating a noise signal from an input speech signal of a next frame input following the input speech signal, obtaining a spectral envelope value of the estimated noise signal, and converting the spectral envelope value of the estimated noise signal into a new adaptive code vector. And removing at least one adaptive codevector from among the N adaptive codevectors stored in the super codebook, and storing the new adaptive codevector in the super codebook. .

The adaptive code vector having the earliest stored time can be removed from the N adaptive code vectors stored in the super codebook.

Codebook-based speech enhancement apparatus using an adaptive codevector according to another embodiment of the present invention, a speech signal input unit for receiving an input speech signal of a clean speech signal and a noise signal in units of frames, Fourier transform the input speech signal A Fourier transform unit for generating an input speech power spectrum, an input speech linear prediction coefficient extractor for obtaining a spectral envelope value of the input speech signal, and estimating the noise signal from the input speech power spectrum, A noise signal linear prediction coefficient extracting unit for obtaining a spectral envelope value, a codebook database including a clean speech codebook having a plurality of clean code vectors and a plurality of noise codebooks having a plurality of noise code vectors and having different types, the plurality of Among the noise codebooks, the estimated noise A codebook selector for selecting a noise codebook having a noise codevector most similar to a spectral envelope value of a call, and setting any N of the spectral envelope values of the estimated noise signal as an adaptive codevector, A control unit for generating a super codebook by storing an adaptive code vector in the selected noise codebook, and a spectral envelope of the input speech signal and an integrated codevector combining the noise codevectors in the super codebook and the clean codevectors in the clean codebook. And a speech synthesizer for extracting the corrected speech signal using the value.

As described above, according to the present invention, an adaptive code vector is generated by extracting a linear predictive coefficient from the estimated noise using a long-term noise estimation technique such as MS or Improved Minima Controlled Recursive Averaging (IMCRA). In addition, by combining the generated adaptive codevector with the existing noise codebook to generate a super codebook, the performance can be guaranteed for the learned noise and the untrained noise, in all cases. In particular, by using the gain value of the clean voice signal and the noise value obtained through the voice absence probability (SAP) technique, the voice signal with improved performance may be output.

1 is a diagram illustrating a configuration of an apparatus for improving speech recognition according to an embodiment of the present invention.
2 is a configuration diagram of a codebook database according to FIG. 1.
3 is a flowchart illustrating a method for improving speech recognition according to an embodiment of the present invention.
4 is a diagram illustrating a process of generating a super codebook according to an embodiment of the present invention.

The details of other embodiments are included in the detailed description and drawings.

Advantages, features, and methods of achieving them will be apparent with reference to the embodiments described below in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

1 is a diagram illustrating a configuration of an apparatus for improving speech recognition according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a codebook database according to FIG. 1.

Referring to FIG. 1, the apparatus 100 for improving speech recognition includes a speech signal input unit 110, a Fourier transform unit 120, an input speech linear predictive coefficient extractor 130, a noise signal linear predictive coefficient extractor 140, The codebook database 150 includes a codebook selector 160, a controller 170, and a speech synthesizer 180.

First, the voice signal input unit 110 receives an input voice signal y (n) in units of frames. The input voice signal is a contaminated voice signal, and represents a voice signal in which a clean voice signal and a noise signal are mixed.

The Fourier transform unit 120 generates an input speech power spectrum by performing Fourier Transform (FFT) on the input speech signal from the time domain signal to the frequency domain signal.

The input speech linear prediction coefficient extractor 130 obtains a spectral envelope value | Y _LPC (k) | ² of the input speech signal. Here, the input speech linear prediction coefficient extractor 130 performs an inverse fast fourier transform (IFFT) on the input speech power spectrum (| Y (k) | ² ) to generate a magnetic field of the input speech signal y (n). correlation value (R _y (n)) the generated and then, the input audio signal (y (n)) autocorrelation values (R _y (n)) the Levinson algorithm (Levinson algorithm) the input speech signal (y applied to the ( The linear predictive coefficient (a _y ) for n)) is extracted. Next, the input speech linear prediction coefficient extractor 130 speculates the input speech signal y (n) using a frequency response value of the linear prediction coefficient a _{y with} respect to the input speech signal y (n). To obtain the parallel envelope value (| Y _LPC (k) | ² ). Here, the spectral envelope value | Y _LPC (k) | ² of the input speech signal y (n) is the frequency response value Y _LPC (k of the linear predictive coefficient with respect to the input speech signal y (n). The square of the magnitude of)).

The noise signal linear prediction coefficient extractor 140 estimates a noise signal from the input speech signal y (n), and estimates the spectral envelope value (| D _LPC (k) |) of the estimated noise signal d (n). ² ) That is, the noise signal linear prediction coefficient extractor 140 extracts the noise signal from the input speech signal y (n) using a long term noise estimation algorithm such as Improved Minima Controlled Recursive Averaging (IMCRA) or Minimum Statistic (MS). Estimate. Then, one noise codebook is selected from a plurality of noise codebooks using the noise type of the estimated noise signal, and an inverse Fourier transform of the noise power spectrum (| D (k) | ² ) for the estimated noise signal is performed. IFFT produces an autocorrelation value R _d (n) of the estimated noise signal d (n). The linear predictive coefficient a of the estimated noise signal d (n) is applied by applying the autocorrelation value R _d (n) of the estimated noise signal d (n) to the Levinson algorithm. _d ) The noise signal linear prediction coefficient extractor 140 speculates the noise signal d (n) estimated using a frequency response value of the linear prediction coefficient a _{d with} respect to the estimated noise signal d (n). To obtain the envelop envelope value (| D _LPC (k) | ² ).

The codebook database 150 includes a clean voice codebook 151 and a plurality of noise codebooks 152a, 152b, 152c, 152d, and 152e as shown in FIG. The clean speech codebook 151 includes a plurality of clean codevectors, and the plurality of noise codebooks 152 have different noise types, and each noise codebook 152 includes a plurality of noise codevectors. . In the embodiment of the present invention, for convenience of description, the codebook database 150 includes one clean voice codebook 151 and five noise codebooks 152, and five noise codebooks 152a, 152b, 152c, 152d, and 152e. Are assumed to include White, Babble, F16, Factory2, and Pink, respectively, depending on the type of noise. In addition, the clean speech codebook 151 is composed of 1024 clean code vectors, and the noise codebooks 152a, 152b, 152c, 152d, and 152e are codebooks for the learned noise. codevector).

Next, the codebook selector 160 has a noise code vector most similar to the spectral envelope value | D _LPC (k) | ^{2 of the} estimated noise signal d (n) among the plurality of noise codebooks. Select the noise codebook.

The controller 170 sets any N of the spectral envelope values | D _LPC (k) | ² of the estimated noise signal d (n) as an adaptive codevector. The N adaptive codevectors are stored in the selected noise codebook to generate a super codebook.

Also, the speech synthesis unit 180 may combine the noise code vectors in the super codebook and the clean code vectors in the clean codebook, and the spectral envelope value (| Y _LPC (k) of the input speech signal y (n). ² ) Extract the corrected speech signal with improved speech recognition.

Hereinafter, a method of improving speech recognition according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 and 4. 3 is a flowchart illustrating a method for improving speech recognition according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating a process of generating a super codebook according to an embodiment of the present invention.

As shown in FIG. 3, first, the voice signal input unit 110 receives an input voice signal y (n) from the outside in units of frames (S310). Here, the input voice signal y (n) is a mixed signal of the clean voice x (n) and the noise signal d (n). The voice signal y (n) contaminated with noise is It may be represented as in Equation 1.

Where x (n) is a clean speech signal and d (n) is a noise signal.

When the input voice signal y (n) is received from the outside, the Fourier transform unit 120 performs Fourier transform (FFT) on the input voice signal y (n) in the time domain to perform the voice signal Y (k) in the frequency domain. )) And generate a voice power spectrum (Y (k) | ² ) (S320).

The input speech linear prediction coefficient extractor 130 performs inverse fast Fourier transform (IFFT) on the voice power spectrum (| Y (k) | ² ) to autocorrelate the input speech signal (y (n)). R _y (n) corresponding to the value is generated (S330).

The input speech linear prediction coefficient extractor 130 applies R _y (n) to the Levinson algorithm to extract the linear prediction coefficient a _y of the input speech signal y (n) (S340). . A method of extracting the linear predictive coefficient a _y from the input speech signal y (n) using the Levinson algorithm can be easily implemented by those skilled in the art, and thus a detailed description thereof will be omitted. When the linear predictive coefficient a _y for the input speech signal is extracted as described above, the input speech linear predictive coefficient extractor 130 inputs the linear predictive coefficient a _y of the input speech signal y (n). The spectral envelope value | Y _LPC (k) | ^{2 of the} speech signal y (n) is obtained (S350).

Meanwhile, the noise signal linear prediction coefficient extractor 140 uses a long-term noise estimation algorithm such as Improved Minima Controlled Recursive Averaging (IMCRA) or Minimum Statistic (MS) to obtain a speech power spectrum (| Y (k) | ² ). The noise signal d (n) included in the input voice signal y (n) is estimated from S360. Here, the noise codebook most similar to the noise type of the estimated noise signal d (n) is selected from the plurality of noise codebooks.

The noise signal linear prediction coefficient extractor 140 generates a noise power spectrum | D (k) | ² from the estimated noise signal d (n), and estimates the noise power spectrum | D (k) | ² ) an inverse Fourier transform (IFFT) to generate an autocorrelation value R _d (n) of the estimated noise signal d (n) (S370).

The noise signal linear prediction coefficient extractor 140 applies the autocorrelation value R _d (n) of the estimated noise signal _d (n) to the Levinson algorithm, thereby estimating the estimated noise signal d ( The linear predictive coefficient a _d of n)) is extracted (S380). The noise signal linear prediction coefficient extracting unit 140 may use the spectral envelope value (|) of the noise signal d (n) estimated using the linear prediction coefficient a _{d of} the estimated noise signal d (n). D _LPC (k) | ² ) is calculated (S390).

P-order linear prediction coefficients of the estimated noise signal d (n) a _d = [1, a _d (1),... , a _d (P)] ^T , and may be expressed in a spectral form in the frequency domain as shown in Equation 2 below.

After extracting the linear predictive coefficient a _d for the estimated noise signal d (n), the noise signal linear predictive coefficient extractor 140 extracts the spectral envelope of the estimated noise signal d (n). _{Find the} value (| D _LPC (k) | ² ).

Next, the codebook selector 160 performs a spectral envelope value of the estimated noise signal d (n) among the plurality of noise codebooks 152a, 152b, 152c, 152d, and 152e stored in the codebook database 150. A noise codebook having a noise code vector most similar to (| D _LPC (k) | ² ) is selected (S400).

Here, the codebook selector 140 selects a codebook type of the noise signal using an ISakura-Saito Distortion (IS-D) algorithm. The codebook type of the noise signal using the IS-D is selected from Equation 3 below. Is made by

는 추정된 잡음신호, N_d는 코드북 데이터베이스 내 잡음 코드북의 개수, v_n은 n번째 코드북 내 코드벡터의 개수이고, a_d ^n,j은 n번째 잡음 코드북의 j번째 코드벡터의 선형 예측 계수이다. 따라서, 도 2에서 예를 든 것에 따르면, N_d는 5이고 v_n은 8에 해당한다. Where n 'is the selected noise codebook,

Is the estimated noise signal, N _d is the number of noise codebooks in the codebook database, v _n is the number of codevectors in the nth codebook, and a _d ^{n, j} is the linear prediction coefficient of the jth codevector of the nth noise codebook . Thus, according to the example in FIG. 2, N _d is 5 and v _n corresponds to 8.

The controller 170 sets any N of the spectral envelope values | D _LPC (k) | ² of the estimated noise signal d (n) as an adaptive codevector. In operation S410, the N adaptive codevectors are stored in the selected noise codebook n '.

Here, the number N of the spectral envelope values | D _LPC (k) | ^{2 of the} estimated noise signal d (n) may be arbitrarily set, and in the embodiment of the present invention, N = Assume four. In addition, assuming that the selected noise codebook n 'is a white type noise codebook for convenience of description, the white type noise codebook 152a has four adaptations to the existing eight fixed code vectors as shown in FIG. An adaptive code vector is added. Accordingly, the white type noise codebook 152a is changed to a super codebook including 12 noise code vectors.

Next, the speech synthesis unit 170 combines the noise code vectors in the super codebook and the clean code vectors in the clean codebook 151 and the spectral envelope values (|) of the input speech signal y (n). The audio signal x '' (n) corrected using the Y _LPC (k) | ² ) is output (S420).

In more detail, first, the speech synthesis unit 170 includes 12 noise code vectors included in the super codebook generated in operation S410 (that is, the white type noise codebook 152a) and 1024 clean codes included in the clean codebook. Combine the codevectors to create an integrated codevector. That is, since the number of clean code vectors stored in the clean codebook is 1024 and the number of noise code vectors stored in the selected super codebook is 12, the total number of integrated code vectors in which the clean code vector and the noise code vector are combined is 12288 in total. Dogs are possible.

The speech synthesis unit 170 measures the similarity between 12288 integrated code vectors and the spectral envelope value | Y _LPC (k) | ² of the input speech signal y (n) estimated in step S320. The measurement is performed using D to estimate a clean speech signal and a noise signal included in the input speech signal.

The process of obtaining the similarity is described in more detail. The gain of the clean voice signal x (n) of the input voice signal y (n) is referred to as c _g and the gain of the noise signal d (n) is defined as n _g. In this case, gain combination values A ₁ to A ₁₂₂₈₈ can be obtained for ₁₂₂₈₈ integrated code vectors as shown in Equation 4 below.

As shown in Equation 4, there are 12288 gains between 1024 code vectors C_①, C_②, ... of the clean codebook and 12 codevectors n_①, n_②, ..., n_⑪, n_⑫ of the noise codebook (ie, super codebook). Combination values A ₁ to A ₁₂₂₈₈ are generated. The gain value (n _g) of the gain value (C _g) and a noise signal (d (n)) of the clean speech signal (x (n)) in the equation (4) are however the same expression, respectively, may have different values, and Depending on the combination, the gain values C _g and n _g may be calculated differently.

Then, the speech synthesis unit 170 performs a spectral envelope value (| Y _LPC (k) | ² ) and a gain combination value (A ₁ to A ₁₂₂₈₈ ) of the input speech signal y (n) corresponding to one frame. Each similarity (L ₁ to L ₁₂₂₈₈ ) between is obtained using IS-D. Using the IS-D to obtain the similarity (L, Likelyhood) between the spectral envelope value (| Y _LPC (k) | ² ) and the gain combination value of the input speech signal (y (n)) is readily available to those skilled in the art. Detailed description thereof will be omitted.

Then, the speech synthesizer 170 separates the clean speech signal and the noise signal from the input speech signal through a maximum likelihood (ML) technique or a Bayesian Minimum Mean Squared Error Estimation (MMSE) technique.

In more detail, first, the speech synthesis unit 170 obtains the spectral envelope value | Y _LPC (k) | ^{2 of the} input speech signal y (n) as shown in Equation 5 below.

Since A ₁ to A ₁₂₂₈₈ shown in Equation 5 are gain combination values obtained in Equation 4, it can be expressed as Equation 6 below by substituting this into Equation 5.

If the spectral envelope value of the input speech signal y (n) obtained from Equation 6 is divided into the spectral envelope value (X ') of the clean speech signal and the spectral envelope value (d') of the estimated noise signal, It can be expressed as Equation 7.

Then, the speech synthesis unit 170 includes a spectral envelope value X 'of the clean speech signal estimated as shown in Equation 8 and a spectral envelope value d' of the noise signal. By multiplying the input voice power spectrum y for the input voice signal by X '/ (X' + d '), the estimated noise signal portion is removed to estimate the corrected voice power spectrum X " Here, the corrected speech power spectrum (X ") is a state in which the ratio of the clean speech signal is increased by removing the estimated noise signal from the input speech signal, and represents the power spectrum of the speech signal with improved speech quality. .

Here, y represents an input speech power spectrum (| Y (k) | ² ) for the input speech signal, x represents a clean speech power spectrum, and d represents a noise power spectrum. The speech synthesizer 170 outputs a speech signal having improved speech quality by performing an inverse Fourier transform (IFFT) on the corrected speech power spectrum X ″.

After the step S420 is completed, if the input voice signal of the next frame is input to the voice signal input unit 110, the control unit 170 outputs a voice signal having improved voice quality by repeating steps S310 to S420. Update the adaptive code vector.

That is, when a new adaptive codevector is generated through steps S310 to S400, the controller 170 may include at least one adaptive code among N adaptive codevectors previously stored in the super codebook 151. The vector is removed and the newly generated adaptive codevector is stored in the super codebook 151. Here, among the N adaptive codevectors stored in advance, the earliest storage time, that is, the oldest adaptive codevector is removed.

As described above, according to the embodiment of the present invention, the adaptive code vector is updated for each frame, so that the corrected speech signal having the improved speech quality can be effectively extracted even for the input speech including the unlearned noise.

On the other hand, according to an embodiment of the present invention, using the speech absorptivity probability (SAP) method in Equation 4, the gain value (c _g ) of the clean speech signal and the gain value (n _g ) of the noise signal More accurate estimates can be made. The noise estimation model using the speech absence probability (SAP) has a method of calculating and adding each gain value separately for each section by dividing it into a section in which a speech is present and a section in which no speech is present.

First, the noise signal linear prediction coefficient extractor 140 estimates the noise signal d (n) by using a long-term noise estimation method in step S360, while the speech absence probability value SAP and the speech existence probability value 1 -SAP). Then, the speech synthesis unit 170 applies the calculated speech absence probability value (SAP) and the speech existence probability value (1-SAP) to the following equation (9), the gain value (c _g ) and the noise signal of the clean speech signal: Estimate the gain value of n _g .

Here, Y (w) represents the frequency response value of the input speech signal, and A ⁿ _d (w) represents the nth code vector stored in the super codebook.

Here, the gain value in the voice presence section may be represented by the inverse matrix of E and the matrix product of F, and E and F are as follows.

이며, A^m _x(w)은 클린 코드북에 저장된 m번째 코드벡터를 나타낸다. here,

A ^m _x (w) represents the m th code vector stored in the clean codebook.

Therefore, according to an embodiment of the present invention, the gain value c _g of the clean speech signal and the gain value n _g of the noise signal can be estimated more accurately using the speech absorptivity probability (SAP) technique. In addition, since the voice synthesizer 170 may output a clean voice signal in operation S420, the voice signal may be further extracted.

Table 1 below shows the results of experiments evaluating the performance of the noise estimation method according to an embodiment of the present invention.

In this experimental example, the conventional IMCRA method, the existing codebook-based speech enhancement method (CBSE), and the super according to the embodiment of the present invention are applied to the internal noise, which is the learned noise, and the external noise, which are not learned. Speech enhancement method (CBSE (S)) using a codebook, codebook based speech enhancement method (SAP-CBSE) using an existing speech absence probability (SAP) technique, and a speech absence probability (SAP) technique according to an embodiment of the present invention. Log-spectral distortion (LSD) experiments were carried out through five methods of speech enhancement (SAP-CBSE (S)) using super codebook.

In particular, the internal noise, the learned noise, is noise for white, babble, F16, factory2, and pink. The noise codebook is stored for the five noises. Includes noise for Buccaneer1, car, restaurant, factory1, and street. In this example, the speech signal codebook was trained using 4620 sentences of the TIMIT database sampled at 8000 Hz, and the test was performed with 1680 sentences. In the case of noise, we experimented with 10 kinds of noise, and experimented under the environment of white, babble, F16, factory2, and pink as existing noise codebooks.

According to the experimental results, the speech enhancement method (CBSE (S)) using the super codebook according to the embodiment of the present invention shows better noise estimation performance than the conventional IMCRA method and the conventional codebook based speech enhancement method (CBSE). In the case of the speech absence probability (SAP) technique and the speech enhancement method using the super codebook (SAP-CBSE (S)) according to an embodiment of the present invention, the codebook-based speech enhancement method (SAP-CBSE) using SAP is used. The noise estimation performance is better than that.

In particular, in the case of speech enhancement methods (CBSE (S) and SAP-CBSE (S)) using the super codebook according to an embodiment of the present invention, in the internal noise which is the learned noise, the conventional IMCRA method and the codebook-based speech enhancement method Although it has similar noise estimation performance to those of CBSE and SAP-CBSE, it has been shown to have excellent noise estimation performance for the outside noise.

As described above, according to an embodiment of the present invention, an adaptive code vector is generated by extracting a linear predictive coefficient from noise estimated by using a long-term noise estimation technique such as MS or IMCRA, and generating an adaptive code vector. By combining type code vectors with existing noise codebooks, we can guarantee the performance of the learned and unlearned noises in all cases. In particular, by using the gain value c _g of the clean voice signal and the gain value n _g of the noise signal obtained through the voice absence probability (SAP) technique, a more improved voice signal may be output.

Although the present invention has been described with reference to the embodiments illustrated in the accompanying drawings, it is merely an example, and a person having ordinary knowledge in the art related to the present invention may have various modifications or equivalents thereto. I will understand that it is possible.

Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (22)

Receiving an input voice signal mixed with a clean voice signal and a noise signal in units of frames,
Generating an input speech power spectrum by Fourier transforming the input speech signal,
Obtaining a spectral envelope value of the input speech signal;
Estimating the noise signal from the input speech power spectrum and obtaining a spectral envelope value of the estimated noise signal,
Selecting a noise codebook having a noise code vector most similar to a spectral envelope value of the estimated noise signal among a plurality of previously stored noise codebooks,
Setting any N of the spectral envelope values of the estimated noise signal as an adaptive codevector, and storing the N adaptive codevectors in the selected noise codebook to generate a super codebook; and
Extracting a corrected speech signal using an integrated codevector combining the noise codevectors in the super codebook and the clean codevectors in the clean codebook and the spectral envelope value of the input speech signal. Codebook based speech enhancement method. The method of claim 1,
Obtaining a spectral envelope value of the input speech signal,
Generating an autocorrelation value of the input speech signal by inverse Fourier transforming the input speech power spectrum,
Extracting a linear predictive coefficient for the input speech signal by applying an autocorrelation value of the input speech signal to a Levinson algorithm, and
And obtaining a spectral envelope value of the input speech signal using a frequency response value of the linear predictive coefficient with respect to the input speech signal. The method of claim 2,
Estimating a noise signal from the input speech signal, and obtaining a spectral envelope value of the estimated noise signal,
Estimating the noise signal from the input speech power spectrum using a long-term noise estimation algorithm,
Selecting one noise codebook among the plurality of previously stored noise codebooks using the noise type of the estimated noise signal,
Inverse Fourier transforming a noise power spectrum for the estimated noise signal to generate an autocorrelation value of the estimated noise signal,
Extracting a linear predictive coefficient for the estimated noise signal by applying an autocorrelation value of the estimated noise signal to a Levinson algorithm, and
And obtaining a spectral envelope value of the input speech signal using the frequency response value of the linear predictive coefficient for the estimated noise signal. The method of claim 3,
Selecting a noise codebook having a noise code vector most similar to the spectral envelope value of the estimated noise signal,
Codebook-based speech enhancement method using an adaptive codevector that selects the noise codebook using an Itakura-Saito Distortion (IS-D) algorithm such as:

Where n 'is the selected noise codebook,

Is the estimated noise signal, N _d is the number of noise codebooks in the codebook database, v _n is the number of codevectors in the nth codebook, and a _d ^{n, j} is the linear prediction coefficient of the jth codevector of the nth noise codebook . 5. The method of claim 4,
Generating the super codebook,
Setting any N of the spectral envelope values of the estimated noise signal as the adaptive code vector and storing the selected N codebook in the selected noise codebook; and
And integrating the noise codevectors previously stored in the selected noise codebook with the N adaptive codevectors to change the selected noise codebook into the super codebook. The method of claim 5,
Extracting the corrected voice signal,
Generating the integrated code vector by combining a noise code vector included in the super codebook and a clean code vector included in the clean codebook;
Generating gain combination values by applying a gain value of a clean speech signal and a gain value of the estimated noise signal with respect to the integrated code vector;
Obtaining respective similarities between the spectral envelope values of the input speech signal and the gain combination values, and
An adaptive code including applying the similarity and gain combination value to a maximum likelihood (ML) or a Bayesian estimation algorithm (MMSE) to obtain a spectral envelope value of the clean speech signal and a spectral envelope value of the noise signal Codebook based speech enhancement method using vectors. The method according to claim 6,
Generating the corrected voice signal,
Generating a corrected speech power spectrum from a spectral envelope value of the clean speech signal and a spectral envelope value of the noise signal through a Wiener filter, and
And inversely Fourier transforming the corrected speech power spectrum to extract the corrected speech signal. The method according to claim 6,
The gain value of the clean speech signal and the gain value of the estimated noise signal are obtained using an adaptive code vector algorithm using an adaptive code vector probability. 9. The method of claim 8,
A codebook-based speech enhancement method using an adaptive code vector, wherein the gain value c _g of the clean speech signal and the gain value n _g of the estimated noise signal are calculated by the following equation:

here,

Where Y (w) represents the frequency response of the input speech signal, A ⁿ _d (w) represents the nth codevector stored in the super codebook, and A ^m _x (w) represents the mth codevector stored in the clean codebook Indicates. The method of claim 1,
Estimating a noise signal from an input speech signal of a next frame input following the input speech signal, and obtaining a spectral envelope value of the estimated noise signal;
Setting the spectral envelope value of the estimated noise signal to a new adaptive codevector, and
Removing at least one adaptive codevector from among the N adaptive codevectors stored in the supercodebook, and storing the new adaptive codevector in the supercodebook. How to improve your voice. The method of claim 10,
Codebook-based speech enhancement method using an adaptive codevector to remove the adaptive codevector having the earliest stored time among the N adaptive codevectors stored in the super codebook. A voice signal input unit for receiving an input voice signal mixed with a clean voice signal and a noise signal in units of frames,
A Fourier transform unit for Fourier transforming the input speech signal to generate an input speech power spectrum;
An input speech linear prediction coefficient extraction unit for obtaining a spectral envelope value of the input speech signal,
A noise signal linear prediction coefficient extractor for estimating the noise signal from the input speech power spectrum and obtaining a spectral envelope value of the estimated noise signal;
A codebook database comprising a clean speech codebook having a plurality of clean codevectors and a plurality of noise codebooks having a plurality of noise codevectors and having different types,
A codebook selector for selecting a noise codebook having a noise code vector most similar to a spectral envelope value of the estimated noise signal among the plurality of noise codebooks;
A controller configured to set any N of the spectral envelope values of the estimated noise signal as an adaptive codevector, and store the N adaptive codevectors in the selected noise codebook to generate a super codebook; and
An adaptive code vector including an integrated code vector that combines noise code vectors in the super codebook and clean code vectors in a clean codebook, and a speech synthesizer for extracting a corrected speech signal using a spectral envelope value of the input speech signal. Codebook-based speech enhancement device using a. The method of claim 12,
The input speech linear prediction coefficient extracting unit,
Generating an autocorrelation value of the input speech signal by inverse Fourier transforming the input speech power spectrum,
Applying the autocorrelation value of the input speech signal to the Levinson algorithm to extract a linear predictive coefficient for the input speech signal,
And a codebook-based speech enhancement apparatus using an adaptive code vector to obtain a spectral envelope value of the input speech signal using a frequency response value of the linear predictive coefficient with respect to the input speech signal. The method of claim 13,
The noise signal linear prediction coefficient extractor,
Estimating the noise signal from the input speech power spectrum using a long-term noise estimation algorithm, selecting one noise codebook among the plurality of noise codebooks, using the noise type of the estimated noise signal, Inverse Fourier transform the noise power spectrum for the estimated noise signal to produce an autocorrelation of the estimated noise signal,
Applying the autocorrelation value of the estimated noise signal to a Levinson algorithm, extracting a linear predictive coefficient for the estimated noise signal, and using the estimated frequency response value of the linear predictive coefficient for the estimated noise signal Codebook-based Speech Enhancement Device Using Adaptive Codevector to Obtain Spectral Envelope of Noise Signal. 15. The method of claim 14,
The codebook selection unit,
Codebook-based speech enhancement device using an adaptive codevector that selects the noise codebook using an Itakura-Saito Distortion (IS-D) algorithm such as:

Where n 'is the selected noise codebook,

Is the estimated noise signal, N _d is the number of noise codebooks in the codebook database, v _n is the number of codevectors in the nth codebook, and a _d ^{n, j} is the linear prediction coefficient of the jth codevector of the nth noise codebook . 16. The method of claim 15,
The control unit,
N arbitrary spectral envelope values of the estimated noise signal are set as the adaptive codevector and stored in the selected noise codebook,
And a codebook-based speech enhancement apparatus using an adaptive codevector integrating the noise codevector previously stored in the selected noise codebook and the N adaptive codevectors into the super codebook. 17. The method of claim 16,
The speech synthesis unit,
The integrated code vector is generated by combining the noise code vector included in the super codebook and the clean code vector included in the clean codebook, and a gain value of a clean speech signal and a gain of the estimated noise signal are generated with respect to the integrated code vector. Apply each value to produce a gain combination,
After obtaining respective similarities between the spectral envelope values of the input speech signal and the gain combination values, the similarity and gain combination values are applied to a maximum likelihood (ML) or a Bayesian estimation algorithm (MMSE) to perform the clean speech signal. Codebook-based speech enhancement apparatus using an adaptive codevector to obtain a spectral envelope of and a spectral envelope of the noise signal. 18. The method of claim 17,
The speech synthesis unit,
A Wiener filter generates a corrected speech power spectrum from the spectral envelope value of the clean speech signal and the spectral envelope value of the noise signal, and inversely transforms the corrected speech power spectrum into the corrected speech power spectrum. Codebook-based speech enhancement device using adaptive codevectors that extract speech signals. 18. The method of claim 17,
And a gain value of the clean speech signal and a gain value of the estimated noise signal using an adaptive code vector obtained by using a speech absent probability (SAP) algorithm. 20. The method of claim 19,
A codebook-based speech enhancement apparatus using an adaptive code vector, wherein the gain value c _g of the clean speech signal and the gain value n _g of the estimated noise signal are calculated by the following equation:

here,

Where Y (w) represents the frequency response of the input speech signal, A ⁿ _d (w) represents the nth codevector stored in the super codebook, and A ^m _x (w) represents the mth codevector stored in the clean codebook Indicates. The method of claim 12,
When the input voice signal of the next frame input following the input voice signal is input,
The Fourier transform unit performs Fourier transform on the input speech signal of the next frame to generate an input speech power spectrum,
The noise signal linear prediction coefficient extractor estimates a noise signal from the input speech signal, obtains a spectral envelope value of the estimated noise signal,
The control unit,
Set a spectral envelope value of the estimated noise signal as a new adaptive codevector, remove at least one adaptive codevector from the N adaptive codevectors stored in the super codebook, and remove the new adaptive codevector Codebook-based speech enhancement apparatus using an adaptive codevector to store in the super codebook. The method of claim 21,
The control unit,
Codebook-based speech enhancement apparatus using an adaptive codevector to remove an adaptive codevector having the earliest stored time among the N adaptive codevectors stored in the super codebook.

Images