KR100587568B1 - Speech enhancement system and method - Google Patents

Abstract

The present invention relates to a voice enhancement system and method that can enhance a voice signal component while minimizing the generation of musical noise during noise suppression processing. After dividing an input voice signal into frame units, the frame has a noise component only. In the case of a frame, after the noise suppression process is performed on the real part and the imaginary part of the noise frame, the voice enhancement process is performed on the real part and the imaginary part of the frame, respectively.

Noise, suppression, voice, enhancement, musical, noise

Description

Speech Enhancement System and Method {SPEECH ENHANCEMENT SYSTEM AND METHOD}

1 is a view for explaining a conventional voice enhancement method.

2 is a block diagram of a speech enhancement system according to the present invention.

3 is a detailed block diagram of the noise average calculator illustrated in FIG. 2.

4 is a flowchart of a voice enhancement method according to the present invention.

5 is a view showing an example in which the negative component is enhanced by the present invention.

Explanation of symbols on the main parts of the drawings

10 Pre-processing unit 20 High-speed Fourier transform unit

30 ... noise suppression unit 40 ... voice detection unit

50 ... noise average calculation unit 51 ... first noise average calculation unit

52 ... second noise average calculation unit 53 ... Hilbert transform unit

54 ... envelope size detector 55 ... Mean calculator

60 ... noise weight calculator 70 ... multiplier

80 ... Voice reinforcement unit 90 ... Voice weight calculation unit

100 ... multiplier 110 ... inverse fast Fourier transform

120.Overlap

The present invention relates to a speech enhancement system and method, and more particularly, to a speech enhancement system and method capable of enhancing speech signal components while minimizing the generation of musical noise during noise suppression processing.

A common problem in speech signal processing is to suppress background noise components and to improve speech signal components. A typical speech enhancement method is to perform a Fourier transform on a noisy input signal as shown in FIG. Spectral Subtraction Method (S) (Subtraction Filter) is used to subtract the noise spectrum from the Fourier transformed signal spectrum to suppress background noise.

However, when the noise suppression process is performed by the spectral subtraction method as described above, since the actual noise spectrum cannot be estimated accurately, when the noise spectrum estimated from the original speech signal spectrum is subtracted, the value of the speech signal has a value of 0 or less. In this case, the spectral subtraction method forcibly assigns a section having a value of 0 or less to a small value, which causes musical noise, which is a characteristic frequency noise, to make the voice after noise suppression extremely unnatural. There is a problem that can be confusing.

In order to solve these problems, US Patent No. 5,742,927 (published: August 8, 1998) suppresses noise by using a spectral subtraction method, and then emphasizes only a formant of speech through a linear prediction coefficient (LPC) spectrum estimator. Although the method is disclosed, the voice enhancement method also has a limitation in that it cannot solve the occurrence of musical noise.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to enhance the sound signal component while suppressing the musical noise as indicated as a major problem in the noise suppression process.

In order to achieve the above object, the voice enhancement system includes a preprocessor that divides an input voice signal into frame units, and a real part of the noise frame when the frame input from the preprocessor is a noise frame having only noise components. And a voice reinforcement unit for performing noise suppression on the imaginary unit, and a voice reinforcement unit performing voice reinforcement on the real part and the imaginary part of the frame inputted from the noise suppression unit.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram of a speech enhancement system according to the present invention. As shown in FIG. 2, the speech enhancement system 1 according to the present invention includes a preprocessor 10, a fast Fourier transform unit 20, and noise suppression. The unit 30 includes a voice reinforcement unit 80, an inverse fast Fourier transform unit 110, and an overlap unit 120.

The preprocessing unit 10 samples the noisy input signal at a predetermined frequency and converts the signal into a digital signal, preemphasis the digital signal with a simple high pass filter to emphasize the high frequency of the signal slightly, and then filter the input signal. The signal is divided into frames, which are the basic units of speech processing.

A Fast Fourier Transform (FFT) 20 applies a window to a frame input from the preprocessor 10 and then performs an N-point Fast Fourier Transform, where the N-point FFT is The equation is given by Equation 1 below.

As described above, the fast Fourier transformed signal is represented by a complex number. In this case, the spectrum of the fast Fourier transformed frame is represented by a real part and an imaginary part as shown in Equation 2 below.

Meanwhile, according to the spectral subtraction method as described above, musical noise may be generated during the noise suppression process. In order to prevent such a phenomenon, the present invention includes the noise suppression unit 30 in the noise frame as follows. Appropriate noise suppression processing is performed according to the characteristics of the noise component. Hereinafter, the noise suppression unit 30 will be described in more detail.

Referring to FIG. 2, the noise suppression unit 30 includes a voice detector 40 and a fast Fourier transform unit 20 that determine whether or not a voice exists in a frame inputted from the fast Fourier transform unit 20. When the input frame is determined to be a noise frame having only a noise component, the noise average calculator 50 and the noise average calculator 50 calculate an average spectrum of the real part and the imaginary part of the noise frame. A noise weight calculator 60 for calculating weights of the real and imaginary parts of the noise frame according to the spectral value, and a noise weight calculator of the real and imaginary parts of the frame inputted from the fast Fourier transform unit 20. And a multiplier 70 that multiplies each weight calculated at 60.

The voice detector 40 determines whether voice is present in the frame inputted from the fast Fourier transform unit 20, and if the input frame is determined to be a noise frame having only a noise component, the voice frame calculates the noise frame as a noise average calculator ( 50), if the voice exists in the input frame, the frame is not output to the noise average calculation unit 50.

Here, the voice detector 40 may be implemented as a voice activity detector (VAD) that calculates the energy of the input frame to find the most suitable threshold value and determines the presence of voice based on this. It is also possible to determine the presence of voice.

The noise average calculator 50 separately obtains an average spectrum of the real part and the imaginary part from the noise frame input from the voice detector 40, and with respect to the noise average calculator 50 with reference to FIG. 3. Explain in more detail.

3 is a detailed configuration diagram of the noise average calculation unit 50 shown in FIG. 2, and as shown in FIG. 3, the noise average calculation unit 50 is a real number in a noise frame input from the voice detector 40. A first noise average calculation unit 51 for calculating a negative average and a second noise average calculation unit 52 for calculating an average of an imaginary part in a noise frame input from the voice detector 40, and a first noise The average calculator 51 and the second noise average calculator 52 include a Hilbert transformer 53, an envelope magnitude calculator 54, and a mean calculator 55.

The Hilbert transform unit 53 is for generating a complex signal for the real part and the imaginary part from the noise frame input from the voice detector 40, respectively, and includes the real part and the imaginary part of the input noise frame. For each Hilbert transform, the Hilbert transform produces an analytically complex signal for the input signal, where the real part of the Hilbert transformed signal is equal to the input signal and the imaginary part is 90 degrees out of phase with the input signal. Since the Hilbert transform is a commonly used method of detecting the envelope of the signal, a detailed description of the Hilbert transform is omitted.

The envelope size calculation unit 54 calculates the magnitude of an envelope for the real part and the imaginary part of the noise frame by taking an absolute value of each complex signal obtained by the Hilbert transform unit 53, and calculating the mean. The unit 55 averages the magnitudes of the envelopes calculated by the envelope size calculator 54 and outputs them as average spectral values for the real part and the imaginary part of the noise frame. In order to increase the efficiency, the envelope size of several noise frames (about 100 msec) is preferably averaged.

As described above, the first noise average calculation unit 51 and the second noise average calculation unit 52 may perform the speech through the Hilbert transform unit 53, the envelope size calculation unit 54, and the mean calculation unit 55. The average spectrum of the real part and the imaginary part is calculated from the noise frame input from the detector 40, and the average spectrum values of the real part and the imaginary part of the noise frame are thus transferred to the noise weight calculation part 60. do.

Referring back to FIG. 2, the noise weight calculator 60 inversely transforms the average spectral value calculated by the noise average calculator 50 to apply weights to be applied to the real part and the imaginary part of the noise frame. In this case, the weight is a value for suppressing a noise component included in the noise frame, and if the average spectral value input from the noise average calculation unit 50 is large, that is, if the input noise frame has a large noise component, In addition, if the average spectral value is small, that is, if there are few noise components in the input noise frame, the weight is increased to suppress the noise components relatively small. To help.

The multiplier 70 multiplies the real part and the imaginary part of the frame inputted from the fast Fourier transform unit 20 by the weights of the real part and the imaginary part of the noise frame calculated by the noise weight calculation unit 70, respectively. .

As described above, the noise suppression unit 30 performs suppression processing at different ratios for the real part and the imaginary part of the noise frame having only the noise component so that the spectrum of the voice signal has a value of 0 or less during noise suppression. By doing so, it is possible to effectively suppress the noise component in the noise frame while suppressing the occurrence of musical noise as much as possible.

On the other hand, the frame input from the noise suppressor 30 to the voice enhancer 80 is subjected to a voice reinforcement process, the voice reinforcement unit 80 will be described in more detail below.

The voice reinforcement unit 80 calculates a voice weight on the frame input from the noise suppressor 30, and calculates a voice weight on the frame input from the noise suppressor 30. The multiplier 100 multiplies the weight calculated in the unit 100.

The voice weight calculator 90 calculates a weight for reinforcing a voice component in a frame input from the noise suppressor 30, and calculates a standard deviation of the real part and the imaginary part in the input frame. The weighted values of the real and imaginary parts of the frame are output. Here, the reason for using the standard deviation as the weight value for the speech component is that the standard deviation of the speech component is larger than that of the noise component. If set to a value, the speech component is relatively enhanced and the noise component of the high frequency band can be suppressed.

The multiplier 110 multiplies the real part and the imaginary part of the frame inputted from the noise suppressor 30 by the weights of the real part and the imaginary part of the frame calculated by the voice weight calculator 100, respectively. Enhances negative ingredients in

As described above, the voice enhancer 80 enhances the voice component at different ratios with respect to the real part and the imaginary part of the input frame, thereby reinforcing only the voice component while suppressing the noise component included in the frame. .

Meanwhile, an inverse fast Fourier transform (IFFT) 120 inversely transforms a frame input from the voice enhancer 80 and returns to a frame of a time domain, and the overlap unit 120 performs the The frames of the time domain output from the inverse fast Fourier transform unit 110 are overlapped to allow the front and rear frames to be seamlessly concatenated. Here, the equation for the inverse fast Fourier transform is given by Equation 3 below.

Accordingly, the speech enhancement system 1 according to the present invention performs appropriate noise suppression processing or speech enhancement processing according to the characteristics of the corresponding frame, thereby minimizing the generation of musical noise while enhancing the speech component and suppressing the noise component. do.

On the other hand, the speech enhancement method according to the present invention, the step of dividing the input speech signal in units of frames, if each frame is a noise frame having only a noise component, the step of suppressing noise for the real part and imaginary part of the noise frame And reinforcing the voice with respect to the real part and the imaginary part of each frame.

Hereinafter, a voice detection method according to the present invention will be described in detail with reference to the accompanying drawings.

4 is a flowchart of a voice enhancement method according to the present invention.

First, when a voice signal is input, the preprocessor 10 samples the input signal at a predetermined frequency and converts the signal into a digital signal. The digital signal is pre-emphasized by a simple high pass filter to slightly reduce the high frequency of the signal. After emphasizing, the filtered signal is divided into frames that are basic units of speech processing (S10).

Next, the fast Fourier transform unit 20 performs a N-point fast Fourier transform after applying a window to each frame input from the preprocessor 10 (S20). Since it has been described in detail in the description related to 2, a detailed description thereof will be omitted.

Next, the voice detector 40 determines whether voice is present in the frame input from the fast Fourier transformer 20 (S30), where the voice detector 40 calculates energy of the input frame. It is possible to determine the existence of the voice based on this to find the most suitable threshold value, it is also possible to determine the presence of the voice in other ways.

On the other hand, if it is determined by the voice detector 40 that the frame does not contain voice, that is, if the frame is determined to be a noise frame having only a noise component, the noise frame is output to the noise averaging calculator 50 and the noise There is a suppression step (S40), which will be described in more detail with respect to the noise suppression step (S40).

First, the noise average calculator 50 separately calculates an average spectrum of the real part and the imaginary part from the noise frame input from the voice detector 40 (S41). The calculation unit 50 obtains an interpretable complex signal for the real part and the imaginary part of the noise frame using the Hilbert transform, and then takes an absolute value of each complex signal to determine the envelope of the real part and the imaginary part of the noise frame. The magnitudes are obtained, and the magnitudes of the respective envelopes are averaged and output as average spectral values of the real part and the imaginary part of the noise frame.

Next, the noise weight calculation unit 60 calculates weights of the real part and the imaginary part of the noise frame by performing inverse transformation on the mean spectral values of the real part and the imaginary part of the noise frame (S42). Here, the weight value is a value for suppressing a noise component included in the noise frame. If the average spectral value of the noise frame is large, that is, if the input noise frame has many noise components, the weight value is reduced to reduce the noise component. If the average spectral value of the noise frame is small, that is, if the noise component is small in the input noise frame, the weight is increased to suppress the generation of musical noise as much as possible in the noise suppression process.

Next, the multiplier 70 weights the real part and the imaginary part of the noise frame calculated by the noise weight calculation unit 60 to the real part and the imaginary part of the frame inputted from the fast Fourier transform unit 20. By multiplying the values (S43), the noise components included in the frame are suppressed at different ratios according to the real part and the imaginary part.

That is, when the frame input from the fast Fourier transform unit 20 by the noise suppression step (S40) is a noise frame having only a noise component, an appropriate noise suppression process may be performed according to the characteristics of the noise component. This effectively suppresses the noise component and minimizes the occurrence of musical noise.

On the other hand, the frame output from the noise suppressor 30 is input to the voice enhancer 80 to go through the voice reinforcement step (S50), which will be described in more detail with respect to the voice reinforcement step (S50).

First, the speech weight calculation unit 90 calculates a standard deviation of the real part and the imaginary part of the input frame and sets the weight as the weights of the real part and the imaginary part of the frame (S51). The reason for using the standard deviation as the weighting factor is that since the standard deviation of the voice component is larger than that of the noise component, the standard deviation is set as the weight so that the voice component can be enhanced and the noise component of the high frequency band can be suppressed.

Next, the multiplier 100 multiplies the real parts and the imaginary parts of the frames inputted from the noise suppressor 30 by the weighted values calculated by the voice weight calculation unit 90 (S52). Voice components with a relatively low frequency band are enhanced and noise components with a relatively high frequency band are suppressed.

That is, by the voice reinforcing step (S50) by reinforcing the voice at different ratios with respect to the real part and the imaginary part of the frame, the voice can be enhanced while effectively suppressing noise components mixed in the voice frame.

On the other hand, the frame that has undergone the noise suppression step (S40) or the voice enhancement step (S50) as described above is inverse Fourier transformed by the inverse fast Fourier transform unit 110 is converted into a frame of the time domain again (S60).

Then, when noise suppression or speech enhancement is performed on all the frames of the voice signal, the overlap unit 120 overlaps the frames of the time domain output from the inverse fast Fourier transform unit 110 to seamlessly connect the front and rear frames and output them. (S70 ~ 80).

As described above, according to the present invention, by performing the appropriate noise suppression process or the speech enhancement process according to the characteristics of the frame, it is possible to minimize the generation of musical noise while enhancing the speech component and suppressing the noise component.

FIG. 5 is a diagram illustrating an example in which a speech component is enhanced by the present invention. When a real part and an imaginary part are separated from a voice signal including noise as shown in FIG. As shown in FIG. 5 (b), it can be seen that the noise component is suppressed and only the voice component is enhanced.

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Therefore, according to the present invention, it is possible to minimize the occurrence of musical noise while enhancing the speech component and suppressing the noise component, thereby improving the performance of speech recognition and speaker recognition system in a noisy environment. .

Claims (17)

A preprocessor dividing the input voice signal by frame unit; A fast Fourier transform unit for fast Fourier transforming the frame input from the preprocessor; A noise suppression unit for performing noise suppression on the real part and the imaginary part of the noise frame when the frame converted by the fast Fourier transform unit is a noise frame having only noise components; And And a voice reinforcement unit configured to perform voice reinforcement on the real part and the imaginary part of the frame inputted from the noise suppressor. The method of claim 1, wherein the noise suppression unit, A voice detector that determines whether voice exists in a frame input from the preprocessor; A noise average calculator for obtaining an average spectral value for the real part and the imaginary part in the noise frame when the input frame is determined to be a noise frame by the voice detector; A noise weight calculator configured to calculate weights for the real part and the imaginary part of the noise frame according to the average spectrum value calculated by the noise average calculator; And And a multiplier for multiplying the real part and the imaginary part of the frame inputted from the preprocessor by the weights of the real part and the imaginary part of the noise frame calculated by the noise weight calculator. The method of claim 2, wherein the noise average calculation unit, A first noise average calculator configured to calculate an average spectral value of a real part in the noise frame; And And a second noise average calculator for calculating an average spectral value of the imaginary part in the noise frame. The method of claim 3, wherein the first noise average calculation unit and the second noise average calculation unit, A Hilbert transform unit for obtaining a complex signal for the real part and the imaginary part in the noise frame input using the Hilbert transform; An envelope size calculator for calculating an envelope size for the real part and the imaginary part of the input noise frame according to each complex signal obtained by the Hilbert transform part; And And a mean calculator configured to calculate an average of envelope sizes of the real part and the imaginary part of the noise frame calculated by the envelope size calculator. The method of claim 1, wherein the voice enhancer, A speech weight calculator for calculating weights for speech components in the real part and the imaginary part of the frame inputted from the noise suppressor; And And a multiplier for multiplying the real part and the imaginary part of the frame input from the noise suppressor by the weights of the real part and the imaginary part of the frame calculated by the voice weight calculator. The method of claim 5, wherein the voice weight calculation unit, And a weight for the speech component according to the calculated standard deviation by calculating standard deviations of the real part and the imaginary part in the frame inputted from the noise suppressor. delete The voice enhancement system of claim 1, further comprising an inverse fast Fourier transform unit for performing an inverse fast Fourier transform on the frame input from the voice enhancer. The voice enhancement system of claim 8, further comprising an overlap unit configured to overlap the frames input from the inverse fast Fourier transform unit. Dividing the input voice signal by frame unit; Fast Fourier transforming the frame; Suppressing noise with respect to the real part and the imaginary part of the noise frame when the fast Fourier transformed frame is a noise frame having only a noise component; And And reinforcing the voice with respect to the real part and the imaginary part of each frame. The method of claim 10, wherein the noise suppression step, Determining whether voice is present in the frame; Obtaining an average spectral value for a real part and an imaginary part in the noise frame when the frame is determined to be a noise frame having only a noise component; Calculating weights for the real part and the imaginary part of the noise frame according to the calculated average spectrum value; And And multiplying the real part and the imaginary part of the frame by the weights of the calculated real part and the imaginary part of the noise frame, respectively. 12. The method of claim 11, wherein the obtaining the average spectral values for the real part and the imaginary part of the noise frame comprises: Obtaining a complex signal for a real part and an imaginary part in an input frame using a Hilbert transform; Calculating envelope sizes for the real part and the imaginary part of the input frame according to each complex signal obtained by the Hilbert transform; And And calculating an average of the calculated envelope size. The method of claim 10, wherein the voice reinforcing step, Calculating weights for speech components in the real part and the imaginary part of the frame, respectively; And multiplying the real part and the imaginary part of the frame by the weights of the calculated real part and the imaginary part of the frame, respectively. The method of claim 13, wherein the weight for the negative component, And a standard deviation of the real part and the imaginary part of the frame. delete 11. The method of claim 10, further comprising performing an inverse Fourier transform on the frame with enhanced speech. 17. The method of claim 16, further comprising overlapping the inverse fast Fourier transformed frames.

Images