JP2013178570A - Method and apparatus for removing noise from electronic signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and system for removing acoustic noise from human speech, in which noise is removed irrespective of the type, magnitude or orientation of noise to restore a signal after the removal.SOLUTION: A system includes a microphone and a sensor connected to a processor. The microphone receives acoustic signals and a VAD supplies a signal of a binary 1 when speech (both voiced and unvoiced) is occurring and of a binary 0 in the absence of speech. The processor includes a denoising algorithm, and it generates transfer functions. The transfer functions include a transfer function generated in response to a determination that voicing information is not present in the acoustic signal received during a specified time period. Also, the transfer functions include a transfer function generated in response to a determination that voicing information is present in the acoustic signal during a specified time period. At least one denoised acoustic data stream is generated using the transfer functions.

Description

  The present invention relates to mathematical methods and electronic systems for removing or suppressing undesirable acoustic noise from acoustic transmissions or recordings.

  In typical acoustic applications, speech from a human user is recorded or stored and transmitted to recipients at various locations.

  In that user's environment, there may be one or more noise sources that contaminate the signal of interest (user's speech) with unwanted acoustic noise. This makes it difficult or impossible for the recipient to understand the user's speech, whether the recipient is a person or a machine. This is currently a particular problem, particularly with the proliferation of portable communication devices such as cellular telephones and personal digital assistants. There are existing ways to suppress these noise adjuncts, all of which require too long computation time or bulky hardware, distort the signal in question too much, or lack useful performance Is. Many of these methods are described in the textbook "Advanced Digital Signal Processing and Noise Reduction" (ISBN 0-471-62692-9) by Vaseghi. As a result, there is a need for a denoising and reduction method that addresses the above shortcomings of typical systems and provides a new technique for cleaning the acoustic signal in question without distortion.

  A method and system is provided for the removal of acoustic noise from human speech, in which noise is removed and its signal restored regardless of noise type, magnitude or orientation. The system includes a microphone and a sensor coupled to the processor. The microphone receives an acoustic signal that includes both noise and a speech signal from a human signal source. The sensor generates binary vocal activity detection (VAD), which is binary “1” when speech (both voiced and unvoiced) is occurring and two when speech is not occurring. Supply a signal with a decimal "0". This VAD signal can be obtained using various methods, such as acoustic gain, accelerometers, and radio frequency (RF) sensors.

  The processor system and method includes a denoizing algorithm that calculates a transfer function between a noise source and a microphone, as well as a transfer function between a human user and a microphone. By using these transfer functions, noise is removed from the received acoustic signal to generate at least one denoised acoustic data stream.

FIG. 1 is a block diagram of a denoising system according to one embodiment. FIG. 2 is a block diagram of an embodiment denoising algorithm assuming a single noise source and a direct path to the microphone. FIG. 3 is a block diagram of a front end of an embodiment denoising algorithm generalized to n distinct noise sources (these noise sources may be reflections or echoes of each other). . FIG. 4 is a block diagram of the front end of one embodiment of the denoising algorithm in the most general case where there are n distinct noise sources and signal reflections. FIG. 5 is a flowchart of a denoising method according to one embodiment. FIG. 6 shows the results of one embodiment of the noise suppression algorithm for an American English-speaking woman in the presence of airport terminal noise (including many other speakers and public announcements). Show.

  FIG. 1 is a block diagram of one embodiment of a denoising system that uses knowledge about when speech is occurring, derived from physiological information about vocal activity. The system includes a plurality of microphones 10 and a plurality of sensors 20 that supply signals to at least one processor 30. The processor includes a subsystem or algorithm that performs denoising processing.

FIG. 2 is a block diagram of one embodiment of a denoising system / algorithm assuming a single noise source and direct path to the microphone. This denoising system diagram includes a schematic description of this process of one embodiment, with a single signal source (100) and a single noise source (101). The algorithm uses (but is not limited to) two microphones: a “signal” microphone (MIC1, 102) and a “noise” microphone (MIC2, 103). Assume that MIC1 captures most of the signal and some noise, while MIC2 captures most of the noise and some signal. This is a configuration common to conventional advanced acoustic systems. Data from the signal to MIC1 is denoted by s (n), data from the signal to MIC2 is denoted by s ₂ (n), data from noise to MIC2 is denoted by n (n), and noise to MIC1 The data from are indicated by n ₂ (n). Similarly, data from MIC1 is denoted m ₁ (n) and data from MIC2 is denoted m ₂ (n), where s (n) represents a discrete sample of the analog signal from the source. Yes.

It is assumed that the transfer function from the signal to MIC1 and the transfer function from noise to MIC2 is 1, but the transfer function from signal to MIC2 is H ₂ (z), and the transfer function from noise to MIC1 is H ₁ ( z). This assumption of unity transfer function does not preclude the generality of the algorithm because the actual relationship between signal, noise and microphone is simply a ratio, and this ratio is It is because it is redefined.

  In conventional denoising systems, information from MIC2 is used in an attempt to remove noise from MIC1. However, an unspoken assumption is that voice activity detection (VAD) is never perfect, so its denoising process does not significantly remove the signal along with the noise. Must be carried out carefully. However, assuming this VAD is perfect and it is equal to zero when no speech is being emitted by the user and equal to 1 when speech is being generated, there is a considerable improvement in this denoising. It can be performed.

In the analysis of a single noise source and direct path to the microphone, in FIG. 2, the acoustic information coming into MIC1 is denoted m ₁ (n). Information coming into MIC2 is similarly denoted m ₂ (n). In the z (digital frequency) domain, these are represented as M ₁ (z) and M ₂ (z). At this time,

here,

Therefore:

  This is the general case for all two microphone systems. In an actual system, there will always be some leakage in the noise to MIC1 and some leakage in the signal to MIC2. Since there are four unknowns and only two known relationships in Equation 1, this cannot be solved clearly.

  However, there are other ways to provide solutions for some of the unknowns in Equation 1. This analysis begins by examining the case where no signal is generated, ie, when the VAD signal is equal to zero and no speech is generated. In this case, s (n) = S (z) = 0, so Equation 1 becomes:

Here, the subscript n of the variable M indicates that only noise is received. This results in the following:

H ₁ (z) can be calculated using any of the available system identification algorithms and microphone outputs if it is certain that the system is receiving only noise. This calculation can be done adaptively, which allows the system to react to changes in its noise.

A solution is now available for one of the unknowns in Equation 1. Another unknown, H ₂ (z), can be determined by using the case where VAD is equal to 1 and speech is being generated. If that happens, but the recent (probably less than 1 second) history of the microphone indicates a low level of noise, it can be considered n (s) = N (z) -0. At this time, Formula 1 is as follows.

This is further:

This is the inverse of H ₁ (z). However, as can be seen, different inputs are used, i.e., this only generated a signal, whereas previously, only noise was generated. During the calculation of H ₂ (z), the value calculated for H ₁ (z) remains constant and vice versa. Accordingly, it is assumed that H ₁ (z) and H ₂ (z) do not change substantially while the other is being calculated.

After calculating H ₁ (z) and H ₂ (z), they are used to remove noise from the signal. If Equation 1 is rewritten as

At this time, N (z) can be substituted as shown below to solve S (z).

If the transfer functions H ₁ (z) and H ₂ (z) can be described with sufficient accuracy, then the noise can be completely removed and the original signal can be restored. it can. This is true regardless of noise magnitude or spectral characteristics. The assumptions made are substantially VAD, sufficiently accurate H ₁ (z) and H ₂ (z) and substantially change when H ₁ (z) and H ₂ (z) are calculating the other It ’s just not. In fact, these assumptions proved to be valid.

The described denoising algorithm can be easily generalized to include any number of noise sources. FIG. 3 is a block diagram of a front end of an embodiment denoising algorithm generalized to n distinct noise sources. These distinguishable noise sources can be (but are not limited to) other reflections or echoes of each other. As shown, there are several noise sources, each having a transfer function or path for each microphone. The path H ₂ previously assigned a name is labeled as H ₀ , which makes it more convenient to label the path of noise source 2 to MIC 1. The output of each microphone is as follows when converted to the z domain.

When there is no signal (VAD = 0), at this time (for the sake of simplicity, z is suppressed):

Thereby, a new transfer function can be defined in the same manner as H ₁ (z).

Thus, H ^~ ₁ depends only on the noise sources and their respective transfer functions and can therefore be calculated at any time when there is no signal being transmitted. Again, the subscript n of the microphone input indicates only that noise has been detected, while the subscript s indicates that the microphone is receiving only the signal.

  Examining Equation 4 while assuming that no noise is generated, the following is obtained.

Thus, H ₀ can be solved as before using any available transfer function calculation algorithm. Mathematically:

Rewriting equation 4 using H ^~ ₁ defined in equation 6 yields:

Solving for S gives:

, Should be the same as Equation 3, where, _{H 0} is replaces the _{H 2,} H ^~ ₁ are replaced the _{H 1.} Thus, this denoising algorithm is still mathematically effective for any number of noise sources including multiple echoes of the noise source. Again, H ₀ and H ^~ ₁ can be estimated with sufficiently high accuracy, and if the assumption that there is only one path from the signal to the microphone, the noise is completely removed. can do.

The most common case is when multiple noise sources and multiple signal sources are involved. FIG. 4 is a block diagram of the front end of one embodiment of the denoising algorithm in the most general case where there are n distinct noise sources and signal reflections. Here, the reflection of the signal enters both microphones. This is the most common case because the reflection of the noise source to the microphone can be accurately modeled as a simple additional noise source. For simplicity, the direct path from the signal to MIC2 has been changed from H ₀ (z) to H ₀₀ (z), and its reflection paths to microphones 1 and 2 are H ₀₁ (z) and H respectively. ₀₂ (z).

  Thereby, the input to the microphone is as follows.

When VAD = 0, the inputs are as follows (suppress z again):

This is the same as Equation 5. Therefore, the calculation of H ^~ ₁ in Equation 6 does not change as expected. Considering this situation without noise, Equation 9 becomes:

This is the definition of H ^~ ₂ .

Again, rewriting Equation 9 using the definition for H ^~ ₁ (as in Equation 7) yields:

With some algebraic manipulation:

Finally,

Equation 12 is the same as Equation 8, except, _{H 0} are replaced by H ^~ _2, it has also been added to the left side elements of the (1 _{+ H 01).} This extra factor means that S cannot be solved directly in this situation, but a solution can be generated for every addition of that echo to the signal. This is not a bad situation, because there are many traditional ways of dealing with echo suppression, and even if these echoes are not suppressed, they have a significant impact on speech comprehension. Is unlikely to happen. More complex calculations of H ^~ ₂ need to take into account signal echoes in microphone 2.

  FIG. 5 is a flowchart of a denoising method according to one embodiment. In operation, an acoustic signal is received (502). In addition, physiological information related to the person's vocal activity is received (504). When it is determined that utterance information from the acoustic signal does not exist for at least one specified time, a first transfer function representing the acoustic signal is calculated (506). A second transfer function representing the acoustic signal is calculated (508) when it is determined that utterance information exists in the acoustic signal for at least one specified time. The removal of noise from the acoustic signal is performed using at least one combination of the first transfer function and the second transfer function, thereby generating a denoised acoustic data stream (510).

An algorithm for denoising, or denoising algorithm, has been described here, from the simplest case of a single noise source with a direct path to multiple noise sources with reflections and echoes. This algorithm has been shown to be executable under any environmental conditions. The type and amount of noise is not important if good estimates are made for H ^~ ₁ and H ^~ ₂ , and if they do not change substantially during the other calculation. If the user environment is such that echoes are present, they can be compensated if they come from a noise source. If signal echoes are also present, they affect the cleaned signal, but the effect is negligible in most environments.

In operation, the algorithm of one embodiment has shown excellent results in handling various noise types, magnitudes and orientations. However, approximations and adjustments must always be made when moving from mathematical concepts to engineering applications. Equation 3 makes one assumption, which assumes that H ₂ (z) is small and therefore H ₂ (z) H ₁ (z) ≈0, so Equation 3 is become.

This means that only H ₁ (z) must be calculated, thereby speeding up the process and significantly reducing the number of calculations required. This approximation can be easily realized if the microphone is selected appropriately.

Another approximation relates to the filter used in one embodiment. The actual H ₁ (z) will undoubtedly have both poles and zeros, but for stability and simplicity, an all-zero finite impulse response (FIR) filter is used. With enough taps (approximately 60), the approximation to the actual H ₁ (z) is very good.

  With respect to subband selection, it becomes more difficult to calculate accurately as the frequency range over which the transfer function must be calculated becomes wider. Therefore, the acoustic data was divided into 16 subbands, and the lowest frequency was 50 Hz and the highest was 3700. Next, the denoising algorithm was applied to each subband in turn, and the 16 denoising data streams were combined to generate denoising acoustic data. This works very well, but any combination of subbands (ie 4, 6, 8, 32 equally spaced and perceptually spaced) can be used, and this is similar It turned out to work.

  The magnitude of the noise is suppressed in one embodiment so that the used microphone does not saturate (ie, operate outside the linear response region). The important thing is to ensure the best performance by the linear operation of the microphone. Even with this suppression, very high signal-to-noise ratio (SNR) tests could be performed (up to about -10 dB).

The calculation of H ₁ (z) was performed every 10 milliseconds using the least mean square method (LMS), a general adaptive transfer function. This explanation can be found in “Adaptive Signal Processing” (1985), ISBN0-13-004029-0 by Widrow and Stearns, published by Prentice-Hall.

  The VAD for one embodiment was obtained from a radio frequency sensor and two microphones, which generated very high accuracy (> 99%) for both voiced and unvoiced speech. This VAD for one embodiment uses a radio frequency (RF) interferometer to detect (but is not limited to) tissue motion related to human speech generation. This is therefore completely acoustic noise free and can therefore function in any acoustic noise environment. By using simple energy measurements, it can be determined whether voiced speech is occurring. Unvoiced speech can be determined using conventional frequency-based methods by proximity to the voiced portion or by a combination of the above. Since unvoiced speech does not have that much energy, its activation accuracy is not as important as voiced speech.

  By detecting voiced speech and unvoiced speech with high reliability, the algorithm of one embodiment can be realized. Here again, it is beneficial to repeat only that the denoising algorithm does not depend on the method of obtaining the VAD, which is particularly accurate for voiced speech. If no speech is detected and training occurs for that speech, the subsequent denoising acoustic data may be distorted.

  Data was collected on four channels, one for MIC1, one for MIC2, and two for radio frequency sensors that detect tissue motion associated with voiced speech. This data was sampled simultaneously at 40 KHz and then digitally filtered and decimated to 8 KHz. By using this high sampling rate, any aliasing that could result from this analog-to-digital process was reduced. A 4-channel National Instruments A / D board was used with Labview to capture and store the data. This data was then loaded into the C program and denoised for 10 milliseconds at a time.

  FIG. 6 shows the results of one embodiment of the noise suppression algorithm for an American English-speaking woman in the presence of airport terminal noise (including many other speakers and public announcements). Show. The speaker is calling the number 406-5562 in the middle of the middle airport terminal noise. Dirty acoustic data was denoised 10 milliseconds at a time and prefiltered to 50-3700 Hz prior to 10 millisecond denoising of the data. A noise reduction of about 17 dB was revealed. Since this sample was not post-filtered, all realized noise reduction is due to this algorithm of one embodiment. The algorithm adapts instantly to noise and is therefore capable of removing the very difficult noise of other people's speakers. Many different types of noise (street noise, helicopters, music, sine waves to name just a few) tested all of them, with similar results. Further, the noise direction can be substantially changed without significantly changing the noise suppression performance. Finally, clean speech distortion is very low, ensuring good performance for speech recognition engines as well as human recipients as well.

It has been shown that the noise removal algorithm of one embodiment can be performed under any environmental conditions. The type and amount of noise is negligible if a good estimate is made for H ^~ ₁ and H ^~ ₂ . If the user environment is such that echoes are present, they can be compensated if they come from noise sources. If signal echoes are also present, these will affect the cleaned signal, but the effect is negligible in most environments.

  While various embodiments have been described with reference to the drawings, the detailed description and drawings are not intended to be limiting. Although various combinations of the described elements have not been shown, they are within the scope of the invention as set forth in the appended claims.

Claims (28)

A noise removal method for removing noise from an acoustic signal,
Receiving a plurality of acoustic signals;
Receiving physiological information related to human vocal activity;
Generating at least one first transfer function representing the plurality of acoustic signals when it is determined that utterance information is not present in the plurality of acoustic signals for at least one specified time;
Generating at least one second transfer function representative of the plurality of acoustic signals when it is determined that speech information is present in the plurality of acoustic signals for at least one specified time;
Using at least one combination of the at least one first transfer function and the at least one second transfer function to remove noise from the plurality of acoustic signals and to perform at least one denoising acoustic Generating a data stream; and
A noise removal method comprising:   The method of claim 1, wherein the plurality of acoustic signals includes at least one reflection of at least one associated noise source signal and at least one reflection of at least one acoustic source signal. To remove noise.   2. The method of claim 1, wherein the step of receiving physiological information uses at least one detector selected from the group consisting of a radio frequency device, an electrical grottograph, an ultrasonic device, an acoustic throat microphone, and an airflow detector. Receiving a physiological data related to human speech by performing a denoising method.   2. The method of claim 1, wherein the step of receiving a plurality of acoustic signals includes receiving using a plurality of independently positioned microphones.   The method of claim 1, wherein removing noise further generates at least one third transfer function using the at least one first transfer function and the at least one second transfer function. The noise removal method characterized by including.   The method of claim 1, wherein generating the at least one first transfer function comprises recalculating the at least one first transfer function during at least one pre-specified interval. Noise elimination method.   The method of claim 1, wherein generating the at least one second transfer function comprises recalculating the at least one second transfer function during at least one pre-specified interval. Noise elimination method.   The method of claim 1, wherein the generating of the at least one first transfer function and the at least one second transfer function comprises using at least one technique selected from the group consisting of adaptive techniques and recursive techniques. A noise removal method characterized by comprising: A noise removal method for removing noise from an electronic signal,
Detecting the absence of utterance information during at least one time;
Receiving at least one noise source signal during said at least one time;
Generating at least one transfer function representative of the at least one noise source signal;
Receiving at least one composite signal including an acoustic signal and a noise signal;
Generating at least one denoised acoustic data stream from the at least one composite signal by removing the noise signal using the at least one transfer function;
A noise removal method comprising:   The method of claim 9, wherein the at least one noise source signal comprises at least one reflection of at least one associated noise source signal.   The method of claim 9, wherein the at least one composite signal includes at least one reflection of at least one associated composite signal.   10. The method of claim 9, wherein detecting is by using at least one detector selected from the group consisting of a radio frequency device, an electrical grottograph, an ultrasound device, an acoustic throat microphone, and an airflow detector. Collecting a physiological data related to human speech.   The method of claim 9, wherein receiving includes receiving the at least one noise source signal using at least one microphone.   14. The method of claim 13, wherein the at least one microphone includes a plurality of independently positioned microphones.   The method of claim 9, wherein removing the noise signal from the at least one composite signal using the at least one transfer function comprises using at least one other transfer function. Generating a function. A noise removal method comprising:   The method of claim 9, wherein generating at least one transfer function comprises recalculating the at least one transfer function during at least one pre-specified interval. Method.   10. The method of claim 9, wherein generating at least one transfer function is to calculate the at least one transfer function using at least one technique selected from the group consisting of adaptive techniques and recursive techniques. And a noise removing method comprising: A noise removal method for removing noise from an electronic signal,
Determining at least one silent period during which voiced information is absent;
Receiving at least one noise signal input during the at least one silent period and generating at least one silent transfer function representing the at least one noise signal;
Determining at least one voicing time during which voiced information exists;
Receiving at least one acoustic signal input from at least one signal sensing device between said at least one speech time and generating at least one speech transfer function representative of said at least one acoustic signal;
Receiving at least one composite signal including an acoustic signal and a noise signal;
At least one denoised acoustic data by removing the noise signal from the at least one composite signal using at least one combination of the at least one unvoiced transfer function and the at least one utterance transfer function. A step of generating a stream;
A noise removal method comprising: A noise removal system for removing noise from an acoustic signal,
At least one receiver for receiving at least one acoustic signal;
At least one sensor for receiving physiological information relating to human vocal activity;
At least one processor coupled between the at least one receiver and the at least one sensor generating a plurality of transfer functions, wherein the at least one first transfer function representing the at least one acoustic signal is , At least one second transfer function representative of the at least one acoustic signal generated in response to determining that utterance information is absent from the at least one acoustic signal for at least one specified time Is generated in response to determining that utterance information is present in the at least one acoustic signal for at least one specified time, and wherein the at least one first transfer function and the at least one Using at least one combination with a second transfer function to remove noise from the at least one acoustic signal; It generates one acoustic data stream denoising process even without, at least one processor of said,
A noise removal system consisting of   21. The denoising system of claim 19, wherein the at least one sensor includes at least one radio frequency (RF) interferometer that detects tissue motion associated with human speech generation.   20. The system of claim 19, wherein the at least one sensor comprises at least one sensor selected from the group consisting of a radio frequency device, an electrical grottograph, an ultrasound device, an acoustic throat microphone, and an airflow detector. A noise removal system characterized by The system of claim 19, further comprising:
Dividing the acoustic data of the at least one acoustic signal into a plurality of subbands;
Using the at least one combination of the at least one first transfer function and the at least one second transfer function to remove noise from each of the plurality of subbands and to provide a plurality of denoised acoustic data streams Is generated,
Generating the at least one denoising acoustic data stream by combining the plurality of denoising acoustic data streams;
Including a noise removal system.   The system of claim 19, wherein the at least one receiver includes a plurality of independently positioned microphones.   A noise removal system for removing noise from an acoustic signal, comprising: at least one processor coupled between at least one microphone and at least one utterance sensor, wherein the at least one utterance sensor Collect relevant physiological data, detect absence of voiced information using the at least one utterance sensor during at least one time, and at least using the at least one microphone at the at least one time Receiving at least one noise source signal, the at least one processor generates at least one transfer function representing the at least one noise source signal, and the at least one microphone includes at least an acoustic signal and a noise signal; Before receiving one composite signal At least one processor generates at least one denoised acoustic data stream by removing the noise signal from the at least one composite signal using the at least one transfer function; Noise removal system.   A signal processing system coupled between at least one user and at least one electronic device, the signal processing system including at least one denoising subsystem for removing noise from the acoustic signal; A denoising subsystem comprising at least one processor coupled between at least one receiver and at least one sensor, the at least one receiver coupled to receive at least one acoustic signal; The at least one sensor is coupled to receive physiological information related to human vocal activity, wherein the at least one processor generates a plurality of transfer functions and represents at least one acoustic signal A first transfer function wherein the utterance information is the at least one specified time; The at least one second transfer function that is generated in response to the determination of being absent from at least one acoustic signal and the at least one second transfer function represents the utterance information for at least one specified time Using at least one combination of the at least one first transfer function and the at least one second transfer function, in response to determining that the at least one acoustic signal is present in the at least one acoustic signal, A signal processing system for generating at least one denoised acoustic data stream by removing noise from the at least one acoustic signal.   26. The system of claim 25, wherein the at least one electronic device is at least selected from the group consisting of a cellular phone, a personal digital assistant, a portable communication device, a computer, a video camera, a digital camera, a telematic system. A signal processing system comprising: one device. A computer readable medium containing executable instructions when the executable instructions are executed in a processing system,
Receiving at least one acoustic signal,
Receiving physiological information related to human vocal activity,
Generating at least one first transfer function representative of the at least one acoustic signal in response to determining that utterance information is absent from the at least one acoustic signal for at least one specified time; ,
Generating at least one second transfer function representative of the at least one acoustic signal in response to determining that utterance information is present in the at least one acoustic signal for at least one specified time period; And
At least one denoising acoustic data by removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function. Generating a stream,
A computer readable medium characterized by removing noise from the received acoustic signal. An electromagnetic medium comprising executable instructions when the executable instructions are executed in a processing system;
Receiving at least one acoustic signal,
Receiving physiological information related to human vocal activity,
Generating at least one first transfer function representative of the at least one acoustic signal in response to determining that utterance information is absent from the at least one acoustic signal for at least one specified time; ,
Generating at least one second transfer function representative of the at least one acoustic signal in response to determining that utterance information is present in the at least one acoustic signal for at least one specified time period; And
At least one denoising acoustic data by removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function. Generating a stream,
An electromagnetic medium comprising executable instructions, characterized by removing noise from a received acoustic signal.

Images