JP2005529379A - Method and apparatus for removing noise from electronic signals - Google Patents

Abstract

Methods and systems for removing acoustic noise from human speech are provided. The acoustic noise is removed regardless of the noise type, amplitude, or direction. The system includes a processor coupled between a microphone and a vocal activity detection (“VAD”) element. The processor executes a noise cancellation algorithm that generates a transfer function. The processor receives acoustic data from the microphone and data from the VAD. The processor generates various transfer functions when the VAD indicates vocal activity and when the VAD does not indicate vocal activity. A transfer function is used to generate a noise cancellation data stream.

Description

  The present invention relates to the field of mathematical methods and electronic systems for removing or suppressing unwanted acoustic noise from acoustic transmission or recording.

Related Application This patent application is a continuation-in-part of US patent application Ser. No. 09 / 905,361 filed on Jul. 12, 2001, the contents of which are hereby incorporated herein by reference. And This patent application also claims the priority of US Provisional Patent Application No. 60 / 332,202, filed Nov. 21, 2001.

  In a typical acoustic application, audio from a human user is recorded or stored and transmitted to receivers at different locations. There may be one or more noise sources in the user's environment, which can contaminate the signal of interest (user's voice) with undesirable acoustic noise. This makes it difficult or impossible for the recipient to understand the user's voice, whether human or machine. This is particularly a problem with the widespread use of portable communication devices such as cellular telephones and personal digital assistants. There are ways to suppress these noise additions, but they have significant drawbacks. For example, existing methods are slow because they require computation time. In addition, existing methods require bulky hardware and may have poor performance such that the signal of interest is unacceptably distorted or cannot be used. Many of these existing methods are described in textbooks such as Vaseghi's "Advanced Digital Signal Processing and Noise Reduction" ISBN 0-471-62692-9.

Advanced Digital Signal Processing and Noise Reduction

  The following description presents specific details in order to provide a thorough understanding of embodiments of the invention, and a description that enables it to be implemented. However, it will be apparent to one skilled in the art that the present invention may be practiced without these details. In other instances, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention.

  Unless otherwise noted below, the structure and operation of the various blocks shown in the figures are identical to conventional designs. As a result, such blocks need not be described in further detail here. Because these should be known to those skilled in the art. Such further details are omitted for the sake of brevity so as not to obscure the detailed description of the invention. Any modifications necessary to the blocks in the figure (or other embodiments) can be readily made by those skilled in the art based on the detailed description presented herein.

  FIG. 1 is a block diagram of one embodiment of a noise cancellation system that uses knowledge of when speech is uttered. This knowledge is obtained from psychological information about vocal activity. The system includes a microphone 10 and a sensor 20 that provide signals to at least one processor 30. The processor includes a noise cancellation subsystem or algorithm 40.

FIG. 2 is a block diagram illustrating the noise removal algorithm of one embodiment, showing the system components used. A direct path to a single noise source and microphone is assumed. FIG. 2 includes a graphical illustration of a process of one embodiment with a single signal source 100 and a single noise source 101. The algorithm uses, but is not limited to, two microphones: a “signal” microphone 1 (“MIC1”) and a “noise” microphone 2 (“MIC2”). MIC1 mainly captures the signal and assumes that the noise is only part, while MIC2 mainly captures the noise and the signal is part. Data from the signal source 100 to the MIC 1 is indicated by s (n). Here, s (n) is a discrete sample of the analog signal from the signal source 100. Data from the signal source 100 to the MIC _{2 is} denoted by s ₂ (n). Data from the noise source 101 to the MIC 2 is indicated by n (n). Data from the noise source 101 to the MIC 1 is denoted by n ₂ (n). Similarly, data from the MIC1 to the noise removal element 105 is denoted by m ₁ (n), and data from the MIC2 to the noise removal element 105 is denoted by m ₂ (n).

  The noise removal element also receives a signal from the vocal activity detection (“VAD”) element 104. The VAD 104 detects and uses psychological information to determine when the speaker is speaking. In various embodiments, the VAD includes a radio frequency device, an electroglottograph, an ultrasound device, an acoustic throat microphone, and / or an airflow detector.

Assume that the transfer function from signal source 100 to MIC1 and the transfer function from noise source 101 to MIC2 are unity. The transfer function from the signal source 100 to MIC2 is denoted by H ₂ (z), and the transfer function from the noise source 101 to MIC1 is denoted by H ₁ (z). Assuming a transfer function of 1 does not preclude the generalization of this algorithm. Because the actual relationship between signal, noise, and microphone is a simple ratio, this ratio is redefined in this way for simplicity.

  In the conventional noise removal system, information from MIC2 is used to remove noise from MIC1. However, there is an implicit assumption that the VAD element 104 is by no means perfect, so noise cancellation must be done carefully to avoid removing too many signals with noise. However, if the VAD 104 is perfect and is assumed to be equal to 0 when the user does not speak, and equals 1 when the user speaks, a significant improvement in noise removal can be obtained.

Referring to FIG. 2, in analyzing the direct path to a single noise source 101 and microphone, the total acoustic information coming into MIC1 is denoted m ₁ (n). Similarly, the total acoustic information coming into MIC2 is indicated by m ₂ (n). In the z (digital frequency) domain, these are represented as M ₁ (z) and M ₂ (z). Then

And

Therefore,

Formula 1
This is the general case for all two-microphone systems. In an actual system, there is always some noise leakage and is mixed into MIC1, and further some signal leakage is mixed into MIC2. Equation 1 is a relationship between four unknowns and only two known numbers, and therefore cannot be solved explicitly.

  However, there is another solution for some of the unknowns in Equation 1. The analysis begins with consideration when no signal is generated, i.e., when the signal from the VAD element 104 is equal to 0 and no sound is being emitted. In this case, s (n) = S (z) = 0, and Equation 1 is transformed as follows.

Here, the subscript n of the variable M indicates that only noise is received. From this, the following equation is derived.

Formula 2
H ₁ (z) can be calculated using available system identification algorithms and the microphone output when it is certain that the system is only receiving noise. Since this calculation can be done adaptively, the system can react to noise changes.

The solution here can be used for one of the unknowns in Equation 1. Another unknown H ₂ (z) can be determined by using when VAD is equal to 1 and speech is being emitted. If this occurs, but n (s) = N (z) -0 can be assumed if the latest (possibly less than 1 second) history of the microphone indicates a low noise level. Therefore, Equation 1 is transformed as follows.

On the other hand, the following equation is derived from this.

This is the reverse of the H ₁ (z) calculation. However, note that different inputs are used. In this case, only the signal is generated, whereas in the past, only the noise is generated. During the calculation of H ₂ (z)), the value calculated for H ₁ (z) remains constant and vice versa. That is, while _one of H ₁ (z) and H ₂ (z) is being calculated, it is considered that the non-calculated one does not change substantially.

After calculating H ₁ (z) and H ₂ (z), they are used to remove noise from the signal. Rewriting equation 1 as follows:

  Next, if N (z) is substituted as shown in the equation, S (z) can be solved as follows.

Formula 3
If the transfer functions H ₁ (z) and H ₂ (z) can be described with sufficient accuracy, the noise can be completely removed and the original signal can be reproduced. This is true unless it relates to noise amplification or spectral characteristics. The only assumptions to make are perfect VAD, i.e. when calculating one of H ₁ (z) and H ₂ (z) and H ₁ (z) and H ₂ (z) that are sufficiently accurate, The other is that it does not change substantially. In practice, these assumptions have proven to be valid.

The denoising algorithm described herein can be easily generalized to include any number of noise sources. FIG. 3 is a block diagram of the front end of an embodiment denoising algorithm generalized to n separate noise sources. These separate noise sources may be reflections or echoes of each other, but are not so limited. Several noise sources are shown, each having a transfer function or path to each microphone. The path previously designated H ₂ has been rewritten as H ₀ so that the display of the path to MIC 1 of noise source 2 is more convenient. When the output of each microphone is converted to the z domain, it is expressed as follows.

Formula 4
When there is no signal (VAD = 0), (z is omitted for clarity):

Formula 5
Here, a new transfer function similar to the aforementioned H ₁ (z) can be defined.

Equation 6
Thus, H ₁ ^~ depends only on the noise sources and their respective transfer functions and can be calculated whenever no signal is being transmitted. Again, the subscript n attached to the microphone input indicates only that noise has been detected, while the subscript s indicates that the microphone is receiving only the signal.

  Examining Equation 4 assuming no noise, the following equation is obtained.

Thus, using any available transfer function calculation algorithm, one can solve for H ₀ as before. Mathematically expressed as:

Rewriting equation 4 using H ₁ ^~ defined in equation 6 yields the following equation:

Equation 7
Solving for S yields:

Equation 8
This is the same as Equation 3, _{H 0} is replaces the _{H 2, H} ^{1 ~} has replaced the _{H 1.} Thus, the denoising algorithm is mathematically effective for any number of noise sources, including multiple echoes of the noise source. In this case also it is possible to estimate H ₀ and H ₁ ^~ with sufficient accuracy, it is maintained the aforementioned assumption that the path from the signal to the microphone, the only, the noise can be completely removed.

The most common case involves multiple noise sources and multiple signal sources. FIG. 4 is a block diagram of the front end of an embodiment denoising algorithm in the most general case where there are n distinct noise sources and signal reflections. Here, signal reflections enter both microphones. This is the most common case and the reflection of the noise source entering the microphone can be accurately modeled as a simple additional noise source. For clarity, the direct path from the signal to MIC2 changes from H ₀ (z) to H ₀₀ (z), and the reflection paths to MIC1 and MIC2 are respectively H ₀₁ (z) and H ₀₂ ( z).

  Then, the input to the microphone is as follows.

Equation 9
When VAD = 0, the input is as follows (again, “z” is omitted).

This is the same as Equation 5. In other words, the calculation of _H ^{1 ~} in Formula 6 is invariant as expected. Examining the situation without noise, Equation 9 is transformed as follows.

This leads ^to the definition of H ₂ ^~ .

Equation 10
Rewriting Equation 9 using the definition for H ₁ ^~ again (as in Equation 7) yields:

Equation 11
Several algebraic operations yield:

Finally, the following equation is obtained.

Formula 12
Equation 12 is the same as Equation 8, except that H ₀ replaces H ₂ ^˜ and a (1 + H ₀₁ ) coefficient is added on the left side. This new coefficient means that in this situation S cannot be solved directly, but a solution can be generated for the signal plus all its echoes. This is not a bad situation. This is because there have been many methods for dealing with echo suppression, and even if the echo cannot be suppressed, it is unlikely that the echo will have an important effect on the understanding of speech. To deal with the signal echo in MIC2 acting as a noise source, a more complex calculation of H ₂ ^~ is required.

  FIG. 5 is a flow diagram of a noise denoising method of one embodiment. In operation, an acoustic signal is received (502). Furthermore, it receives psychological information associated with human voice activity (504). If it is determined that the utterance information is not present in the acoustic signal for at least the designated one hour period, a first transfer function representing the acoustic signal is calculated (506). If it is determined that the utterance information is in the acoustic signal for at least one hour, a second transfer function representing the acoustic signal is calculated (508). At least one combination of the first transfer function and the second transfer function is used to remove noise from the acoustic signal and generate a noise-canceled acoustic data stream (510).

Here, the noise removal algorithm, ie, the noise cancellation algorithm, will be described from the simplest case of a single noise source having direct paths to multiple noise sources with reflections and echoes. This algorithm is shown here because it can be executed under any environmental conditions. The type and amount of noise is obtained the correct estimate of ^~ H ₁ ^~ and H ₂ is and, if the other does not change substantially during the calculated one, is that insignificant. If echoes are present in the user environment, they can be compensated if they come from noise sources. If signal echoes are also present, they affect the cleaned signal, but the effect can be considered negligible in most cases.

In operation, the algorithm of one embodiment has shown excellent results when processing various noise types, amplitudes, and orientations. However, approximations and adjustments must always be made when moving from mathematical concepts to design applications. Assuming that H ₂ (z) is small in equation 3 and therefore H ₂ (z) H ₁ (z) ≈0, equation 3 is transformed as follows.

This means that only H ₁ (z) needs to be calculated, resulting in a faster process and a significant reduction in the number of calculations required. This approximation is easily achieved by proper selection of the microphone.

In one embodiment, another approximation involves the use of a filter. The actual H ₁ (z) undoubtedly has both poles and zeros, but uses an all-zero finite impulse response (FIR) filter for stability and simplicity. With enough taps (about 60), the approximation to the actual H ₁ (z) is very good.

  With respect to subband selection, the wider the frequency range for which the transfer function must be calculated, the more difficult it is to calculate it with high accuracy. Therefore, the acoustic data is divided into 16 subbands, with the lowest frequency being 50 Hz and the highest frequency being 3700 Hz. A noise cancellation algorithm is then applied to each subband in turn, and the 16 noise cancellation data streams are recombined to generate noise cancellation acoustic data. This works very well, but any subband combination (ie, 4, 6, 8, 32, equally spaced, perceptible intervals, etc.) can be used and has been found to work as well.

  In one embodiment, the noise amplitude was limited so that the microphone used was not saturated (ie, operating outside the linear response region). It is important that the microphone behaves linearly to ensure the best performance. Even with this constraint, even a signal with a very low signal-to-noise ratio (SNR) can eliminate noise (down to -10 dB or less).

The calculation of H ₁ (z) is performed every 10 milliseconds using a least mean square (LMS) method and a common adaptive transfer function. The explanation can be found in "Adaptive Signal Processing" (1985) by Widrow and Steams, Prentic-Hall, ISBN 0-13-004029-0.

  The VAD for one embodiment is obtained from a radio frequency sensor and two microphones, resulting in very high accuracy (> 99%) for both voiced and unvoiced speech. The VAD of one embodiment uses a radio frequency (RF) interferometer to detect tissue motion associated with human speech production, but is not so limited. It is therefore completely free of acoustic noise and can function in any acoustic noise environment. A simple energy measurement of the RF signal can be used to determine whether voiced speech is being emitted. The determination of unvoiced speech can be made by proximity to the voiced interval determined using a conventional acoustic method, using an RF sensor or similar utterance sensor, or a combination of the foregoing. Since the energy in unvoiced speech is much less, its activation accuracy is not as important as voiced speech.

  By detecting voiced and unvoiced speech reliably, the algorithm of one embodiment can be realized. Here again, it is useful to repeat that the noise removal algorithm does not depend on how the VAD is acquired, and particularly that it is highly accurate for voiced speech. If no speech is detected and training is performed on the speech, subsequent noise-canceling acoustic data may be distorted.

  Data was collected on 4 channels. That is, one was assigned to MIC1, one was assigned to MIC2, and two radio frequency sensors were detected that detected tissue movements associated with voiced speech. Data was sampled simultaneously at 40 kHz, then digitally filtered and decimated to 8 kHz. By using a high sampling rate, the aliasing that can be caused by the analog-digital process has been reduced. A National Instruments 4-channel A / D board was used with Labview to capture and store the data. The data was then read into the C program and noise was erased 10 milliseconds at a time.

  FIG. 6 shows the results for an English speaking American woman of one embodiment of the noise suppression algorithm in the presence of airport terminal noise including many other speakers and public calls. The speaker pronounced the number 406-5562 in moderate airport terminal noise. The dirty acoustic data is noise-erased 10 milliseconds at a time, and the 10-millisecond data is pre-filtered at 50 to 3700 Hz before noise cancellation. A noise reduction of about 17 dB is evident. This sample was not post-filtered. That is, all of the realized noise reduction is due to the algorithm of one embodiment. It is clear that the algorithm adapts quickly to noise and can remove the very difficult noise of other speakers. Many different types of noise were examined and similar results were obtained. Among them are town noise, helicopters, music, and sine waves, and many others. In addition, the noise direction can be significantly changed, and the noise suppression performance is hardly changed. Lastly, the noise-removed speech has very little distortion and ensures excellent performance for speech recognition engines and human receivers as well.

It has been shown that the denoising algorithm of one embodiment can be performed under any environmental conditions. If H ₁ ^- and H ₂ correct estimate of ^~ is obtained, is that insignificant. If echoes are present in the user environment, they can be compensated if they come from noise sources. If signal echoes are also present, they affect the purification signal, but the effect can be considered negligible in most cases.

FIG. 7 is a block diagram of a physical configuration that performs noise cancellation using the unidirectional microphone M2 for noise and the omnidirectional microphone M1 for speech under the embodiment of FIGS. FIG. As mentioned above, the path from the voice to the noise microphone (MIC2) is approximated to 0, and the approximation is achieved by careful placement of omnidirectional and unidirectional microphones. If the noise is opposite to the signal position (noise source N1), this has a very high effect (20-40 dB noise suppression). However, if the noise source is facing the same side as the speaker (noise source N2), the performance can be reduced to only 10-20 dB noise suppression. This reduction in suppression capability can be attributed to actions taken to ensure that H ₂ is close to zero. These treatments involve using a unidirectional microphone for the noise microphone (MIC2), so that almost no signal appears in the noise data. The unidirectional microphone cancels acoustic noise coming from a specific direction, and thus cancels noise coming from the same direction as the voice. For this, the adaptive algorithm can also be the ability to characterize the noise is then removed in a position such as N ₂ is restricted. The same effect can be seen when a unidirectional microphone is used for the voice microphone M1.

However, when the unidirectional microphone M2 is replaced with an omnidirectional microphone, a large amount of signal is captured by M2. This would violate the above assumption that H ₂ is 0, so that a large amount of signal is removed during speech and noise cancellation and “de-signaling” is performed. Become. This is unacceptable if signal noise must be minimized. To reduce the distortion, thus, to calculate the value of H _2. However, the value of H ₂ can not calculate in the presence of noise, otherwise noise is recognized incorrectly voice (mislabel), it will not be removed.

From experience with acoustic-only microphone arrays, a small two-microphone array can be a solution to this problem. FIG. 8 is a noise cancellation microphone configuration that includes two omnidirectional microphones under an embodiment. The same effect can be obtained by using two unidirectional microphones oriented in the same direction (towards the signal source). In yet another embodiment, one unidirectional microphone and one omnidirectional microphone are used. The idea is to capture similar information from the acoustic source in the direction of the signal source. The relative positions of the signal source and the two microphones are constant and known. Microphone and spaced apart by a distance d corresponding to n discrete time sample was placed, by placing the speaker on the axis of the array can be fixed and H ₂ in the form of Cz ^-n. Here, C is a difference in amplitude of signal data in M ₁ and M ₂ . For the discussion that follows, assume n = 1. However, any integer other than 0 can be used. The use of positive integers is recommended for causality. As the amplitude of the spherical pressure source changes as 1 / r, this allows not only the direction of the pressure source but also the designation of its distance. The required C can be estimated as follows.

  FIG. 9 is a plot of required C versus distance under the embodiment of FIG. It can be seen that the asymptote is at C = 1.0, and C reaches 0.9 at about 38 centimeters, slightly more than 1 foot, and reaches 0.94 at about 60 centimeters. For distances typically involved in handsets and earpieces (4 to 12 cm), C will be between about 0.5 and 0.75. This is a difference of about 19 to 44% from a noise source located at about 60 cm, and it is clear that most noise sources are located further away. Therefore, a system using this configuration can very effectively discriminate between noise and signal even if their orientations are similar.

To determine the effect on noise cancellation when the estimated value of C is poor, it is assumed that C = nC _0. Here, C is an estimated value, and C ₀ is an actual value of C. Using the signal definition from the above,

Since H ₂ (z) is assumed to be very small, the signal can be approximated as:

This is true when there is no audio. This is because H ₂ = 0 by definition. However, the voice is uttered, the H ₂ is non-zero, and the setting to Cz ^-1, as follows.

This can be rewritten as follows.

  The last coefficient in the denominator determines the error due to poor estimation of C. If this coefficient is E,

It becomes.
Since z ⁻¹ H ₁ (z) is a filter, its amount is always positive. Therefore, the change of the calculated signal magnitude due to E depends entirely on (1-n).

  There are two possibilities for error: C underestimation (n <1) and C overestimation (n> 1). In the first case, C is estimated smaller than actual, i.e. the signal is closer than estimated. In this case, (1-n), that is, E is positive. Therefore, the denominator becomes excessively large, and the amplitude of the signal from which noise is removed becomes excessively small. This indicates signal erasure. In the second case, the signal is farther than estimated, E is negative, and S is greater than actual. In this case, noise erasure is insufficient. Since very small signal distortion is desirable, the signal should deviate towards an overestimation of C.

Further, as a result, when the noise is located at the same solid angle (direction from M1) as the signal, the signal is almost removed according to the change of C between the signal position and the noise position. Thus, if a handset is used and M ₁ is about 4 cm from the mouth, the required C is about 0.5, and for noise, at about 1 meter, C is about 0.96. Thus, for noise, the estimated value of C = 0.5 means that C is underestimated for noise and the noise is removed. The removal amount depends directly on (1-n). The algorithm therefore uses the direction and distance to the signal to separate the signal from the noise.

One problem that arises with this technique is its stability. In other words, the deconvolution of (1-H ₁ H ₂ ) raises the question of stability. This is because at the beginning of each voiced segment, it becomes necessary to calculate the reciprocal of 1-H ₁ H ₂ . This helps to reduce the computation time required to implement the algorithm, ie the number of instructions per cycle. Because since H ₂ is considered to be constant, there is no need to calculate the inverse for every voiced window, because initial or only one. However, because it is necessary to calculate the reciprocal of 1-H ₁ H ₂ each time a false positive occurs, in this approximation, false positives are more costly to calculate.

Fortunately, the selection of H ₂ eliminates the need for deconvolution. From the above discussion, the signal can be written as:

This can be rewritten as:

Or

However, since it is in the form of H ₂ (z) Cz ^−1, the sequence in the time domain looks as follows:

  This means that the current signal sample requires the current MIC1 signal, the current MIC2 signal, and the previous signal sample. This means that deconvolution is not necessary and simple division and then convolution as before are sufficient. The required increase in computational complexity is minimal. This improvement is therefore easy to implement.

  The effect of the difference in microphone response for this embodiment can be shown by examining the configuration described with reference to FIGS. 2, 3, and 4, where the frequency response of MIC1 and MIC2 is Only the transfer functions A (z) and B (z) represented along with their filtering and amplification responses are included. FIG. 10 is a block diagram of the front end of the denoising algorithm under one embodiment where the two microphones MIC1 and MIC2 have different response characteristics.

FIG. 10 includes a schematic description of the process of one embodiment, where there is a single signal source 1000 and a single noise source 1001. This algorithm uses, but is not limited to, two microphones: a “signal” microphone 1 (“MIC1”) and a “noise” microphone 2 (“MIC2”). Assume that MIC1 primarily captures signals and also captures some noise, while MIC2 primarily captures noise and also captures some signal. A signal from the signal source 1000 to the MIC 1 is denoted by s (n). s (n) is a discrete sample of the analog signal from the signal source 1000. Data from the signal source 1000 to the MIC _{2 is} denoted by s ₂ (n). Data from the noise source 1001 to the MIC2 is indicated by n (n). Data from the noise source 1001 to the MIC 1 is denoted by n ₂ (n).

The transfer function A (z) represents the frequency response of MIC1, along with its filtering and amplification response. The transfer function B (z) represents the frequency response of MIC2, along with its filtering and amplification response. The output of the transfer function A (z) is denoted by m ₁ (n), and the output of the transfer function B (z) is denoted by m ₂ (n). The signals m ₁ (n) and m ₂ (n) are accepted by the noise removal element 1005, which processes these signals and outputs “noise removal speech”.

  From now on, the term “MIC X frequency response” is intended to include the combined effects of the microphone and any amplification and filtering processes that occur during the data recording process for that microphone. Solving for signal and noise (omitting “z” for clarity)

Here, if the latter is substituted for the former, the following equation is obtained.

This seems to indicate that the difference in frequency response (between MIC1 and MIC2) has an effect. However, you must pay attention to what you are measuring. Previously (before taking into account the frequency response of the microphone), the following equation was used to measure H ₁ :

Here, the subscript n indicates that this calculation is only performed between windows containing only noise. However, considering the above equation, it can be seen that in the absence of a signal, the following is measured at the microphone:

Therefore, H ₁ may be calculated as follows.

However, B (z) and A (z) are not taken into account when calculating H ₁ (z). Therefore, what is actually measured is simply the ratio of the signal at each microphone.

Here, H ₁ ^~ represents a measured response, H ₁ represents the actual response. The calculation of H ₂ is the same, and the following equation is obtained.

Substituting H ₁ ^- and H ₂ ^- for the above-described formula of S, the following formula is obtained.

Or

This is the same as before if the frequency response of the microphone is not included. Here, S (z) A (z ) is replaces the S (z), value ^{_(H 1} ~ _(z) and _H ² ~ (z)) the actual _H 1 (z) and _H 2 (z ). That is, this algorithm is theoretically independent of the microphone and associated filter and amplifier response.

However, in the implementation, it is assumed that H ₂ = Cz ⁻¹ (C is a constant).

Therefore, the following result is obtained.

This depends on the unknown B (z) and A (z). This can cause problems when the frequency responses of the microphones are quite different, especially if they are used frequently and cheaply. This means that the data from MIC2 must be compensated to have a proper relationship to the data coming from MIC1. This can be done by recording at both MIC1 and MIC2 a broadband signal from a signal source located remotely and in the expected orientation for the actual signal (the actual signal source can also be used). Next, a discrete Fourier transform (DFT) is calculated for each microphone signal, and the degree of conversion in each frequency bin is calculated. Next, the degree of DFT for MIC2 in each frequency bin is set to be equal to the value obtained by multiplying the degree of DFT for MIC1 by C. If M ₁ [n] represents the degree in the nth frequency bin of the DFT for MIC _1, the coefficients to be multiplied with M ₂ [n] are as follows:

Next, an inverse transform is applied to the new MIC2DFT amplitude using the previous MIC2DFT phase. In this way, MIC2 is resynchronized and

To be correct when only audio is occurring. This transformation can also be performed in the time domain using a filter that emulates the characteristics of F as closely as possible (eg, using the Matlab function FFT2.M with the calculated value of F [n] A simple FIR filter can be constructed).

  FIG. 11A is a plot of the frequency response difference (percent) between microphones (4 centimeters apart) before compensation. FIG. 11B is a plot of the frequency response difference (percent) between microphones (4 centimeters apart) after DFT compensation. FIG. 11C is a plot of the frequency response difference (percentage) between microphones (4 centimeters apart) after time domain filter compensation. These plots show the effectiveness of the compensation method described above. That is, even with two very inexpensive omnidirectional or unidirectional microphones, both compensation methods restore the correct relationship between the microphones.

The transformation should be relatively constant as long as the relative amplification and filtering process is unchanged. Thus, it is possible that the compensation process only needs to be performed once in the manufacturing stage. However, if necessary, the algorithm can be set to operate assuming that H ₂ = 0 until the system is used where there is little noise and a strong signal. The compensation factor F [n] can then be calculated and used thereafter from that point on. This calculation does not add undue distortion to the noise cancellation algorithm because noise cancellation is not necessary when there is little noise. Also, the noise cancellation coefficient can be updated whenever a noise environment is preferred for maximum accuracy.

  Each of the blocks and steps shown in the figures presented herein may include a series of operations that need not be described here. Those skilled in the art can create routines, algorithms, source code, microcode, program logic arrays, or otherwise implement the invention based on the drawings and detailed descriptions presented herein. . The routines described herein can include any of the following, or a combination of one or more of the following. Routines stored in non-volatile memory (not shown) forming part of one or more associated processors, routines implemented using conventional program logic arrays or circuit elements, removable such as disks Routines stored on media, routines downloaded from a server or stored in the client, and chips such as electrically erasable programmable read only memory (“EEPROM”) semiconductor chips, several applications A pre-programmed routine wired to a circuit (ASIC) or digital signal processing (DSP) integrated circuit.

  Unless specifically required by context, throughout this description and the claims, the words “comprise”, “comprising”, etc. are not inclusive or exhaustive, but are inclusive It shall be interpreted in meaning. In other words, it means “including but not limited to”. Words using the singular or plural number also include the plural or singular number, respectively. In addition, `` herein '', `` hereunder '', and similar words when used herein refer to the entire application and not to any particular part of the application. To do.

  The above description of illustrated embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. While specific embodiments and examples of the invention have been described herein for purposes of illustration, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the invention. The teachings of the present invention presented here can be applied not only to the data collection symbology reader described above, but also to other machine vision systems. Furthermore, further embodiments can be obtained by combining the elements and actions of the various embodiments described above.

  Any references or US patent applications cited herein are hereby incorporated by reference. If necessary, aspects of the present invention can be modified to employ these various cited systems, functions and concepts to yield further embodiments of the present invention.

FIG. 1 is a block diagram of a noise removal system in one embodiment. FIG. 2 is a block diagram illustrating a denoising algorithm in one embodiment assuming a single noise source and a direct path to the microphone. FIG. 3 is a block diagram illustrating a front end of an embodiment denoising algorithm generalized to n noise sources (these noise sources may be reflections or echoes of each other). It is. FIG. 4 is a block diagram illustrating the front end of an embodiment denoising algorithm in the general case where there are n distinct noise sources and signal reflections. FIG. 5 is a flow diagram of a noise removal method in one embodiment. FIG. 6 shows the results for an English-speaking American woman of one embodiment of the noise suppression algorithm in airport terminal noise, including many other speakers and public calls. FIG. 7 is a block diagram of a physical configuration for noise removal using unidirectional and omnidirectional microphones in the embodiments of FIG. 2, FIG. 3, and FIG. FIG. 8 is a diagram illustrating a noise cancellation microphone configuration including two omnidirectional microphones in one embodiment. FIG. 9 is a graph showing the relationship between request C and distance in the embodiment of FIG. FIG. 10 is a block diagram of the front end of a denoising algorithm in one embodiment when two microphones have different response characteristics. FIG. 11A is a graph showing the difference (in percent) in frequency response between microphones (4 centimeters away) before compensation. FIG. 11B is a graph illustrating the frequency response difference (in percent) between microphones (4 centimeters apart) after DFT compensation in one embodiment. FIG. 11C is a graph showing the difference (in percent) in frequency response between microphones (4 centimeters apart) after time domain filter compensation in an alternative embodiment.

Claims (33)

A method for removing noise from an electronic signal,
Receiving a plurality of acoustic signals at a first receiving device;
Receiving a plurality of acoustic signals at a second receiving device, the plurality of acoustic signals comprising at least one signal generated by at least one noise source and at least one voiced generated by at least one signal source; And wherein the at least one signal source comprises a speaker, and the relative positions of the signal source, the first receiving device, and the second receiving device are constant and known;
Receiving psychological information associated with the speaker's human vocalization activity, including presence or absence of vocalization activity;
Generating at least one first transfer function representing the plurality of acoustic noise signals if it is determined that there is no vocal activity in the plurality of acoustic signals over at least a specified period;
Generating at least one second transfer function representing the plurality of acoustic signals when it is determined that there is voiced information in the plurality of acoustic signals in at least the specified period;
Removing at least one noise from the plurality of acoustic signals using at least one combination of the at least one first transfer function and the at least one second transfer function to generate at least one noise cancellation data stream; When,
A method consisting of:   The method of claim 1, wherein the first receiving device and the second receiving device each comprise a microphone selected from the group consisting of a unidirectional microphone and a unidirectional microphone.   The method of claim 1, wherein the plurality of acoustic signals are received in the form of discrete time samples, the first receiving device and the second receiving device being located a distance "d" apart, d Corresponds to n discrete time samples.   2. The method of claim 1, wherein the at least one second transfer function is fixed as a function of a difference between an amplitude of signal data at the first receiving device and an amplitude of signal data at the second receiving device. ,Method.   The method of claim 1, wherein removing noise from the plurality of acoustic signals comprises using a direction and distance from the at least one first receiving device to the at least one signal source.   The method of claim 1, wherein the frequency response of each of the at least one receiving device and the at least one second receiving device is different, compensating signal data from the at least one second receiving device, and A method having a proper relationship to signal data from at least one first receiving device.   7. The method of claim 6, wherein compensating the signal data from the at least one second receiving device is the at least one signal at the at least one first receiving device and the at least one second receiving device. Recording a broadband signal from a signal source located at a distance and orientation expected for the signal from the source.   The method of claim 6, wherein the compensation of signal data from the at least one first receiving device comprises frequency domain compensation. 9. The method of claim 8, wherein the frequency compensation is
Calculating a frequency transform for signal data from each of the at least one first receiving device and the at least one second receiving device;
Calculating the degree of frequency conversion in each frequency bin;
Setting, for each frequency, the degree of frequency conversion for signal data from the at least one second receiving device to a value relating to the degree of frequency conversion for signal data from the at least one receiving device;
A method consisting of:   The method of claim 6, wherein the compensation of signal data from the at least one first receiving device comprises time domain compensation. The method of claim 6, further comprising:
Initially setting the at least one second transfer function to zero;
Calculating a compensation factor when the at least one noise signal is relatively small with respect to the at least one voiced signal;
Including a method.   The method of claim 1, wherein the plurality of acoustic signals includes at least one reflection of the at least one noise signal and at least one reflection of the at least one voiced signal.   The method of claim 1, wherein the step of receiving psychological information is selected from the group consisting of an acoustic microphone, a radio frequency device, an electronic language recorder, an ultrasound device, an acoustic throat microphone, and an airflow detector. A method comprising receiving psychological data associated with a person's utterance using at least one detector.   The method of claim 1, wherein generating the at least one first transfer function and the at least one second transfer function comprises using at least one technique selected from the group consisting of adaptive techniques and recursive techniques. Including. A system for removing noise from an acoustic signal,
At least one receiver,
At least one signal receiver configured to receive at least one acoustic signal from a signal source;
At least one noise receiver configured to receive at least one noise signal from a noise source, relative to the signal source, the at least one signal receiver, and the at least one noise receiver. Said at least one receiver, wherein the general position is constant and known;
At least one sensor for receiving psychological information associated with human vocal activity;
At least one processor coupled between the at least one receiver and the at least one sensor and generating a plurality of transfer functions, wherein there is no voicing information in the plurality of acoustic signals for at least a specified period of time; And at least one first transfer function representing the plurality of acoustic noise signals is generated and responds to the determination that utterance information is present in the plurality of acoustic signals in at least the specified period. And generating at least one second transfer function representing the plurality of acoustic signals, and using the at least one combination of the at least one first transfer function and the at least one second transfer function. A processor that removes noise from the acoustic signal of
System with.   16. The system of claim 15, wherein the at least one sensor includes at least one radio frequency ("RF") interferometer that detects tissue motion associated with human speech.   16. The system of claim 15, wherein the at least one sensor is at least one selected from the group consisting of an acoustic microphone, a radio frequency device, an electronic language recorder, an ultrasonic device, an acoustic throat microphone, and an airflow detector. A system that includes a sensor. 16. The system of claim 15, wherein the at least one processor is
Dividing the acoustic data of the at least one acoustic signal into a plurality of subbands;
Removing noise from each of the plurality of subbands using at least one combination of the at least one first transfer function and the at least one second transfer function to generate a plurality of noise canceled acoustic data streams;
Combining the plurality of noise canceling acoustic data streams to generate the at least one noise canceling acoustic data stream;
Configured as a system.   16. The system of claim 15, wherein the at least one signal receiver and the at least one noise receiver are each a microphone selected from the group consisting of a unidirectional microphone and an omnidirectional microphone. A signal processing system coupled between at least one user and at least one electronic device,
At least one first receiving device configured to receive at least one acoustic signal from a signal source;
At least one second receiving device configured to receive at least one noise signal from a noise source, the signal source, the at least one first receiving device, and the at least one second receiving device. A second receiving device, the relative position of which is constant and known;
At least one noise cancellation subsystem for removing noise from the acoustic signal, comprising:
At least one processor coupled between the at least one first receiver and the at least one second receiver;
At least one sensor coupled to the at least one processor, wherein the at least one sensor is configured to receive psychological information associated with human vocal activity, the at least one processor Generates a plurality of transfer functions and generates at least one first transfer function representing the plurality of acoustic noise signals in response to determining that there is no utterance information in the plurality of acoustic signals for at least a specified period of time. And at least one second transfer function representing the plurality of acoustic signals is generated in response to determining that utterance information is present in the plurality of acoustic signals in at least the specified one period, The plurality of acoustic signals using at least one combination of a first transfer function and the at least one second transfer function Removing Luo noise, and noise cancellation subsystem,
A signal processing system.   21. The signal processing system of claim 20, wherein the first receiving device and the second receiving device are each a microphone selected from the group consisting of a unidirectional microphone and an omnidirectional microphone.   21. The signal processing system of claim 20, wherein the at least one acoustic signal is received in the form of discrete time samples, wherein the first receiving device and the second receiving device are located a distance "d" apart, d Is a signal processing system corresponding to n discrete time samples.   21. The signal processing system of claim 20, wherein the at least one second transfer function is fixed as a function of a difference between an amplitude of signal data at the first receiving device and an amplitude of signal data at the second receiving device. A signal processing system.   The signal processing system of claim 20, wherein removing noise from the at least one acoustic signal comprises using a direction and distance from the at least one first receiving device to the at least one signal source. Signal processing system.   21. The signal processing system according to claim 20, wherein each of the at least one first receiving device and the at least one second receiving device has a different frequency response, and the signal data from the at least one second receiving device is obtained. A signal processing system that has been compensated to have a proper relationship to signal data from the at least one first receiving device.   26. The signal processing system of claim 25, wherein compensating the signal data from the at least one second receiving device is a sound source located at an expected distance and orientation relative to a signal from the at least one signal source. A broadband signal from the at least one first receiving device and the at least one second receiving device.   26. The signal processing system of claim 25, wherein compensating signal data from the at least one second receiving device includes frequency domain compensation. 28. The signal processing system of claim 27, wherein the frequency compensation is
Calculating a frequency transform for signal data from each of the at least one first receiving device and the at least one second receiving device;
Calculating the degree of frequency conversion in each frequency bin, and for each frequency, calculating the degree of frequency conversion for the signal data from the at least one second receiving device for the signal data from the at least one receiving device; Setting a value relating to the degree of frequency conversion;
A signal processing system comprising:   26. The signal processing system of claim 25, wherein compensating signal data from the at least one second receiving device includes time domain compensation. 26. The signal processing system of claim 25, further comprising compensation.
Initially, the at least one second transfer function is set to 0,
Calculating a compensation factor when the at least one noise signal is relatively small with respect to the at least one acoustic signal;
A signal processing system comprising:   21. The signal processing system of claim 20, wherein the at least one acoustic signal includes at least one reflection of the at least one noise signal and at least one reflection of the at least one acoustic signal.   21. The signal processing system of claim 20, wherein the receiving of psychological information is selected from the group consisting of an acoustic microphone, a radio frequency device, an electronic language recorder, an ultrasound device, an acoustic throat microphone, and an airflow detector. A signal processing system comprising: using at least one detector to receive psychological data associated with human speech.   21. The signal processing system of claim 20, wherein generating the at least one first transfer function and the at least one second transfer function uses at least one technique selected from the group consisting of adaptive techniques and recursive techniques. A signal processing system.