KR20040077661A - Method and apparatus for removing noise from electronic signals - Google Patents

Abstract

A method and system for removing acoustic noise from human speech is described. Acoustic noise is removed regardless of noise type, amplitude, or direction. The system includes a processor coupled between the microphone and the voice activity detection (VAD) element. The processor runs a noise cancellation algorithm that generates transfer functions. The processor receives acoustic data from the microphone and data from the VAD. When the VAD indicates voice activity, and when the VAD indicates no voice activity, the processor generates various various transfer functions. The transfer function is used to generate a noise canceled data stream.

Description

METHOD AND APPARATUS FOR REMOVING NOISE FROM ELECTRONIC SIGNALS

In a typical acoustic device, sound from a person is recorded or stored and transmitted to a receiver in another location. In the user environment, there may be one or more kinds of noise sources that contaminate the target signal (user's voice) with unwanted acoustic noise. This makes it difficult and even impossible for the receiver to understand the user's voice (whether it is a person or a machine). This is a particular problem as the current proliferation of portable communication devices such as cellular phones and PDAs expands. There are many ways to suppress this noise generation, but these methods require long computational time, cumbersome hardware, large distortion of the target signal, or lack of useful performance. Many of these methods are described in textbooks such as Vaseghi's "Advanced Digital Signalal Processing and Noise Reduction" in ISBN 0-471-62692-9.

This application is the Serial Split Application (CIP) of US Patent Application Serial No. 09 / 905,361, filed Jul. 12, 2001, the contents of which are incorporated herein by reference. This patent application claims priority to US patent application Ser. No. 60 / 332,202, filed November 21, 2001.

The present invention relates to the field of mathematical methods and electronic systems for the removal or suppression of unwanted acoustic noise in sound transmission or recording.

1 is a block diagram of a noise cancellation system of one embodiment.

2 is a block diagram of an noise cancellation algorithm of one embodiment, assuming a direct path to a microphone and a single noise source.

3 is a block diagram of the front end of an embodiment noise cancellation algorithm generalized to n distinct noise sources.

4 is a front end block diagram of the noise cancellation algorithm of one embodiment of the most generalized case where n distinct noise sources and signal reflections exist.

5 is a flow chart of a noise cancellation method of an embodiment.

FIG. 6 shows the results of an embodiment noise suppression algorithm for an American woman using English language at an airport terminal noise including a number of speakers and announcements.

7 is a block diagram of a physical implementation for noise cancellation using unidirectional and omnidirectional microphones in the embodiments of FIGS. 2, 3, and 4;

8 is a diagram of a noise canceling microphone structure including two omnidirectional microphones, in one embodiment.

9 is a diagram of a plot of C required for distance in the embodiment of FIG. 8;

FIG. 10 is a block diagram of the front end of a noise canceling algorithm in an embodiment where the two microphones exhibit different response characteristics. FIG.

11A shows a plot of the difference in frequency response (percent) between microphones (at a distance of 4 cm) before compensation.

11B shows a plot of the difference in frequency response (percent) between microphones (at a distance of 4 cm) after DFT compensation in an embodiment.

11C shows a plot of the difference in frequency response (percent) between microphones (at a distance of 4 cm) after time domain filter compensation, in another embodiment.

1 is a block diagram of one embodiment of a noise reduction system that utilizes knowledge at the time of speech generation derived from physiological information when speaking. The system includes a sensor 10 and a microphone 10 that provide signals to one or more processors 30. The processor includes a noise canceling subsystem or algorithm 40.

2 is a block diagram of an embodiment noise cancellation system / algorithm, assuming a single noise source and a direct path to a microphone. The noise reduction system diagram is a graphical representation of the process of one embodiment, with a single signal source 100 and a single noise source 101. This algorithm uses, but is not limited to, two microphones, "signal" microphones MIC1, 102 and "noise" microphones MIC2, 103. Assume that MIC1 captures most of the signal with some noise, and that MIC2 captures most of the noise with some signal. This is a common setting seen in conventional improved acoustic systems. Data from the signal source 100 to MIC1 is represented by s (n), data from the signal source 100 to MIC2 is s ₂ (n), and data from the noise source 101 to MIC2 is n (n). The data from the noise source 101 to MIC1 is represented by n ₂ (n). Similarly, data from MIC1 to noise canceling element 105 is represented by m ₁ (n) and data from MIC2 to noise canceling element 105 is represented by m ₂ (n), where s (n) is the source ( Display a separate sample of the analog signal from 100).

The noise canceling element also receives a signal from the voice activity detection (VAD) element 104. The VAD element 104 may determine the point in time at which the speaker is speaking using physiological information. In various embodiments, the VAD device includes an RF device, an electroglotto graph, an ultrasound device, an acoustic throat microphone, and an airflow detector.

Assume that the transfer function from signal source 100 to MIC1 and from noise source 101 to MIC2 has a unit value. The transfer function from signal source 100 to MIC2 is represented by H2 (z) and the transfer function from noise source 101 to MIC1 is represented by H1 (z). Assuming a unit transfer function does not impair the generality of this algorithm, because the actual relationship between signal, noise, and microphone is only a simple ratio, and these ratios are simply redefined in this way.

In conventional noise cancellation systems, information from MIC2 is used to attempt to remove noise from MIC1. However, the unspoken assumption is that voice activity detection (VAD) is by no means complete, and therefore must be exercised carefully so that noise cancellation does not remove much of the signal with noise. However, if the VAD 104 is assumed to be complete and equals 0 when there is no speech produced by the user at all and equals 1 when the speech is generated, then a noticeable improvement in noise cancellation can be made.

When analyzing the direct path and the single noise source for the microphone with reference to Figure 2, the total acoustic information flowing into MIC1 is represented by m ₁ (n). The total acoustic information flowing into MIC2 is likewise represented by m ₂ (n). In the z (digital frequency) domain, they are represented by M ₁ (z) and M ₂ (z). so,

M ₁ (z) = S (z) + N ₂ (z)

M ₂ (z) = N (z) + S ₂ (z)

ego

N ₂ (z) = N (z) H ₁ (z)

S ₂ (z) = S (z) H ₂ (z)

next,

M ₁ (z) = S (z) + N (z) H ₁ (z)

M ₂ (z) = N (z) + S (z) H ₂ (z) Equation 1

This is a common case for all two microphone systems. In a real system, there will always be some noise leakage with MIC1 and some signal leakage with MIC2. Equation 1 has four unknown relationships and only two known relationships, and therefore cannot be solved explicitly.

However, there is another way to solve some of the unknown relationships in equation 1. This analysis begins with the examination when no signal is generated. That is, the test starts when the signal from the VAD element is equal to 0 and no speech is generated. In this case, s (n) = S (z) = 0 and equation 1 is

M _1n (z) = N (z) H ₁ (z)

M _2n (z) = N (z)

In this case, the subscript n in the M variable means that only noise is received.

This leads to equation 2 below.

M _1n (z) = M _2n (z) H ₁ (z)

H ₁ (z) = M _1n (z) / M _2n (z) Equation 2

When the system is confident that only noise is being received, H ₁ (z) can be calculated using either the available system identification algorithm and the microphone output. This calculation can be performed adaptively so that the system can respond to noise changes.

A solution to one of the unknown values of equation 1 is now possible. Another unknown value H ₂ (z) can be determined using the case where VAD is equal to 1 and voice is being generated. This happens, but if the microphone's recent history shows a low level of noise, then n (s) = N (z) ~ 0. Equation 1 is then simplified to

M _1s (z) = S (z)

M _2s (z) = S (z) H ₂ (z)

This again,

M _2s (z) = M _1s (z) H ₂ (z)

H ₂ (z) = M _2s (z) / M _1s (z)

This is the inverse of the H ₁ (z) operation. However, it should be noted that several different inputs are used. Now only signals are generated, but in the past only noise was generated. When calculating H ₂ (z), the values calculated for H ₁ (z) remain constant, and vice versa. Thus, it is assumed that when one of H ₁ (z) and H ₂ (z) is computed, the other one does not substantially change.

After calculating H ₁ (z) and H ₂ (z), they are used to remove noise from the signal. Equation 1 is rewritten as

S (z) = M ₁ (z)-N (z) H ₁ (z)

N (z) = M ₂ (z)-S (z) H ₂ (z)

S (z) = M ₁ (z)-[M ₂ (z) -S (z) H ₂ (z)] H ₁ (z)

S (z) [1-H ₂ (z) H ₁ (z)] = M ₁ (z)-M ₂ (z) H ₁ (z)

To get S (z) we can substitute N (z) as

S (z) = [M ₁ (z) -M ₂ (z) H ₁ (z)] / [1-H ₂ (z) H ₁ (z)] Equation 3

If the transfer functions H ₁ (z) and H ₂ (z) can be described with sufficient accuracy, the noise can be completely eliminated and the original signal can be recovered. This is true regardless of the amplitude / frequency characteristics of the noise. The only assumption is that VAD is perfect, H ₁ (z) and H ₂ (z) are sufficiently accurate, and the other one does not change substantially when one of H ₁ (z) and H ₂ (z) is being calculated. to be. In fact, these assumptions turned out to be reasonable.

The noise cancellation algorithm described herein is easily generalized to include any number of noise sources. 3 is a front end block diagram of an embodiment of a noise cancellation algorithm, generalized to n distinct noise sources. These distinct noise sources may reflect or echo each other, but are not limited thereto. Several noise sources are shown, with each noise source having a transfer function or path for each microphone. The previously named path H ₂ is again marked as H ₀ , making the path representation of noise source 2 to MIC1 more convenient. The output of each microphone, when converted to the z domain, is as follows.

M ₁ (z) = S (z) + N ₁ (z) H ₁ (z) + N ₂ (z) H ₂ (z) + ... N _n (z) H _n (z)

M ₂ (z) = S (z) H ₀ (z) + N ₁ (z) G ₁ (z) + N ₂ (z) G ₂ (z) + ... N _n (z) G _n (z )

In the absence of any signal (VAD = 0) (z is suppressed for clarity)

M _1n = N ₁ H ₁ + N ₂ H ₂ + ... N _n Hn Equation 4

M _2n = N ₁ G ₁ + N ₂ G2 + ... N _n G _n equation

Now the new transfer function can be defined as H1 (z).

= M _1n / M _12n = (N ₁ H ₁ + N ₂ H ₂ + ... N _n Hn) / (N ₁ G ₁ + N ₂ G2 + ... N _n G _n ) Equation 6

Depends on the noise source and its transfer function and can be calculated at any moment when no signal is transmitted. Once again, the n subscript of the microphone input indicates that only noise is detected, and the s subscript indicates that only a signal is being received by the microphone.

If you verify equation 4 under the assumption that there is no noise,

M _1s = S

M _2s = SH ₀

H ₀ can be solved as before using any available transfer function calculation algorithm. Mathematically,

H0 = M _2s / M _1s

Defined in equation 6 Rewrite equation 4 using

= (M ₁ -S) / (M ₂ -SH ₀ ) Equation 7

If you solve for S,

S = (M ₁ -M ₂ ) / (1-H ₀ ) Equation 8

This is equal to equation 3. H ₀ occupies the place of H ₂ Occupies H ₁ . Thus, the noise cancellation algorithm is still mathematically valid for any number of noise sources, including multiple echoes of noise sources. In addition, H ₀ and This can be estimated with very high accuracy and noise can be completely eliminated if the previous assumption of only one path from the signal to the microphone is maintained.

The most common case is having multiple noise sources and multiple signal sources. 4 is a block diagram of the front end of the noise cancellation algorithm of one embodiment in the most common case where there are n distinct noise sources and signal reflections. The reflected signal enters the two microphones. This is the most common case. Because the reflection of the noise source into the microphone can be accurately modeled as a simple additional noise source. For clarity, the direct path from the signal to MIC2 was changed from H ₀ (z) to H ₀₀ (z), and the reflected paths towards microphones 1 and 2 are labeled H ₀₁ (z) and H ₀₂ (z), respectively. do.

The input to the microphone is as follows.

M ₁ (z) = S (z) + S (z) H ₀₁ (z) + N ₁ (z) H ₁ (z) + N ₂ (z) H ₂ (z) + ... N _n (z H _n (z)

M ₂ (z) = S (z) H ₀₀ (z) + S (z) H ₀₂ (z) + N ₁ (z) G ₁ (z) + N ₂ (z) G ₂ (z) + .. .N _n (z) G _n (z)

... Equation 9

When VAD = 0, the input is as follows (z is suppressed again):

M _1n = N ₁ H ₁ + N ₂ H ₂ + ... N _n H _n

M _2n = N ₁ G ₁ + N ₂ G ₂ + ... N _n G _n

This is equal to equation 5. Thus, in equation 6 The calculation does not change as expected. Checking for the absence of noise, Equation 9 is simplified to

M _1s = S + SH ₀₁

M _2s = SH ₀₀ + SH ₀₂

This is shown below Leads to justice.

= M _2s / M _1s = (H ₀₀ + H ₀₂ ) / (1 + H ₀₁ ) Equation 10

(As in equation 7) Rewrite equation 9 using the definition of

= [M ₁ -S (1 + H ₀₁ )] / [M ₂ -S (H ₀₀ + H ₀₂ )] equation 11

Due to arithmetic operations,

S (1 + H _01- (H ₀₀ + H ₀₂ )) = M ₁ -M ₂

S (1 + H ₀₁ ) [1- (H ₀₀ + H ₀₂ ) / (1 + H ₀₁ )] = M ₁ -M ₂

S (1 + H ₀₁ ) [1- ] = M ₁ -M ₂

therefore,

S (1 + H ₀₁ ) = M ₁ -M ₂ /(One- ) Equation 12

Equation 12 gives H ₀ By substituting and adding the argument of (1 + H ₀₁ ) to the left side. An additional factor cannot be solved directly by S in this situation, but a solution may occur for the signal that adds all the echoes of the signal. This is not such a poor situation as there are many existing methods to deal with echo suppression, and even if the echoes are not suppressed, they do not easily affect the speech decipherability to any significant degree. More complex The calculation of is necessary to describe the signal echo of microphone 2 acting as a noise source.

5 is a flow chart of a noise canceling method of an embodiment. In operation, an acoustic signal is received (step 502). In addition, physiological information associated with voice activity is received (step 504). A first transfer function representing the acoustic signal is calculated based on the lack of speech information in the acoustic signal for one or more specified time periods (step 506). A second transfer function representing the acoustic signal is calculated by determining whether speech information is present in the acoustic signal for one or more specified time periods (step 508). Noise is removed from the acoustic signal using one or more combinations of the first transfer function and the second transfer function to generate a noise canceled acoustic data stream (step 510).

The noise cancellation algorithm is described from the simplest case of a single noise source with a direct path to multiple noise sources with reflections and echoes. The algorithm appears to work under any environmental condition. The type and amount of noise and It does not matter if a good estimate is made for, and does not matter if one does not change when the other is calculated. If the user environment is an echo, they can be compensated if they come from a noise source. If a signal echo is also present, it will affect the cleaned signal, but in most circumstances this effect should be negligible.

In operation, the algorithm of one embodiment shows excellent results in relation to various noise types, amplitudes, and orientations. However, approximations and adjustments must always be made when moving from a mathematical concept to a real environment. One assumption is made in equation 3, where H ₂ (z) is assumed to be small, and therefore H ₂ (z) H ₁ (z) to 0. Thus, equation 3 is summarized as follows.

S (z) to M ₁ (z) -M ₂ (z) H ₁ (z)

This means that only H ₁ (z) needs to be calculated, which speeds up the process and significantly reduces the number of operations involved. By appropriate selection of the microphone, this approximation is easily realized.

Another approximation relates to the filter used in one embodiment. Actual H ₁ (z) has poles and zeros, and all finite impulse response (FIR) filters are used for stability and simplicity. If you have enough taps (around 60), the approximation to the actual H ₁ (z) is very good.

In subband selection, the wider the frequency range over which the transfer function is to be calculated, the more difficult it is to calculate accurately. Thus, the acoustic data is divided into 16 subbands, with a minimum frequency of 50 Hz and a maximum frequency of 3700 Hz. A noise cancellation algorithm is then applied to each subband and 16 noise canceled data streams are recombined to yield noise canceled acoustic data. This works very well, but any other combination of subbands (eg subbands evenly spaced 4, 6, 8, 32) can be used and found to work as well.

The amplitude of the noise was constrained in one embodiment so that the microphone used was not saturated (ie, operated outside the linear response range). It is important that the microphone behaves linearly to ensure optimal performance. Even with this limitation, very low signal-to-noise ratio (SNR) signals can be noise canceled (<-10dB).

H ₁ (z) is calculated every 10 milliseconds using a common adaptive transfer function of least mean square (LMS). This is described in the book "Adaptive Signal Processing" (1985) by Widrow and Stearns, published by Prentice-Hall and published in ISBN 0-13-004029-0.

The VAD for one embodiment is obtained from a high frequency (RF) sensor and two microphones, showing very high accuracy (> 99%) for speech and non-speech speech. In one embodiment, the VAD detects, but is not limited to, tissue motion related to human speech generation using a radio frequency (RF) interferometer. Thus, it is completely free from acoustic noise and can function in any acoustic noise environment. Simple energy measurements of the RF signal can be used to determine if speech speech is occurring. Non-speech speech can be determined using existing acoustic-based methods, using RF sensors or similar voicing sensors, or in a similar manner to speech sections determined through a combination thereof. Since there is far less energy in non-speech speech, its activity accuracy is not as important as speech speech.

The algorithm of one embodiment may be implemented with reliable speech and non-speech speech being reliably detected. It is also useful to repeat that the noise cancellation algorithm does not depend on how VAD is obtained and is only accurate, especially for speech speech. If speech is not detected and training occurs in speech, subsequent noise canceled acoustic data may be distorted.

Data is collected in four channels. One channel is for MIC1, one is for MIC2, and the other two are for high-frequency sensors that detect tissue movement related to speech speech. Data was sampled simultaneously at 40 kHz and digitally filtered down to 8 kHz. High sampling rates have been used to reduce aliasing that can occur during analog-to-digital conversion. Four-channel National Instruments A / D boards were used with Labview for data capture and storage. The data is read into a C program and de- noised 10 milliseconds at a time.

FIG. 6 shows the results of an embodiment noise suppression algorithm for a woman speaking American English in the presence of airport terminal noise, including several speakers and announcements. The speaker is numbering 406-5562 under normal airport terminal noise. Acoustic data was de- noised 10 milliseconds at a time, and 10 milliseconds of data were pre-filtered at 50 to 3700 kHz before noise was removed. A noise reduction of approximately 17 dB is evident. No post filtering is done on this sample. Thus, all noise reduction is due to the algorithm of one embodiment. It is clear that the algorithm is adjusted for noise instantaneously and can remove very difficult noises caused by other speakers. Several different kinds of noise were examined and yielded similar results, including noise in the streets, helicopter sounds, music, sine waves, and the like. Also, the orientation of the noise can be changed without significantly altering the noise suppression performance. Finally, the distortion of the processed speech is very low, ensuring good performance for speech recognition engines and human receivers.

The noise cancellation algorithm of one embodiment has been shown to function under any environmental condition. The type and magnitude of noise and It is not important if we estimate If the user environment is in the presence of an echo, it can be compensated if it comes from a noise source. If signal echoes are also present, they will affect the processed signal, but in most circumstances the effects should be negligible.

7 is a block diagram of a physical implementation of noise cancellation using, in FIGS. 2, 3 and 4, one way microphone M2 for noise and omnidirectional microphone M1 for speech. As mentioned above, the path from speech to noise microphone (MIC 2) is approximated to zero. This approximation is implemented by carefully placing the omni- and unidirectional microphones. This works very well when the noise is directed opposite the signal position (20-40 dB noise suppression). However, when the noise source is on the same side as the speaker (noise source N2), its performance is only 10-20dB of noise suppression. This inhibition can be attributed to the steps handled to ensure the fact that H2 is close to zero. These steps involved the use of one-way microphones for noise microphones (MIC2). So there is a very weak signal in the noise data. As the one-way microphone cancels sound information coming from a particular direction, the one-way microphone also cancels noise received in the same direction as the speech. This may limit the ability of an adaptive algorithm to characterize and remove noise at one location, such as N2. The same effect occurs when a unidirectional microphone is used for the speech microphone M1.

However, if a single microphone M ₂ is replaced by another omnidirectional microphone, a significant amount of signal is captured by M ₂ . This is contrary to the assumption that H ₂ is zero, and as a result a significant amount of signal is removed during speech, resulting in noise cancellation and signal rejection. This is unacceptable if signal distortion must be kept to a minimum. In order to reduce the distortion, the value for H ₂ is calculated as a result. However, the value for H ₂ cannot be calculated when there is noise, or the noise is misclassified as speech and is not removed.

In the case of acoustic-only microphone arrangements, a small two microphone arrangement can be a solution to the problem. FIG. 8 shows the structure of a noise canceling microphone including two omnidirectional microphones, in one embodiment. The same effect can be achieved by using two unidirectional microphones in the same direction (toward the signal source). Another embodiment uses one unidirectional microphone and one omnidirectional microphone. This concept is to capture similar information from the sound source in the direction of the signal source. The relative position of the signal source and the two microphones is fixed and known. H ₂ can be fixed in the form of C _z ^-n by placing the microphones so that they are separated by d distances corresponding to n separate samples, and by placing the speakers on the axis of the array. Where C is the difference in signal data amplitude between signal data M ₁ and M ₂ . In the description below, n = 1 is assumed, although any integer except zero is used. For causality, the use of positive integers is preferred. As the height of the spherical pressure source changes to 1 / r, not only the direction specification of the source, but even its distance is allowed. The required C can be estimated by

9 is a plot of the distance to C required in the embodiment of FIG. 8. Asymptotes can be seen at C = 1.0 and C reaches 0.9 at approximately 39 cm. Slightly larger, at about 60 cm C reaches 0.94. At distances encountered mainly between handsets and receivers (4 to 12 cm), C is approximately between 0.5 and 0.75. This is a difference of about 19% to 44% for a noise source located approximately 60cm. And it is obvious that most noise sources are located farther than this. Thus, systems using this configuration can distinguish noise and signals quite efficiently, even when they have a similar direction.

To determine the result for C's poor noise canceling estimate, assume C = nC ₀ , where C is an approximation and C ₀ is the actual value of C. Using these signal definitions from above,

Assume H ₂ (z) is very small, so the signal is roughly

This is true if there is no speech. Because, by definition, H2 = 0. However, if speech is occurring, H2 is not 0 and if Cz ^-1 is set,

This can be rewritten as:

The final element of the denominator determines the error based on C's poor estimate. This element is named E and is

Since z ⁻¹ H ₁ (z) is just a filter, its magnitude is always positive. Thus, the change in signal magnitude calculated by E depends entirely on (1-n).

There are two possible errors: C is depreciated (n <1) and C is raised (n> 1). In the first case, C is evaluated smaller than it actually is, or the signal is closer than the estimate. In this case (1-n) is positive, so E is positive. In this case the denominator is too large and the signal is too small. This represents signal-rejection. In the second case, the signal is further away than the estimate, and E is negative, which makes S too large. In this case, noise cancellation is insufficient. Because very low signal distortion is desirable, the estimate will deviate towards the C's raise estimate.

These results also show that noise located at the same solid line angle as the signal (direction from M ₁ ) will be substantially removed as the C changes between the signal position and the noise position. Thus, when using a handset with M _1d approximately 4 cm away from the mouth, the required C is about 0.5 and for noise at about 1 meter C is about 0.96. Thus, in noise, an estimate of C = 0.5 means that C has been devalued for this noise and will be removed. The amount of removal will depend directly on (1-n). This algorithm uses the direction and range for the signal to separate the signal from noise.

One discussion that occurs includes the stability of this technique. Specifically, as the need arises to calculate the reciprocal of 1-H ₁ H ₂ at the beginning of each voiced fragment, the deconvolution of (1-H ₁ H ₂ ) raises the issue of stability. . This helps to reduce the computation time or instructions per cycle needed to implement this algorithm. Because you don't have to compute the inverse of all voiced windows, you only need to calculate the first one. This is because H ₂ is considered a constant. However, this would make the calculation of false positives difficult because every time a false positive is encountered, an inverse of 1-H ₁ H ₂ is required.

Fortunately, the choice of H ₂ eliminates the need for deconvolution. From the above description, the signal can be written as

This can be rewritten as:

or

However, since H ₂ (z) is in the form of Cz ⁻¹ , the sequence in the time domain is as follows.

This means that the current signal sample requires the current MIC 1 signal, the current MIC 2 signal, and the previous signal sample. This means that no deconvolution is needed, just a simple subtraction as before, followed by the old convolution. The increase in computation required will also be minimal. This is therefore easy to realize.

The effect of the difference in the microphone response in this embodiment can be seen by examining the configuration described with reference to Figures 2, 3 and 4, which only include these time transfer functions A (z) and B (z). This represents the frequency response of MIC 1 and MIC 2 together with the filtering and amplifying response. 10 is a block diagram of the front end of a noise cancellation algorithm in an embodiment in which two microphones have different response characteristics.

10 is a graphical representation of an embodiment of a process including a single signal source 1000 and a single noise source 1001. This algorithm uses two microphones. That is, signal microphone 1 (MIC1) and noise microphone 2 (MIC2) are included, but are not limited thereto. MIC1 is primarily assumed to capture a signal with some noise, and MIC2 is primarily assumed to capture noise with some signal. Data from the signal source 1000 to MIC1 is represented by s (n), where s (n) is a separate sample of the analog signal from the signal source 1000. Data from the signal source 1000 to MIC2 is represented by s ₂ (n). Data from the noise source 1001 to MIC2 is represented by n (n), and data from the noise source 1001 to MIC1 is represented by n ₂ (n).

Transfer function A (z) represents the frequency response of MIC1 along with its filtering and amplifying response. Transfer function B (z) expresses the frequency response of MIC2 along with the filtering and amplifying response. The output of transfer function A (z) is represented by m ₁ (n) and the output of transfer function B (z) is represented by m ₂ (n). Signals m ₁ (n) and m ₂ (n) are received by noise canceling element 1005 and operate on the signal to output a “cleaned speech”.

The term "frequency response of MIC X" then includes the coupling effect of the microphone and any amplification or filtering process that occurs during the data recording process at that microphone. Solve for signal and noise (suppress z for clarity)

If you replace the latter with the previous one at this time,

This seems to indicate that the difference in frequency response (between MIC1 and MIC2) is influential. However, attention should be paid to what is measured. Previously (before considering the microphone's frequency response), H ₁ was measured using

At this time, the n subscript indicates that this operation is implemented for the duration of the window containing only noise. However, when observing the above equation, it should be noted that in the absence of a signal, the microphone is measured as follows.

Therefore, H ₁ should be calculated as follows.

However, B (z) and A (z) are not taken into account when calculating H ₁ (z).

Therefore, what is actually measured is the ratio of the signals at each microphone.

At this time, Represents the measured response and H ₁ is the actual response. The calculation of H ₂ is similar. the results are as follow.

In the expression for S above, and If you substitute,

or

This is the same as the previous case when the frequency response of the microphone is not included. Where S (z) A (z) replaces S (z) and its value ( (z) and (z)) replaces actual H ₁ (z) and H ₂ (z). Thus, this algorithm is, in theory, microphone independent and independent of the associated filter and amplifier response.

In practice, however, it is assumed that H ₂ = C _z ^-1 , where C is a constant.

So the result is

This depends on the unknown B (z) and A (z). This can cause problems if the frequency response of the microphones is substantially different, which is a common case, especially for inexpensive microphones that are commonly used. This means that the data from MIC2 should be compensated for to have a proper relationship to the data received from MIC1. This can be implemented by recording the wideband signals of MIC1 and MIC2 from sources located in the direction and distance expected for the actual signal. The DFT (Discrete Fourier Transform) of each microphone signal is then calculated, and the magnitude of the transform at each frequency bin is also calculated. The magnitude of the DFT for MIC 2 in each frequency bin is determined to be equal to the product of the magnitude of the DFT for MIC 1 and C. If we represent the magnitude of the nth frequency bin of the DFT for this M ₁ [n], the element to multiply with M ₂ [n] is

Becomes

Using the previous MIC 2 DFT phase, an inverse transform is applied to the new MIC 2 DFT amplitude. In this way, MIC 2 is resynthesized,

The above relationship holds when only speech is occurring. This transformation can also be performed in the time domain, using a filter that emulates as close as possible to the characteristics of F. (For example, the Matlab function FFT2.M can be used with the calculated value of F [n] to design a suitable FIR filter.)

11A is a plot of the frequency response difference (percent) between microphones (spaced 4 cm apart) before compensation. 11B is a plot of the frequency response difference (percent) between microphones (spaced 4 cm apart) after DFT compensation. 11C is a plot of the frequency response difference (percent) between microphones (spaced 4 cm apart) after time domain filter compensation. This plot shows the effectiveness of the compensation method mentioned above. Thus, with the use of two very inexpensive omni / unidirectional microphones, the two compensation methods restore the correct relationship between the microphones.

The transformation should be relatively constant unless the relative amplification and filtering process changes. Thus, the compensation process only needs to be performed once in the manufacturing stage. However, if necessary, this algorithm can be set to operate under the assumption that H ₂ = 0 until the system is used in an atmosphere with very little noise and a strong signal. The compensation factor F [n] can then be calculated and used from this time. Since no noise cancellation is required when there is little noise, this calculation does not impose an improper burden on the noise cancellation algorithm. The noise cancellation coefficient can also be updated at any time when the noise environment is in favor of maximum accuracy.

Each of the blocks and steps depicted in the figures shown herein may comprise a series of actions that need not be described here on their own. Those skilled in the relevant art will be able to implement routines, algorithms, source code, microcode, program logic arrays, and may implement the inventions based on the detailed description and drawings described herein. The routines described herein may be provided by one or more of the following paragraphs, or by one or more combinations of the following paragraphs. It is stored in non-volatile memory (not shown) that forms the processor or part of the processor, or in removable media such as a disk, or downloaded from a server and stored in a client area, or a semiconductor chip such as an EEPROM. , ASIC, or DSP (Digital Signal Processing) integrated circuits are wired and preprogrammed.

Claims (33)

As a method for removing noise from an electrical signal, Receiving a plurality of acoustic signals at the first receiving element, Receiving a plurality of acoustic signals at a second receiving element, wherein the plurality of acoustic signals are one or more noise signals generated by one or more noise sources and one or more voices generated by one or more signal sources A signal, wherein the one or more signal sources comprise one speaker (or speaker), the relative positions of the signal source, the first receiving element, and the second receiving element being known and fixed, Receiving physiological information related to the voice activity of the speaker (or speaker), including whether or not voice activity is present, Determining that there is a lack of speech activity from the plurality of acoustic signals for one or more specified periods, generating one or more first transfer functions representing the plurality of acoustic noise signals, Determining that voice information is present in the plurality of acoustic signals during the one or more specified periods, generating one or more second transfer functions representing the plurality of acoustic signals, and One or more combinations of the one or more first transfer functions and the one or more second transfer functions to remove noise from a plurality of acoustic signals, thereby generating one or more noise canceled data streams. And removing the noise of the electrical signal. The method of claim 1, wherein the first receiving element and the second receiving element each comprise one microphone selected from the group consisting of a unidirectional microphone and an omnidirectional microphone. The method of claim 1, wherein a plurality of acoustic signals are received in separated time samples, and the first receiving element and the second receiving element are spaced apart by a distance d, where d corresponds to n separated time samples. Noise canceling method of an electrical signal, characterized in that. The method of claim 1, wherein the at least one second transfer function is fixed as a function of the difference between the signal data amplitude of the first receiving element and the signal data amplitude of the second receiving element. 2. The noise of an electrical signal as recited in claim 1, wherein said step of removing noise from the plurality of acoustic signals comprises using directions and ranges for one or more signal sources from one or more first receiving elements. How to remove. 2. The apparatus of claim 1, wherein the frequency response of the at least one first receiving element and the at least one second receiving element are different, and the signal data from the at least one second receiving element is a signal from the at least one first receiving element. A method of removing noise from an electrical signal, characterized in that it is compensated to have a proper relationship to the data. The method of claim 6, Compensating signal data from at least one second receiving element comprises: receiving at least one first receiving element a wideband signal from one source in a direction and position anticipated with respect to a signal from the at least one signal source; And recording at the at least one second receiving element. 7. The method of claim 6, wherein said step of compensating signal data from at least one second receiving element comprises a frequency domain compensation process. The method of claim 8, wherein the frequency domain compensation process, Compute a frequency transform on the signal data from each of the at least one receiving element and at least one second receiving element signal, Determine the magnitude of the frequency transform in each frequency bin, and Setting the magnitude of the frequency transform for signal data from at least one second receiving device at each frequency to a value related to the magnitude of the frequency transform for signal data from at least one first receiving element. And removing the noise of the electrical signal. 7. The method of claim 6, wherein compensating signal data from the at least one second receiving element comprises a time domain compensation process. The method of claim 6, wherein the method is -First set one or more second transfer functions to zero, and Compensation coefficients are calculated whenever one or more noise signals are smaller than one or more voice signals. The method of claim 1, further comprising the step of noise reduction. 10. The method of claim 1, wherein the plurality of acoustic signals comprises one or more reflections of one or more noise signals and one or more reflections of one or more speech signals. The method of claim 1, wherein the step of receiving physiological information is selected from the group consisting of an acoustic microphone, an RF element, an electroglottograph, an ultrasonic element, acoustic throat microphones, and an airflow detector. Receiving physiological data related to human speech using one or more detectors. The method of claim 1, wherein generating at least one first transfer function and at least one second transfer function comprises using at least one technique selected from the group comprising adaptive techniques and iterative techniques. Noise reduction method of electric signal. A system for removing noise from an acoustic signal, The system includes one or more receivers, one or more sensors, and one or more processors, The one or more receivers include one or more signal receivers for receiving one or more acoustic signals from one signal source, and a noise receiver for receiving one or more noise signals from one noise source, wherein the signal source, The relative positions of one or more signal receivers and one or more noise receivers are fixed and known, The one or more sensors receive physiological information related to human voice activity, The one or more processors are coupled between one or more receivers and one or more sensors to generate a plurality of transfer functions, Upon determining that there is lack of speech information from one or more acoustic signals for one or more specified time periods, one or more first transfer functions representing one or more acoustic signals are generated, If it is determined that speech information is present in the one or more acoustic signals for one or more specified time periods, one or more second transfer functions representing one or more acoustic signals are generated, And noise is removed from at least one sound signal using at least one combination of the at least one first transfer function and the at least one second transfer function. 16. The system of claim 15, wherein the one or more sensors include one or more radio frequency (RF) interferometers that detect tissue movement related to human speech. 16. The apparatus of claim 15, wherein the one or more sensors comprise one or more sensors selected from the group consisting of acoustic microphones, high frequency devices, electroglottographs, ultrasonic devices, acoustic throat microphones, and airflow detectors. Noise canceling system of the acoustic signal comprising a. The method of claim 15, wherein the one or more processors, -Divide the acoustic data of one or more acoustic signals into multiple subbands, One or more combinations of one or more first transfer functions and one or more second transfer functions to remove noise from each of the plurality of subbands, wherein a plurality of noise canceled acoustic data streams are generated, and Combining multiple noise canceled acoustic data streams to produce one or more noise canceled acoustic data streams. Noise cancellation system of the acoustic signal, characterized in that. 16. The system of claim 15, wherein said at least one signal receiver and said at least one noise receiver are respective microphones selected from the group consisting of unidirectional and omnidirectional microphones. A signal processing system connected between one or more users and one or more electrical components, The signal processing system comprises one or more first receiving elements, one or more second receiving elements, and one or more noise canceling subsystems, The one or more first receiving elements are arranged to receive one or more acoustic signals from one signal source, the one or more second receiving elements are arranged to receive one or more noise signals from one noise source, In this case, the relative positions of the signal source, the one or more first receiving elements, and the one or more second receiving elements are fixed and known, The one or more noise canceling subsystems remove noise from an acoustic signal, The noise canceling subsystem includes one or more processors and one or more sensors, The at least one processor is coupled between at least one first receiver and at least one second receiver, and The one or more sensors coupled to one or more processors, the one or more sensors configured to receive physiological information related to human voice activity, wherein the one or more processors generate a plurality of transfer functions, Upon determining that there is lack of speech information from one or more acoustic signals for one or more specified time periods, one or more first transfer functions representing one or more acoustic signals are generated, If it is determined that speech information is present in the one or more acoustic signals for one or more specified time periods, one or more second transfer functions representing one or more acoustic signals are generated, Using one or more combinations of the one or more first transfer functions and the one or more second transfer functions to remove noise from one or more acoustic signals, thereby generating one or more noise canceled data streams. Signal processing system. 21. The signal processing system according to claim 20, wherein the first receiving element and the second receiving element are respective microphones selected from the group consisting of unidirectional microphones and omnidirectional microphones. 21. The apparatus of claim 20, wherein the one or more acoustic signals are received in separated time samples, wherein the first receiving element and the second receiving element are spaced apart by a distance d, wherein d is n Signal processing system, characterized in that the corresponding time sample. 21. The signal processing system of claim 20, wherein at least one second transfer function is fixed as a function of the difference between the signal data amplitude of the first receiving element and the signal data amplitude of the second receiving element. 21. The signal processing system of claim 20, wherein removing noise from the one or more acoustic signals utilizes directions and ranges for one or more signal sources from the one or more first receiving elements. The method of claim 20, The frequency response of at least one first receiving element and at least one second receiving element is different, And the signal data from the one or more second receiving elements is compensated to have an appropriate relationship to the signal data from the one or more first receiving elements. 26. The method of claim 25, wherein compensating signal data from the one or more second receiving elements comprises: receiving one or more broadband signals from a source located in a direction and distance expected for a signal from one or more signal sources. And recording to one or more second receiving elements. 26. The signal processing system of claim 25, wherein compensating signal data from the one or more second receiving elements comprises frequency domain compensation. The method of claim 27, wherein the frequency compensation, Compute a frequency transform on the signal data from each of the at least one receiving element and at least one second receiving element signal, Compute the magnitude of the frequency transform at each frequency bin, and Setting the magnitude of the frequency transform for signal data from at least one second receiving device at each frequency to a value related to the magnitude of the frequency transform for signal data from at least one first receiving element. Signal processing system, characterized in that. 26. The signal processing system of claim 25, wherein compensating signal data from the one or more second receiving elements comprises time domain compensation. The method of claim 25, wherein the compensation, -Initially set one or more second transfer functions to zero, and Compensation coefficients are calculated whenever one or more noise signals are smaller than one or more acoustic signals. Signal processing system, characterized in that. 21. The signal processing system of claim 20, wherein the one or more acoustic signals comprise one or more reflections of one or more noise signals and one or more reflections of one or more acoustic signals. The method of claim 20, wherein receiving physiological information comprises: Receive physiological data related to human speech using one or more detectors selected from the group consisting of acoustic microphones, RF devices, electroglottographs, ultrasonic devices, acoustic throat microphones, and airflow detectors Signal processing system, characterized in that. 21. The signal processing system of claim 20, wherein generating at least one first transfer function and at least one second transfer function utilizes one or more techniques selected from the group comprising adaptive techniques and iterative techniques.