CA2397009C - Directional audio signal processing using an oversampled filterbank - Google Patents
Directional audio signal processing using an oversampled filterbank Download PDFInfo
- Publication number
- CA2397009C CA2397009C CA002397009A CA2397009A CA2397009C CA 2397009 C CA2397009 C CA 2397009C CA 002397009 A CA002397009 A CA 002397009A CA 2397009 A CA2397009 A CA 2397009A CA 2397009 C CA2397009 C CA 2397009C
- Authority
- CA
- Canada
- Prior art keywords
- processing system
- output
- signal
- filter
- directional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/40—Arrangements for obtaining a desired directivity characteristic
- H04R25/405—Arrangements for obtaining a desired directivity characteristic by combining a plurality of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/40—Arrangements for obtaining a desired directivity characteristic
- H04R25/407—Circuits for combining signals of a plurality of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
- H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
- H04R25/507—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing implemented by neural network or fuzzy logic
Landscapes
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Circuit For Audible Band Transducer (AREA)
- Complex Calculations (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Measurement Of Radiation (AREA)
Abstract
A directional signal processing system for beamforming information signals. The system includes an oversampled filterbank, which has an analysis filterbank for transforming the information signals in time domain into channel signals in transform domain, a synthesis filterbank and a signal processor. The signal processor processes the outputs of the analysis filterbank for beamforming the information signals. The synthesis filterbank transforms the outputs of the signal processor to a single information signal in time domain.
Description
DIRECTIONAL AUDIO SIGNAL PROCESSING
USING AN OVERSAMPLED FILTERBANK
FIELD OF THE INVENTION
The present invention relates to audio signal processing applications where the direction of arrival of the audio signal(s) is the primary parameter for signal processing. The invention can be used in any application that requires the input audio signal(s) to be processed based on the spatial direction from which the signal arrives.
Application of this invention includes, but is not limited to, audio surveillance systems, hearing aids, voice-command systems, portable communication devices, speech recognition/transcription systems, and any application where it is desirable to process signal(s) based on the direction of arrival.
BACKGROUND OF THE INVENTION
Directional processing can be used to solve a multitude of audio signal processing problems. In hearing aid applications, for example, directional processing can be used to reduce the environmental noise that originates from spatial directions different from the desired speech or sound, thereby improving the listening comfort and speech perception of the hearing aid user. In audio surveillance, voice-command and portable communication systems, directional processing can be used to enhance the reception of sound originating from a specific direction, thereby enabling these systems to focus on the desired sound. In other systems, directional processing can be used to reject interfering signal(s) originating from specific direction(s), while maintaining the perception of signal(s) originating from all other directions, thereby insulating the systems from the detrimental effect of interfering signal(s).
Beamfonning is the term used to describe a technique which uses a mathematical model to maximise the directionality of an input device. In such a technique filtering weights may be adjusted in real time or adapted to react to changes in the environment of either the user or the signal source, or both.
USING AN OVERSAMPLED FILTERBANK
FIELD OF THE INVENTION
The present invention relates to audio signal processing applications where the direction of arrival of the audio signal(s) is the primary parameter for signal processing. The invention can be used in any application that requires the input audio signal(s) to be processed based on the spatial direction from which the signal arrives.
Application of this invention includes, but is not limited to, audio surveillance systems, hearing aids, voice-command systems, portable communication devices, speech recognition/transcription systems, and any application where it is desirable to process signal(s) based on the direction of arrival.
BACKGROUND OF THE INVENTION
Directional processing can be used to solve a multitude of audio signal processing problems. In hearing aid applications, for example, directional processing can be used to reduce the environmental noise that originates from spatial directions different from the desired speech or sound, thereby improving the listening comfort and speech perception of the hearing aid user. In audio surveillance, voice-command and portable communication systems, directional processing can be used to enhance the reception of sound originating from a specific direction, thereby enabling these systems to focus on the desired sound. In other systems, directional processing can be used to reject interfering signal(s) originating from specific direction(s), while maintaining the perception of signal(s) originating from all other directions, thereby insulating the systems from the detrimental effect of interfering signal(s).
Beamfonning is the term used to describe a technique which uses a mathematical model to maximise the directionality of an input device. In such a technique filtering weights may be adjusted in real time or adapted to react to changes in the environment of either the user or the signal source, or both.
Traditionally, directional processing for audio signals has been implemented in the time-domain using Finite Impulse Response (FIR) filters and/or simple time-delay elements. For applications dealing with simple narrow band signals, these approaches are generally sufficient. To deal with complex broadband signals such as speech, however, these time-domain approaches generally provide poor performance unless significant extra resources, such as large microphone arrays, lengthy filters, complex post-filtering, and high processing power are committed to the application.
Examples of these technologies are described in "Analysis of Noise Reduction and Dereverberation Techniques Based on Microphone Arrays with Postfiltering", C.
Marro, Y. Mahieux and K. U. Simmer, IEEE Trans. Speech and Audio Processing, vol. 6, no. 3, 1998, and in "A Microphone Array for Hearing Aids", B. Widrow, IEEE Adaptive Systems for Signal Processing, Communications and Control Symposium, pp.7-11, 2000.
In any directional processing algorithm, an array of two or more sensors is required. For audio directional processing, either omni-directional or directional microphones are used as the sensors. Figure 1 shows a high4evel block diagram of a general directional processing system. As seen in the figure, while there are two or more inputs 100, 105 to the system 110, there is generally only one output 120.
There are two common types of directional processing algorithms: adaptive beamforming and fixed beamforming. In fixed beamforming, the spatial response -or beampattern - of the algorithm does not change with time, as opposed to a time-varying beampattern in adaptive beamforming. A beampattern is a polar graph that illustrates the gain response of the beamforming system at a particular signal frequency over different directions of arrival. Figure 2 shows an example of two different beampattems in which signals from certain directions of arrival are attenuated (or enhanced) relative to signals from other directions. The first is the cardioid pattern 200, typical of some end-fire microphone arrays, and the other 205 is the beampattem typical of broad-side microphone arrays. Figure 3 illustrates typical configurations for end-fire 300, 305, 310 and broadside 320, 325, 330 microphone arrays.
Examples of these technologies are described in "Analysis of Noise Reduction and Dereverberation Techniques Based on Microphone Arrays with Postfiltering", C.
Marro, Y. Mahieux and K. U. Simmer, IEEE Trans. Speech and Audio Processing, vol. 6, no. 3, 1998, and in "A Microphone Array for Hearing Aids", B. Widrow, IEEE Adaptive Systems for Signal Processing, Communications and Control Symposium, pp.7-11, 2000.
In any directional processing algorithm, an array of two or more sensors is required. For audio directional processing, either omni-directional or directional microphones are used as the sensors. Figure 1 shows a high4evel block diagram of a general directional processing system. As seen in the figure, while there are two or more inputs 100, 105 to the system 110, there is generally only one output 120.
There are two common types of directional processing algorithms: adaptive beamforming and fixed beamforming. In fixed beamforming, the spatial response -or beampattern - of the algorithm does not change with time, as opposed to a time-varying beampattern in adaptive beamforming. A beampattern is a polar graph that illustrates the gain response of the beamforming system at a particular signal frequency over different directions of arrival. Figure 2 shows an example of two different beampattems in which signals from certain directions of arrival are attenuated (or enhanced) relative to signals from other directions. The first is the cardioid pattern 200, typical of some end-fire microphone arrays, and the other 205 is the beampattem typical of broad-side microphone arrays. Figure 3 illustrates typical configurations for end-fire 300, 305, 310 and broadside 320, 325, 330 microphone arrays.
More recent Fast Fourier Transform (FFT)-based approaches attempt to improve upon the traditional time-domain approaches by implementing directional processing in the frequency-domain. However, many of these FFT-based approaches suffer from wide sub-bands that are highly overlapped, and therefore provide poor frequency resolution. They also require longer group delays and more processing power in computing the FFT.
Accordingly, there is a need to solve the problems noted above and also a need for an innovative approach to enhance and/or replace the current technologies.
SUMMARY OF THE INVENTION
The invention described herein is applicable to both the end-fire and broadside microphone configurations in solving the problems found in conventional beamforming solutions. It is also possible to apply the invention to other geometric configurations of the microphone array, as the underlying processing architecture is flexible enough to accommodate a wide range of array configurations. For example, more complex directional systems based on two or three-dimensional arrays, used to produce beampatterns having three dimensions, are known and are suitable for used with this invention.
In accordance with an aspect of the present invention, there is provided a directional signal processing system for beamforming a plurality of information signals, which includes: a plurality of microphones; an oversampled filterbank comprising at least one analysis filterbank for transforming a plurality of information signals in time domain from the microphones into a plurality of channel signals in transform domain, and one synthesis filterbank; and a signal processor for processing the outputs of said analysis filterbank for beamforming said information signals. The synthesis filterbank transforming the outputs of said signal processor to a single information signal in time domain.
In accordance with a further aspect of the present invention, there is provided a method of processing a plurality of channel signals for achieving approximately linear phase response within the channel, which includes a step of performing filtering by applying more than one filter to at least one channel signal.
Accordingly, there is a need to solve the problems noted above and also a need for an innovative approach to enhance and/or replace the current technologies.
SUMMARY OF THE INVENTION
The invention described herein is applicable to both the end-fire and broadside microphone configurations in solving the problems found in conventional beamforming solutions. It is also possible to apply the invention to other geometric configurations of the microphone array, as the underlying processing architecture is flexible enough to accommodate a wide range of array configurations. For example, more complex directional systems based on two or three-dimensional arrays, used to produce beampatterns having three dimensions, are known and are suitable for used with this invention.
In accordance with an aspect of the present invention, there is provided a directional signal processing system for beamforming a plurality of information signals, which includes: a plurality of microphones; an oversampled filterbank comprising at least one analysis filterbank for transforming a plurality of information signals in time domain from the microphones into a plurality of channel signals in transform domain, and one synthesis filterbank; and a signal processor for processing the outputs of said analysis filterbank for beamforming said information signals. The synthesis filterbank transforming the outputs of said signal processor to a single information signal in time domain.
In accordance with a further aspect of the present invention, there is provided a method of processing a plurality of channel signals for achieving approximately linear phase response within the channel, which includes a step of performing filtering by applying more than one filter to at least one channel signal.
In accordance with a further aspect of the present invention, there is provided a method of processing at least one information signal in time domain for achieving approximately linear phase response, which includes a step of performing an oversampling using at least one oversatnpled analysis filterbank. The oversampled analysis filterbank applies at lease one fractional delay impulse response to.
at least one filterbank prototype window time:
The directional processing system of the invention takes advantage of oversampled analysis/synthesis filterbanks to transform the input audio signals in time domain to a transform domain. Example of common transforniation methods includes GDFT (Generalized Discrete Fourier Transform), FFT, DCT (Discrete Cosine Transform), Wavelet Transform and other generalized transforms. The emphasis of the invention described herein is on a directional processing system employing oversampled filterbanks, with the FFT method being one possible embodiment of said filterbanks. An example of the oversampled, FFT-based filterbanks is described in United States Patent 6,236,731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids" by R. Brennan and T. Schneider.
An example of an hearing aid apparatus employing said oversampled filterbanks is described in United States Patent 6,240,192 "Apparatus for and Method for Filtering in an Digital Hearing Aid, Including an Application Specific Integrated Circuit and a Propa*amrnable Digital Signal Processor" by R. Brennan and T. Schneider;
However, this use ofoversampled analysis/synthesis filterbanks in the general framework of the directional processing system disclosed herein has not been reported before.
The sub-band signal processing approach described henceforth, with its corresponding FFT-based method being one possible embodiment of the oversampled filterbanks employed in the invention disclosed herein, has the advantage of directly addressing the frequency-dependent characteristics in the directional processing of broadband signals. Compared to traditional time-domain and FFT-based approaches, the advantages of using an oversampled filterbank in sub-band signal processing according to the present invention are as follows:
1) Equal or greater signal processing capability at a fraction of the processing power, 2) Orthogonalization effect of the subband signals in the different frequency bins due to the FFT of the oversampled filterbank, 5 3) Improved high frequency resolution, 4) Better spatial filtering, 5) Wide range of gain adjustment at a very low cost of processingpower, and 6) Ease of integration with other algorithms.
As a result, the sub-band directional processing approach with an oversampled filterbank allows powerful directional processing capability to be implemented on miniature low-power devices. For applications employing the invention, this means:
1) Better listening comfort and speech perception (particularly important for hearing aids), 2) More accurate recognition for speech and speaker recognition systems, 3) Better directionality and higher SNR, 4) Low group delay, and 5) Lower power consumption.
Thus, the present invention is applicable for audio applications that require a high fidelity and ultra low-power processing platform.
A further understanding of the other features, aspects, and advantages of the present invention will be realized by reference to the following description, appended claims, and accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the invention will now be described with reference to the accompanying drawings, in which:
Figure 1 shows a block diagram of a general directional processing system;
Figure 2 shows an example of two different beampatterns;
Figure 3 shows the array configuration of the end-fire and broadside arrays;
Figure 4 shows a block diagram of the adaptive beamformer system according to one embodiment of the invention;
Figure 5 shows a block diagram of the adaptive beamformer system according to another embodiment of the invention;
Figure 6 shows a traditional time-domain beamformer structure;
Figure 7 shows a sub-band beamformer using an oversampled filterbank according to another embodiment of the present invention;
Figure 8 shows another preferred embodiment modified for compensating the bandwidth of the sub-bands;
Figure 9 shows another preferred embodiment modified for compensating the undesirable low-frequency beamformer response; and Figure 10 show another preferred embodiment using a neural network as a beamformer filter according to the invention.
DETAILED DESCRIPTION OF THE INVENTION
Turning now to Figure 4 an adaptive beamformer system embodying the invention in block diagram form is shown. Note that it is assumed that the outputs of the L microphones 400 (L _ 2) are already converted to digital form by a set of analogue-to-digital converters (ADC) (not shown). Similarly, the output is assumed to be converted from digital form by an digital-to-analogue converter (DAC) (not shown) to produce an appropriate output signal 490. The digitized outputs of the L
microphones 400 are first combined in a combination matrix 415. The combination matrix 415 can be any Finite Impulse Response (FIR) filter with multiple input and outputs (the number of outputs Mbeing less or equal to the number of inputs L(M_<
L)). Suitable matrices include a delay-and-sum network, a sigma-delta network, and a one-to-one mapping of the inputs to the outputs (for example some general matrix through which L inputs are transformed into L (i.e. M=L) outputs)). TheM
outputs of the combination matrix 415 are then transformed to the frequency domain by an analysis filterbank 420, with N sub-bands per combination matrix output to produce M x N signals for processing. The (oversampled) analysis filterbank 420 used in this embodiment is the weighted-overlap-add (WOLA) filterbank described in United States Patent 6,236, 731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids" by R. Brennan and T. Schneider. An adaptive system 460 then generates a weighted sum of the analysis filterbank outputs which are applied to the outputs by the multiplier 425. The weights (also known as filter taps) of the adaptive system 460 are adapted according to well known adaptive strategies including, but not limited to, those based on Least Mean Squares (LMS), and Recursive Least Squares (RLS).
The outputs of the multiplier 425 are then passed to a summer 430 which producesN
outputs, each a weighted sub-band derived from the original microphone signals. The overall adaptation process is further controlled by the outputs of a side process comprising an estimations block 450, and a post-filter adapter 455. The estimations block of the side process 450 may include one or more of a Voice Activity Detector (VAD), a Target-to-Jammer Ratio (TJR) estimator, and a Signal-to-Noise Ratio (SNR) estimator. The outputs of the estimations block 450 are then used to slow down, speed up, or inhibit the adaptation process by controlling the weight adaptation 460, and also combined with post-filter adaptation 455 to control the post-filter 435.
After passing through a summer 430 which combines the processed Mx N inputs received from the adaptive processor 460, 425 intoN sub-bands, the post-filter operates in the frequency domain to further process the signal depending on the output from the post-filter adapter 455, After post-filtering the N sub-band frequency domain outputs are processed by a synthesis filterbank 440 to generate a time-domain output 490.
Oversampled filterbanks offer the general advantages explained in the summary above by virtue of their flexibility and the fabrication technology.
Further advantages of their use for the adaptive beamformer application of the present invention are:
1) Directional processing using prior art techniques requires very long adaptive filter lengths particularly in reverberant environments, as reported by other researchers (see J. E. Greenberg, "Improved Design of Microphone-Array Hearing Aids", Ph.D Thesis, MIT, Sept. 1994). The sub-band adaptation using the oversampled filterbank can efficiently implement the equivalent of a long filter through parallel sub-band processing.
2) In frequency domain beamforming (both adaptive and fixed), there is a need to weight the Fast Fourier Transform (FFT) coefficients in a highly unconstrained way. A typical adaptive post-filtering operation is the multiple-microphone Wiener filtering, in which the frequency response is adapted depending on the Signal-to-Noise Ratio (SNR) of the received signal. In this process, there is a need for unconstrained gain adjustments across the frequency bands. The oversampled filterbank implementation allows a wide range of gain adjustments without creating the so-called "time-aliasing" problem that happens in the critically sampled filterbanks. It has been observed that the operation cost is not much higher than the critically sampled filterbanks and much lower than the undecimated filterbanks. For more information see United States Patent 6,236, 731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids". R. Brennan and T.
Schneider, and "A Flexible Filterbank Structure for Extensive Signal Manipulations in Digital Hearing Aids", R. Brennan and T. Schneider,Proc. IEEE Int. Symp.
Circuits and Systems, pp.569-572, 1998.
3) The so-called "Misadjustment" error, where there is excessive Mean Square Error when compared to an optimal Wiener filter, is typically present in adaptive systems. It is well known and understood that sub-band and orthogonal decomposition reduces this problem. The oversampled filterbank used in the invention employs such decomposition in at least one preferred embodiment.
4) Estimation of Target-to-Jammer Ratio (TJR) usually requires the cross-correlation of two or more microphone outputs (as described in "Improved Design of Microphone-Array Hearing Aids", J. E. Greenberg, Ph.D Thesis, NIIT, Sept.
1994).
The frequency domain implementation of the process using the oversampled filterbank is much faster and more efficient than the time-domain methods previously used.
5) By using the side process outputs of the Voice Activity Detector (VAD),_ the Target-to-Jammer Ratio (TJR).estimator, and the Signal-to-Noise Ratio (SNR) estimator, the adaptation process can be slowed down or totally inhibited when there is a strong target (like speech) presence. This enables the system to work in reverberant environments. There are enough pauses in speech signal to ensure that the inhibition process does not disturb the system performance. A suitable efficient frequency domain VAD that uses the oversampled filterbank is described in a co--pending patent application "Sub-band Adaptive Signal Processing in an Oversampled Filterbank", K. Tam et. al., Canadian Patent Application Serial 2,354,808, August 2001, US Patent Application Publication No. 20030108214.
According to a further preferred embodiment of the invention, shown in Figure 5, the weight adaptation process is performed on a set of Bfixed beams for each sub-band constructed or synthesised from the sub-bands derived from each microphone output, rather than the microphone outputs themselves or the sub bands of such outputs. Within Figure 5 most of the elements*are the same as Figure 4, and have been notated with the same reference numbers. Therefore these elements will not be described again. The new elements introduced in this embodiment are the Fixed Beamformer 510 which produces B main beams from the sub-bands, and a weight adaptation block 520 which controls the multiplier 425, based on inputs from the VAD, TJR and SNR estimations block 450, and the sub-band signals output by the Fixed Beamformer 510. Generally this strategy provides a smoother and more robust transition when the adaptive filtering weights are changed. The weight adaptation is ~ . ~ .
controlled by some TJR and/or SNR estimations based on, but not limited to, one or more of the following signal statistics: auto-correlation, cross-correlation, subband magnitude level, subband power level, cross-power spectrum, cross-power phase, cross-spectral density, etc. One possible filtering weight adaptation strategy based on 5 a simplified SNR estimation is proposed here, and other similar or related methods may occur to those skilled in the art, and it is our intention that these be covered.
When the side process detects the absence (or near absence) of the target, the tim&
averaged energy of the noise in each of the beams (denoted by En(n, I=1,2,...,B) is measured. When the target reappears, the time-averaged energy of the target (Et(1) ) 10 and the SNR in each beam (SNR(1) ) are estimated, given the total averaged energy in the beam Etot(.l), by:
Et(1)= Etot(1)-En(I) , P--1,2,...,B
SNR(1)= EtQ)/En(1) If the noise statistics, and noise and target directions do not change much from one target signal pause to the next pause, the SNR(1) for each beam can be used to make a weighted sum of the beams. However, if the noise is highly nonstationary, or if the noise and/or target sources are moving quickly, an adaptive processor should be employed to adjust the weights. For improved performance, the fixed beamformer can be designed with a set of narrow beams covering the azimuth and elevation angles of interest for a particular application.
A further embodiment of the invention in a fixed beamforming application will now be discussed. The classical method of implementing a fixed beamformer is the delay-and-sum method. Because of the physical spacing of the microphones in the array, there is an inherent time delay between the signals received at each microphone. Hence, the delay-and-sum method utilizes a simple time-delay element to properly align the received signals so that the signals arriving from certain directions can be maximally in-phase, and contribute coherently to the summed output signal. Any signal arriving from other directions then contributes incoherently to the output signal, so that its signal power can be reduced at the output.
k 1 With the FIR-filter method, the FIR filters are generally designed so that their phase responses take on the role of aligning the received signals to create the desired beampattern. These filters can be designed using transformation from analogue filters, or direct FIR filter design approaches. When complex broadband signals are involved, such time-domain filter designs generally require the availability of a significant amount of computation power. For comparison, Figure 6 shows a fixed beamformer structure using the prior art time-domain approach. In the figure an array of three microphones 600, 601, 602 is disposed in a known pattern, although a greater number of microphones might also be used. The outputs of each microphone in the array 600, 601, 602 is passed to a separate time-delay element(or FIR Filter) 610, 611,612, whose outputs are passed in turn to a summer 620. The summer 620, when the time delay elements are correctly set as described above, provides an enhanced output 630 for a particular spatial direction with respect to the microphone array.
Usually, this setting of the time delay elements 610, 611,612, is accomplished dynamically, but is often a compromise depending on the factors including the frequency of the signal, and the relative spacing of the microphones in the array. Ifa number of beams were required, each would be constructed or synthesised using a similar circuit.
For that reason these systems are expensive, high in power consumption, complex and hence limited in application.
Further preferred embodiments of the invention described herein perfprm a series of narrowband processing steps to solve the more complex broadband problem.
The use of the oversampled filterbank allows the narrowband processing to be done in an efficient and practical manner. Figure 7 shows a sub-band fixed beamformer using an oversampled filterbank according to another embodiment of the present invention.
The system is very similar to that described in Figure 4. For convenience and clarity, the same components are identified by the same reference numbers in both figures.
The digital versions of the signals received at theL-microphone array 400 are combined through a combination matrix 415 into M signal channels (M <_ L) before being sent to the analysis filterbank 420. The analysis filterbank 420generates N
frequency sub-bands for each channel, whereupon the beamforming filter 710 applies complex-valued gain factors for achieving the desired beampattern, based on inputs from the VAD, TJR and SNR estimation block 450, and the level of signal in thesub-, < .
at least one filterbank prototype window time:
The directional processing system of the invention takes advantage of oversampled analysis/synthesis filterbanks to transform the input audio signals in time domain to a transform domain. Example of common transforniation methods includes GDFT (Generalized Discrete Fourier Transform), FFT, DCT (Discrete Cosine Transform), Wavelet Transform and other generalized transforms. The emphasis of the invention described herein is on a directional processing system employing oversampled filterbanks, with the FFT method being one possible embodiment of said filterbanks. An example of the oversampled, FFT-based filterbanks is described in United States Patent 6,236,731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids" by R. Brennan and T. Schneider.
An example of an hearing aid apparatus employing said oversampled filterbanks is described in United States Patent 6,240,192 "Apparatus for and Method for Filtering in an Digital Hearing Aid, Including an Application Specific Integrated Circuit and a Propa*amrnable Digital Signal Processor" by R. Brennan and T. Schneider;
However, this use ofoversampled analysis/synthesis filterbanks in the general framework of the directional processing system disclosed herein has not been reported before.
The sub-band signal processing approach described henceforth, with its corresponding FFT-based method being one possible embodiment of the oversampled filterbanks employed in the invention disclosed herein, has the advantage of directly addressing the frequency-dependent characteristics in the directional processing of broadband signals. Compared to traditional time-domain and FFT-based approaches, the advantages of using an oversampled filterbank in sub-band signal processing according to the present invention are as follows:
1) Equal or greater signal processing capability at a fraction of the processing power, 2) Orthogonalization effect of the subband signals in the different frequency bins due to the FFT of the oversampled filterbank, 5 3) Improved high frequency resolution, 4) Better spatial filtering, 5) Wide range of gain adjustment at a very low cost of processingpower, and 6) Ease of integration with other algorithms.
As a result, the sub-band directional processing approach with an oversampled filterbank allows powerful directional processing capability to be implemented on miniature low-power devices. For applications employing the invention, this means:
1) Better listening comfort and speech perception (particularly important for hearing aids), 2) More accurate recognition for speech and speaker recognition systems, 3) Better directionality and higher SNR, 4) Low group delay, and 5) Lower power consumption.
Thus, the present invention is applicable for audio applications that require a high fidelity and ultra low-power processing platform.
A further understanding of the other features, aspects, and advantages of the present invention will be realized by reference to the following description, appended claims, and accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the invention will now be described with reference to the accompanying drawings, in which:
Figure 1 shows a block diagram of a general directional processing system;
Figure 2 shows an example of two different beampatterns;
Figure 3 shows the array configuration of the end-fire and broadside arrays;
Figure 4 shows a block diagram of the adaptive beamformer system according to one embodiment of the invention;
Figure 5 shows a block diagram of the adaptive beamformer system according to another embodiment of the invention;
Figure 6 shows a traditional time-domain beamformer structure;
Figure 7 shows a sub-band beamformer using an oversampled filterbank according to another embodiment of the present invention;
Figure 8 shows another preferred embodiment modified for compensating the bandwidth of the sub-bands;
Figure 9 shows another preferred embodiment modified for compensating the undesirable low-frequency beamformer response; and Figure 10 show another preferred embodiment using a neural network as a beamformer filter according to the invention.
DETAILED DESCRIPTION OF THE INVENTION
Turning now to Figure 4 an adaptive beamformer system embodying the invention in block diagram form is shown. Note that it is assumed that the outputs of the L microphones 400 (L _ 2) are already converted to digital form by a set of analogue-to-digital converters (ADC) (not shown). Similarly, the output is assumed to be converted from digital form by an digital-to-analogue converter (DAC) (not shown) to produce an appropriate output signal 490. The digitized outputs of the L
microphones 400 are first combined in a combination matrix 415. The combination matrix 415 can be any Finite Impulse Response (FIR) filter with multiple input and outputs (the number of outputs Mbeing less or equal to the number of inputs L(M_<
L)). Suitable matrices include a delay-and-sum network, a sigma-delta network, and a one-to-one mapping of the inputs to the outputs (for example some general matrix through which L inputs are transformed into L (i.e. M=L) outputs)). TheM
outputs of the combination matrix 415 are then transformed to the frequency domain by an analysis filterbank 420, with N sub-bands per combination matrix output to produce M x N signals for processing. The (oversampled) analysis filterbank 420 used in this embodiment is the weighted-overlap-add (WOLA) filterbank described in United States Patent 6,236, 731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids" by R. Brennan and T. Schneider. An adaptive system 460 then generates a weighted sum of the analysis filterbank outputs which are applied to the outputs by the multiplier 425. The weights (also known as filter taps) of the adaptive system 460 are adapted according to well known adaptive strategies including, but not limited to, those based on Least Mean Squares (LMS), and Recursive Least Squares (RLS).
The outputs of the multiplier 425 are then passed to a summer 430 which producesN
outputs, each a weighted sub-band derived from the original microphone signals. The overall adaptation process is further controlled by the outputs of a side process comprising an estimations block 450, and a post-filter adapter 455. The estimations block of the side process 450 may include one or more of a Voice Activity Detector (VAD), a Target-to-Jammer Ratio (TJR) estimator, and a Signal-to-Noise Ratio (SNR) estimator. The outputs of the estimations block 450 are then used to slow down, speed up, or inhibit the adaptation process by controlling the weight adaptation 460, and also combined with post-filter adaptation 455 to control the post-filter 435.
After passing through a summer 430 which combines the processed Mx N inputs received from the adaptive processor 460, 425 intoN sub-bands, the post-filter operates in the frequency domain to further process the signal depending on the output from the post-filter adapter 455, After post-filtering the N sub-band frequency domain outputs are processed by a synthesis filterbank 440 to generate a time-domain output 490.
Oversampled filterbanks offer the general advantages explained in the summary above by virtue of their flexibility and the fabrication technology.
Further advantages of their use for the adaptive beamformer application of the present invention are:
1) Directional processing using prior art techniques requires very long adaptive filter lengths particularly in reverberant environments, as reported by other researchers (see J. E. Greenberg, "Improved Design of Microphone-Array Hearing Aids", Ph.D Thesis, MIT, Sept. 1994). The sub-band adaptation using the oversampled filterbank can efficiently implement the equivalent of a long filter through parallel sub-band processing.
2) In frequency domain beamforming (both adaptive and fixed), there is a need to weight the Fast Fourier Transform (FFT) coefficients in a highly unconstrained way. A typical adaptive post-filtering operation is the multiple-microphone Wiener filtering, in which the frequency response is adapted depending on the Signal-to-Noise Ratio (SNR) of the received signal. In this process, there is a need for unconstrained gain adjustments across the frequency bands. The oversampled filterbank implementation allows a wide range of gain adjustments without creating the so-called "time-aliasing" problem that happens in the critically sampled filterbanks. It has been observed that the operation cost is not much higher than the critically sampled filterbanks and much lower than the undecimated filterbanks. For more information see United States Patent 6,236, 731 "Filterbank Structure and Method for Filtering and Separating an Information Signal into Different Bands, Particularly for Audio Signal in Hearing Aids". R. Brennan and T.
Schneider, and "A Flexible Filterbank Structure for Extensive Signal Manipulations in Digital Hearing Aids", R. Brennan and T. Schneider,Proc. IEEE Int. Symp.
Circuits and Systems, pp.569-572, 1998.
3) The so-called "Misadjustment" error, where there is excessive Mean Square Error when compared to an optimal Wiener filter, is typically present in adaptive systems. It is well known and understood that sub-band and orthogonal decomposition reduces this problem. The oversampled filterbank used in the invention employs such decomposition in at least one preferred embodiment.
4) Estimation of Target-to-Jammer Ratio (TJR) usually requires the cross-correlation of two or more microphone outputs (as described in "Improved Design of Microphone-Array Hearing Aids", J. E. Greenberg, Ph.D Thesis, NIIT, Sept.
1994).
The frequency domain implementation of the process using the oversampled filterbank is much faster and more efficient than the time-domain methods previously used.
5) By using the side process outputs of the Voice Activity Detector (VAD),_ the Target-to-Jammer Ratio (TJR).estimator, and the Signal-to-Noise Ratio (SNR) estimator, the adaptation process can be slowed down or totally inhibited when there is a strong target (like speech) presence. This enables the system to work in reverberant environments. There are enough pauses in speech signal to ensure that the inhibition process does not disturb the system performance. A suitable efficient frequency domain VAD that uses the oversampled filterbank is described in a co--pending patent application "Sub-band Adaptive Signal Processing in an Oversampled Filterbank", K. Tam et. al., Canadian Patent Application Serial 2,354,808, August 2001, US Patent Application Publication No. 20030108214.
According to a further preferred embodiment of the invention, shown in Figure 5, the weight adaptation process is performed on a set of Bfixed beams for each sub-band constructed or synthesised from the sub-bands derived from each microphone output, rather than the microphone outputs themselves or the sub bands of such outputs. Within Figure 5 most of the elements*are the same as Figure 4, and have been notated with the same reference numbers. Therefore these elements will not be described again. The new elements introduced in this embodiment are the Fixed Beamformer 510 which produces B main beams from the sub-bands, and a weight adaptation block 520 which controls the multiplier 425, based on inputs from the VAD, TJR and SNR estimations block 450, and the sub-band signals output by the Fixed Beamformer 510. Generally this strategy provides a smoother and more robust transition when the adaptive filtering weights are changed. The weight adaptation is ~ . ~ .
controlled by some TJR and/or SNR estimations based on, but not limited to, one or more of the following signal statistics: auto-correlation, cross-correlation, subband magnitude level, subband power level, cross-power spectrum, cross-power phase, cross-spectral density, etc. One possible filtering weight adaptation strategy based on 5 a simplified SNR estimation is proposed here, and other similar or related methods may occur to those skilled in the art, and it is our intention that these be covered.
When the side process detects the absence (or near absence) of the target, the tim&
averaged energy of the noise in each of the beams (denoted by En(n, I=1,2,...,B) is measured. When the target reappears, the time-averaged energy of the target (Et(1) ) 10 and the SNR in each beam (SNR(1) ) are estimated, given the total averaged energy in the beam Etot(.l), by:
Et(1)= Etot(1)-En(I) , P--1,2,...,B
SNR(1)= EtQ)/En(1) If the noise statistics, and noise and target directions do not change much from one target signal pause to the next pause, the SNR(1) for each beam can be used to make a weighted sum of the beams. However, if the noise is highly nonstationary, or if the noise and/or target sources are moving quickly, an adaptive processor should be employed to adjust the weights. For improved performance, the fixed beamformer can be designed with a set of narrow beams covering the azimuth and elevation angles of interest for a particular application.
A further embodiment of the invention in a fixed beamforming application will now be discussed. The classical method of implementing a fixed beamformer is the delay-and-sum method. Because of the physical spacing of the microphones in the array, there is an inherent time delay between the signals received at each microphone. Hence, the delay-and-sum method utilizes a simple time-delay element to properly align the received signals so that the signals arriving from certain directions can be maximally in-phase, and contribute coherently to the summed output signal. Any signal arriving from other directions then contributes incoherently to the output signal, so that its signal power can be reduced at the output.
k 1 With the FIR-filter method, the FIR filters are generally designed so that their phase responses take on the role of aligning the received signals to create the desired beampattern. These filters can be designed using transformation from analogue filters, or direct FIR filter design approaches. When complex broadband signals are involved, such time-domain filter designs generally require the availability of a significant amount of computation power. For comparison, Figure 6 shows a fixed beamformer structure using the prior art time-domain approach. In the figure an array of three microphones 600, 601, 602 is disposed in a known pattern, although a greater number of microphones might also be used. The outputs of each microphone in the array 600, 601, 602 is passed to a separate time-delay element(or FIR Filter) 610, 611,612, whose outputs are passed in turn to a summer 620. The summer 620, when the time delay elements are correctly set as described above, provides an enhanced output 630 for a particular spatial direction with respect to the microphone array.
Usually, this setting of the time delay elements 610, 611,612, is accomplished dynamically, but is often a compromise depending on the factors including the frequency of the signal, and the relative spacing of the microphones in the array. Ifa number of beams were required, each would be constructed or synthesised using a similar circuit.
For that reason these systems are expensive, high in power consumption, complex and hence limited in application.
Further preferred embodiments of the invention described herein perfprm a series of narrowband processing steps to solve the more complex broadband problem.
The use of the oversampled filterbank allows the narrowband processing to be done in an efficient and practical manner. Figure 7 shows a sub-band fixed beamformer using an oversampled filterbank according to another embodiment of the present invention.
The system is very similar to that described in Figure 4. For convenience and clarity, the same components are identified by the same reference numbers in both figures.
The digital versions of the signals received at theL-microphone array 400 are combined through a combination matrix 415 into M signal channels (M <_ L) before being sent to the analysis filterbank 420. The analysis filterbank 420generates N
frequency sub-bands for each channel, whereupon the beamforming filter 710 applies complex-valued gain factors for achieving the desired beampattern, based on inputs from the VAD, TJR and SNR estimation block 450, and the level of signal in thesub-, < .
bands produced by the analysis filterbank 420. The gain factors can be applied either independently for each channel and sub-band, or jointly through all channels and/or sub-bands by some matrix operation. After the gain factors are applied by the multiplier 425, the Mchannels are combined to form a single channel through a summation operation 430. A post-filtering process 435 can then be applied to provide further enhancement as before (such as improving the SNR) making use of the side process 450, 455. Afterwards, the synthesis filterbank 440 transforms the single channel composed of N sub-bands back to time-domain. In further embodiments, the post-filtering is applied in the time-domain, after the signal channel is converted back to time-domain by the synthesis filterbank, although, compared to frequency4domain post-filtering, this typically requires more processing power.
The complex-valued gain factors of the beamforming filter can be derived in a number of ways. For example, if an analogue filterhas been designed, then it can be implemented directly in sub-bands by simply using the centre frequency of each sub-band to look up the corresponding complex response of the analogue filter (frequency sampling). With sufficiently narrow sub-bands, this method can create a close digital equivalent of the analogue filter. In a further embodiment of the invention, to closely approximate the ideal phase and amplitude responses for wider sub-bands, a narrowband filter to each sub-band output is applied as will now be described in relation to Figure 8 in which again, many of the components are the same as for the earlier Figure 7, and for which those same components are for convenience and clarity referred to by the same reference numbers. The additional functim for this embodiment is performed in the Narrowband Prototype Filters 815. To approximate an ideal linear phase response of the beamforrner, the filters 815 are designed as all-pass with a narrowband linear phase response. In a further embodiment, the filtrs are further constrained to being identical, and are moved back before the FFT
modulation stage by combining its impulse response with the filterbank prototype window.
One possible combination is a time convolution of the filterbank prototype window with a fractional delay impulse response. As a means of eliminating the external noise at the acoustic output stage, an Active Noise Cancellation (ANC) module is optionally added to the system in a manner similar to the system described in a co-pending patent application "Sound Intelligibility Enhancement Using a Psychoacoustic Model and an Oversampled Filterbank", T. Schneider et. al., Canadian Patent Application, seria12,354,755, US Patent Application Publication No. 20030198357. The ANC, as also shown in Figure 8, consists of a microphone 820 positioned at the output 490, plus a loop filter 830 to provide feedback to the combination matrix 415.
Almost all iunplementations of beamformers suffer from a low-frequency roll-off effect. To compensate for this effect, most systems, including the proposed system, introduce low-frequency amplification. However, because of the unavoidable microphone interna.l noise, this inherently leads to a high level of output noise at very low frequencies. As is well known, the result is that the desired beampattem can only be obtained for the frequencies above some cut-off value (usually around 1 kHz based on a particular microphone separation distance). In a further embodiment, shown in Figure 9, to avoid a high-level of low-frequency noise, the microphone signals are separated into high frequency and'low-frequency components by high-pass filter (HPF) 920 and low-pass filter (LPF) 910. Again, many of the same components used in the preferred embodiment described with reference to Figure 7 are used, performing the same function, and are given the same reference numbers. The high frequency components output by the high pass filter 920 are processed by the beamforming filter 710, multiplier 7425, and Narrow band prototype filters *815, as before. The low-frequency components by-pass the beamforming filter 710, multiplier 7425, and Narrow band prototype filters 815, relying solely on the post-filter 435 to provide low-frequency signal enhancement.
Besides the conventional digital filter design methods, the beamformer filter 710 in Figure 7 can also be implemented using an Artificial Neural Network (ANN).
The ANN can be employed as a type of non-parametric; robust adaptive filter, and has been increasingly investigated as a viable signal processing approach. One further possible embodiment of the present invention is to implement a neural network as a complete beamforming filter, as shown in Figure 10. Once again-the same reference numbers as Figure 4 are used for those components that are unchanged in function. The neural network 1010 accepts inputs from the sub-bands output by the analysis filterbank, and uses these to control the multiplier 425 which affect those sub-bands. The post filter adaptor 455 in this case accepts as input the results of each sub-band after the multiplier operation 425, and is again used to adapt the post filtering block 435.
The Cascaded Hybrid Neural Network (CHNN), designed specifically for sub-band signal processing, can be used to implement a beamforming filter. The CHNN
consists of two classical neural networks -the Self-Organising Map (SOM) and Radial Basis Function Network (RBFN) -connected in a tapped-delay line structure (for example, see "Adaptive Noise Reduction Using a Cascaded Hybrid Neural Network", E. Chau, MSc. Thesis, School of Engineering, University of Guelph, 2001. The neural network can also be used to provide integrated functions of the ANC, the beamforming filter and other signal processing algorithms in the sub-band signal processing system.
While the present invention has been described with reference to specific embodiments, the description is illustrative of the invention and is not to be construed as limiting the invention. Various modifications may occur to those skilled in the art without departing from the true spirit and scope of the invention as defined by the appended claims.
The complex-valued gain factors of the beamforming filter can be derived in a number of ways. For example, if an analogue filterhas been designed, then it can be implemented directly in sub-bands by simply using the centre frequency of each sub-band to look up the corresponding complex response of the analogue filter (frequency sampling). With sufficiently narrow sub-bands, this method can create a close digital equivalent of the analogue filter. In a further embodiment of the invention, to closely approximate the ideal phase and amplitude responses for wider sub-bands, a narrowband filter to each sub-band output is applied as will now be described in relation to Figure 8 in which again, many of the components are the same as for the earlier Figure 7, and for which those same components are for convenience and clarity referred to by the same reference numbers. The additional functim for this embodiment is performed in the Narrowband Prototype Filters 815. To approximate an ideal linear phase response of the beamforrner, the filters 815 are designed as all-pass with a narrowband linear phase response. In a further embodiment, the filtrs are further constrained to being identical, and are moved back before the FFT
modulation stage by combining its impulse response with the filterbank prototype window.
One possible combination is a time convolution of the filterbank prototype window with a fractional delay impulse response. As a means of eliminating the external noise at the acoustic output stage, an Active Noise Cancellation (ANC) module is optionally added to the system in a manner similar to the system described in a co-pending patent application "Sound Intelligibility Enhancement Using a Psychoacoustic Model and an Oversampled Filterbank", T. Schneider et. al., Canadian Patent Application, seria12,354,755, US Patent Application Publication No. 20030198357. The ANC, as also shown in Figure 8, consists of a microphone 820 positioned at the output 490, plus a loop filter 830 to provide feedback to the combination matrix 415.
Almost all iunplementations of beamformers suffer from a low-frequency roll-off effect. To compensate for this effect, most systems, including the proposed system, introduce low-frequency amplification. However, because of the unavoidable microphone interna.l noise, this inherently leads to a high level of output noise at very low frequencies. As is well known, the result is that the desired beampattem can only be obtained for the frequencies above some cut-off value (usually around 1 kHz based on a particular microphone separation distance). In a further embodiment, shown in Figure 9, to avoid a high-level of low-frequency noise, the microphone signals are separated into high frequency and'low-frequency components by high-pass filter (HPF) 920 and low-pass filter (LPF) 910. Again, many of the same components used in the preferred embodiment described with reference to Figure 7 are used, performing the same function, and are given the same reference numbers. The high frequency components output by the high pass filter 920 are processed by the beamforming filter 710, multiplier 7425, and Narrow band prototype filters *815, as before. The low-frequency components by-pass the beamforming filter 710, multiplier 7425, and Narrow band prototype filters 815, relying solely on the post-filter 435 to provide low-frequency signal enhancement.
Besides the conventional digital filter design methods, the beamformer filter 710 in Figure 7 can also be implemented using an Artificial Neural Network (ANN).
The ANN can be employed as a type of non-parametric; robust adaptive filter, and has been increasingly investigated as a viable signal processing approach. One further possible embodiment of the present invention is to implement a neural network as a complete beamforming filter, as shown in Figure 10. Once again-the same reference numbers as Figure 4 are used for those components that are unchanged in function. The neural network 1010 accepts inputs from the sub-bands output by the analysis filterbank, and uses these to control the multiplier 425 which affect those sub-bands. The post filter adaptor 455 in this case accepts as input the results of each sub-band after the multiplier operation 425, and is again used to adapt the post filtering block 435.
The Cascaded Hybrid Neural Network (CHNN), designed specifically for sub-band signal processing, can be used to implement a beamforming filter. The CHNN
consists of two classical neural networks -the Self-Organising Map (SOM) and Radial Basis Function Network (RBFN) -connected in a tapped-delay line structure (for example, see "Adaptive Noise Reduction Using a Cascaded Hybrid Neural Network", E. Chau, MSc. Thesis, School of Engineering, University of Guelph, 2001. The neural network can also be used to provide integrated functions of the ANC, the beamforming filter and other signal processing algorithms in the sub-band signal processing system.
While the present invention has been described with reference to specific embodiments, the description is illustrative of the invention and is not to be construed as limiting the invention. Various modifications may occur to those skilled in the art without departing from the true spirit and scope of the invention as defined by the appended claims.
Claims (28)
1. A directional signal processing system for beamforming a plurality of information signals, said directional signal processing system comprising:
a plurality of microphones;
an oversampled filterbank comprising an analysis filterbank for transforming said plurality of information signals in time domain from said microphones into a plurality of channel signals in transform domain, and a synthesis filterbank;
and a signal processor for processing said channel signals to beamform said information signals, said synthesis filterbank transforming outputs from said signal processor to a single output signal in said time domain.
a plurality of microphones;
an oversampled filterbank comprising an analysis filterbank for transforming said plurality of information signals in time domain from said microphones into a plurality of channel signals in transform domain, and a synthesis filterbank;
and a signal processor for processing said channel signals to beamform said information signals, said synthesis filterbank transforming outputs from said signal processor to a single output signal in said time domain.
2. A directional processing system as claimed in claim 1, wherein said transform domain is a frequency domain.
3. A directional processing system as claimed in claim 1 or 2, wherein said signal processor further comprises:
a multiplier for multiplying said channel signals with at least one weight factor for said beamforming; and an adaptive processor for adjusting said at least one weight factor based on outputs of said multiplier.
a multiplier for multiplying said channel signals with at least one weight factor for said beamforming; and an adaptive processor for adjusting said at least one weight factor based on outputs of said multiplier.
4. A directional processing system as claimed in claim 1 or 2, wherein said signal processor further comprises:
a fixed beamformer for processing said channel signals to beamform said information signals with a specific beampattern;
a multiplier for multiplying outputs from said fixed beamformer with at least one weight factor; and an adaptive processor for adjusting said at least one weight factor based on outputs of said multiplier.
a fixed beamformer for processing said channel signals to beamform said information signals with a specific beampattern;
a multiplier for multiplying outputs from said fixed beamformer with at least one weight factor; and an adaptive processor for adjusting said at least one weight factor based on outputs of said multiplier.
5. A directional processing system as claimed in claim 1 or 2, wherein said signal processor further comprises:
a beamforming filter for producing complex-valued gain factors based on said channel signals to achieve a specific beampattern; and a multiplier for applying said complex-valued gain factors to said channel signals.
a beamforming filter for producing complex-valued gain factors based on said channel signals to achieve a specific beampattern; and a multiplier for applying said complex-valued gain factors to said channel signals.
6. A directional processing system as claim 1 or 2, wherein said signal processor further comprises:
a multiplier for multiplying said channel signals with at least one beamforming filter tap; and an adaptive processor for adjusting said at least one beamforming filter tap based on outputs of said multiplier.
a multiplier for multiplying said channel signals with at least one beamforming filter tap; and an adaptive processor for adjusting said at least one beamforming filter tap based on outputs of said multiplier.
7. A directional processing system as claimed in claim 1 or 2, wherein said signal processor comprises:
a neural network for implementing beamform filtering; and a multiplier for multiplying said channel signals with an output of said neural network.
a neural network for implementing beamform filtering; and a multiplier for multiplying said channel signals with an output of said neural network.
8. A directional processing system as claimed in claim 7, wherein said neural network is a Cascaded Hybrid Neural Network.
9. A directional processing system as claimed in any one of claims 3, 4 and 6, further comprising:
a summation circuit for summing said outputs of said multiplier, and wherein said adaptive processor performs adaptive processing based on an output of said summation circuit.
a summation circuit for summing said outputs of said multiplier, and wherein said adaptive processor performs adaptive processing based on an output of said summation circuit.
10. A directional processing system as claimed in any one of claims 5, 7 and 8, further comprising:
a summation circuit for summing outputs of said multiplier.
a summation circuit for summing outputs of said multiplier.
11. A directional processing system as claimed in claim 3, wherein said signal processor includes:
an estimations block having a voice activity detector, a target-to jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said adaptive processor adjusts said at least one weight factor based on an output of said voice activity detector, an output of said target-to jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
an estimations block having a voice activity detector, a target-to jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said adaptive processor adjusts said at least one weight factor based on an output of said voice activity detector, an output of said target-to jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
12. A directional processing system as claimed in claim 5, wherein said signal processor includes:
an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said beamforming filter produces said complex-valued gain factors based on an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said beamforming filter produces said complex-valued gain factors based on an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
13. A directional processing system as claimed in claim 6, wherein said signal processor includes:
an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said adaptive processor adjusts said at least one beamforming filter tap based on an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein said adaptive processor adjusts said at least one beamforming filter tap based on an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
14. A directional processing system as claimed in claim 9, wherein said signal processor includes:
a module for processing an output of said summation circuit, said module includes an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein adaptive processing of said adaptive processor is controlled by an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
a module for processing an output of said summation circuit, said module includes an estimations block having a voice activity detector, a target-to-jammer ratio estimator, a signal-to-noise ratio estimator, or combinations thereof, and wherein adaptive processing of said adaptive processor is controlled by an output of said voice activity detector, an output of said target-to-jammer ratio estimator, an output of said signal-to-noise ratio estimator, or combinations thereof.
15. A directional processing system as claimed in any one of claims 1-10, further comprising:
a post-filter provided between said signal processor and said synthesis filterbank, and a post-filter adaptor for controlling said post-filter.
a post-filter provided between said signal processor and said synthesis filterbank, and a post-filter adaptor for controlling said post-filter.
16. A directional processing system as claimed in any one of claims 11-14, further comprising:
a post-filter provided between said estimations block and said synthesis filterbank, and a post-filter adaptor for controlling said post-filter based on said output of said voice activity detector, said output of said target-to-jammer ratio estimator, said output of said signal-to-noise ratio estimator, or combinations thereof.
a post-filter provided between said estimations block and said synthesis filterbank, and a post-filter adaptor for controlling said post-filter based on said output of said voice activity detector, said output of said target-to-jammer ratio estimator, said output of said signal-to-noise ratio estimator, or combinations thereof.
17. A directional processing system as claimed in any one of claims 1-16, further comprising:
an analog-to-digital convertor for converting said information signals in an analog form to a plurality of digital information signals to supply said digital information signals to said analysis filterbank.
an analog-to-digital convertor for converting said information signals in an analog form to a plurality of digital information signals to supply said digital information signals to said analysis filterbank.
18. A directional processing system as claimed in any one of claims 1-17, further comprising:
a digital-to-analog convertor for converting said single output signal in a digital form to an analog single output signal.
a digital-to-analog convertor for converting said single output signal in a digital form to an analog single output signal.
19. A directional processing system as claimed in any one of claims 1-18, further comprising:
a combination matrix for pre-processing said information signals in said time domain.
a combination matrix for pre-processing said information signals in said time domain.
20. A directional processing system as claimed in claim 19, wherein said combination matrix is a FIR filter.
21. A directional processing system as claimed in claim 19, wherein said combination matrix is an IIR filter.
22. A directional processing system as claimed in any one of claims 1-20, wherein said analysis filterbank applies at least one fractional delay impulse response to at least one filterbank prototype window.
23. A directional processing system as claimed in any one of claims 1-22, wherein said signal processor comprises:
a circuit for processing said channel signals to achieve a linear phase response within a channel, and wherein said circuit applies a filter to at least one of said channel signals.
a circuit for processing said channel signals to achieve a linear phase response within a channel, and wherein said circuit applies a filter to at least one of said channel signals.
24. A directional processing system as claimed in claim 23, wherein said filter is an IIR filter.
25. A directional processing system as claimed in any one of claims 1-24, wherein said signal processor further comprises:
a processing block for dividing said channel signals into two groups such that one of said two groups can be processed differently from the other group.
a processing block for dividing said channel signals into two groups such that one of said two groups can be processed differently from the other group.
26. A directional processing system as claimed in any one of claims 9, 10 and 14, wherein said signal processor further comprises:
a processing block for dividing said channel signals into two groups such that one of said two groups can be processed differently from the other group, and wherein said summation circuit receives said outputs of said multiplier and at least one of any of said channel signals that have been processed differently.
a processing block for dividing said channel signals into two groups such that one of said two groups can be processed differently from the other group, and wherein said summation circuit receives said outputs of said multiplier and at least one of any of said channel signals that have been processed differently.
27. A directional processing system as claimed in claim 25 or 26, wherein said processing block includes:
a high-pass filter for said one of said two groups, and a low-pass filter for said other group.
a high-pass filter for said one of said two groups, and a low-pass filter for said other group.
28. A directional processing system as claimed in any one of claims 1-27, further comprising an active noise cancellation (ANC) block, said ANC block including a further microphone provided at an output of said synthesis filterbank, and a loop filter for provide a feedback control based on an output from said further microphone.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA002594362A CA2594362C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
| CA002397009A CA2397009C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA002354858A CA2354858A1 (en) | 2001-08-08 | 2001-08-08 | Subband directional audio signal processing using an oversampled filterbank |
| CA2,354,858 | 2001-08-08 | ||
| CA002397009A CA2397009C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002594362A Division CA2594362C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2397009A1 CA2397009A1 (en) | 2003-02-08 |
| CA2397009C true CA2397009C (en) | 2007-10-23 |
Family
ID=25682678
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002397009A Expired - Lifetime CA2397009C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
| CA002594362A Expired - Lifetime CA2594362C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002594362A Expired - Lifetime CA2594362C (en) | 2001-08-08 | 2002-08-07 | Directional audio signal processing using an oversampled filterbank |
Country Status (1)
| Country | Link |
|---|---|
| CA (2) | CA2397009C (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2452945C (en) * | 2003-09-23 | 2016-05-10 | Mcmaster University | Binaural adaptive hearing system |
| US9947323B2 (en) * | 2016-04-01 | 2018-04-17 | Intel Corporation | Synthetic oversampling to enhance speaker identification or verification |
| EP3598777B1 (en) | 2018-07-18 | 2023-10-11 | Oticon A/s | A hearing device comprising a speech presence probability estimator |
-
2002
- 2002-08-07 CA CA002397009A patent/CA2397009C/en not_active Expired - Lifetime
- 2002-08-07 CA CA002594362A patent/CA2594362C/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| CA2397009A1 (en) | 2003-02-08 |
| CA2594362C (en) | 2009-12-15 |
| CA2594362A1 (en) | 2003-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7359520B2 (en) | Directional audio signal processing using an oversampled filterbank | |
| AU2002325101A1 (en) | Directional audio signal processing using an oversampled filterbank | |
| JP2004537944A6 (en) | Directional audio signal processing using oversampled filter banks | |
| US7110554B2 (en) | Sub-band adaptive signal processing in an oversampled filterbank | |
| US8184801B1 (en) | Acoustic echo cancellation for time-varying microphone array beamsteering systems | |
| Simmer et al. | Post-filtering techniques | |
| Elko | Microphone array systems for hands-free telecommunication | |
| US20060013412A1 (en) | Method and system for reduction of noise in microphone signals | |
| CA2652847C (en) | Blind signal extraction | |
| WO2008061534A1 (en) | Signal processing using spatial filter | |
| WO2002018969A1 (en) | System and method for processing a signal being emitted from a target signal source into a noisy environment | |
| CN108694956A (en) | Hearing device and correlation technique with adaptive sub-band beam forming | |
| Neo et al. | Robust microphone arrays using subband adaptive filters | |
| WO2019060251A1 (en) | Cost effective microphone array design for spatial filtering | |
| CA2397009C (en) | Directional audio signal processing using an oversampled filterbank | |
| Leese | Microphone arrays | |
| Masuyama et al. | Causal distortionless response beamforming by alternating direction method of multipliers | |
| Chau et al. | A subband beamformer on an ultra low-power miniature DSP platform | |
| Liu et al. | A novel method for partially adaptive broadband beamforming | |
| CA2397080C (en) | Sub-band adaptive signal processing in an oversampled filterbank | |
| ULTRA et al. | email address:{echau, hsheikh, rbrennan, tschneid}@ dspfactory. com | |
| Zhang et al. | Adaptive null-forming algorithm with auditory sub-bands | |
| Dugger et al. | An analog adaptive beamforming circuit for audio noise reduction | |
| Doclo et al. | Design of broadband speech beamformers robust against errors in the microphone array characteristics | |
| AU2004310722A1 (en) | Method and apparatus for producing adaptive directional signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKEX | Expiry |
Effective date: 20220808 |