Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
  • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
  • H04R25/40—Arrangements for obtaining a desired directivity characteristic
  • H04R25/407—Circuits for combining signals of a plurality of transducers
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
  • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
  • H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
  • H04R25/552—Binaural

Definitions

• the present invention is directed to the processing of acoustic signals, and more particularly, but not exclusively, relates to the localization and extraction of acoustic signals emanating from different sources.
• the difficulty of extracting a desired signal in the presence of interfering signals is a long-standing problem confronted by acoustic engineers.
• This problem impacts the design and construction of many kinds of devices such as systems for voice recognition and intelligence gathering.
• hearing aid devices do not permit selective amplification of a desired sound when cont.aminated by noise from a nearby source - - particularly when the noise is more intense.
• This problem is even more severe when the desired sound is a speech signal and the nearby noise is also a speech signal produced by multiple talkers (e.g. babble).
• noise refers to random or nondeterministic signals and alternatively or additionally refers to any undesired signals and/or any signals interfering with the perception of a desired signal.
• the noise cancellation provided by the beam-former varies with the location of the noise source in relation to the microphone array.
• R.W. Stadler and W.M. Rabinowitz, On the Potential of Fixed Arrays for Hearing Aids. 94 Journal Acoustical Society of America 1332 (September 1993). and W. Soede et al.. Development of a Directional Hearing Instrument Based on Array Technology. 94 Journal of Acoustical Society of America 785 (March 1993) are cited as additional background concerning the beamforming approach.
• Still another approach has been the application of two microphones displaced from one another to provide two signals to emulate certain aspects of the binaural hearing system common to humans and many types of animals.
• biologic binaural hearing Although certain aspects of biologic binaural hearing are not fully understood, it is believed that the ability to localize sound sources is based on evaluation by the auditory system of binaural time delays and sound levels across different frequency bands associated with each of the two sound signals. The localization of sound sources with systems based on these interaural time and intensity differences is discussed in W. Lindemann, Extension of a Binaural Cross-Correlation Model bv Contralateral Inhibition - 1. Simulation of Lateralization for Stationary Sismals, 80 Journal of the Acoustical Society of America 1608 (December 1986).
• the present invention relates to the processing of acoustic signals.
• Various aspects of the invention are novel, nonobvious, and provide various advantages. While the actual nature of the invention covered herein can only be determined with reference to the claims appended hereto, selected forms and features of the preferred embodiments as disclosed herein are described briefly as follows.
• One form of the present invention includes a unique signal processing technique for localizing and characterizing each of a number of differently located acoustic sources.
• This form may include two spaced apart sensors to detect acoustic output from the sources. Each, or one particular selected source may be extracted, while suppressing the output of the other sources.
• a variety of applications may benefit from this technique including hearing aids, sound location mapping or tracking devices, and voice recognition equipment, to name a few.
• a first signal is provided from a first acoustic sensor and a second signal from a second acoustic sensor spaced apart from the first acoustic sensor.
• the first and second signals each correspond to a composite of two or more acoustic sources that, in turn, include a plurality of interfering sources and a desired source.
• the interfering sources are localized by processing of the first and second signals to provide a corresponding number of interfering source signals.
• These signals each include a number of frequency components. One or more the frequency components are suppressed for each of the interfering source signals. This approach facilitates nulling a different frequency component for each of a number of noise sources with two input sensors.
• a further form of the present invention is a processing system having a pair of sensors and a delay operator responsive to a pair of input signals from the sensors to generate a number of delayed signals therefrom.
• the system also has a localization operator responsive to the delayed signals to localize the interfering sources relative to the location of the sensors and provide a plurality of interfering source signals each represented by a number of frequency components.
• the system further includes an extraction operator that serves to suppress selected frequency components for each of the interfering source signals and extract a desired signal corresponding to a desired source.
• An output device responsive to the desired signal is also included that provides an output representative of the desired source.
• This system may be incorporated into a signal processor coupled to the sensors to facilitate localizing and suppressing multiple noise sources when extracting a desired signal.
• Still another form is responsive to position-plus-frequency attributes of sound sources. It includes positioning a first acoustic sensor and a second acoustic sensor to detect a plurality of differently located acoustic sources.
• First .and second signals are generated by the first and second sensors, respectively, that receive stimuli from the acoustic sources.
• a number of delayed signal pairs are provided from the first and second signals that each correspond to one of a number of positions relative to the first and second sensors.
• the sources are localized as a function of the delayed signal pairs and a number of coincidence patterns. These patterns are position and frequency specific, and may be utilized to recognize and correspondingly accumulate position data estimates that map to each true source position. As a result, these patterns may operate as filters to provide better localization resolution and eliminate spurious data.
• a system in yet another form, includes two sensors each configured to generate a corresponding first or second input signal and a delay operator responsive to these signals to generate a number of delayed signals each corresponding to one of a number of positions relative to the sensors.
• the system also includes a localization operator responsive to the delayed signals for deter ⁇ iining the number of sound source localization signals. These localization signals are determined from the delayed signals and a number of coincidence patterns that each correspond to one of the positions. The patterns each relate frequency varying sound source location information caused by ambiguous phase multiples to a corresponding position to improve acoustic source localization.
• the system also has an output device responsive to the localization signals to provide an output corresponding to at least one of the sources.
• a further form utilizes two sensors to provide corresponding binaural signals from which the relative separation of a first acoustic source from a second acoustic source may be established as a function of time, and the spectral content of a desired acoustic signal from the first source may be representatively extracted. Localization and identification of the spectral content of the desired acoustic signal may be performed concurrently. This form may also successfully extract the desired acoustic signal even if a nearby noise source is of greater relative intensity.
• Another form of the present invention employs a first and second sensor at different locations to provide a binaural representation of an acoustic signal which includes a desired signal emanating from selected source and interfering signals emanating from several interfering sources.
• a processor generates a discrete first spectral signal and a discrete second spectral signal from the sensor signals.
• the processor delays the first and second spectral signals by a number of time intervals to generate a number of delayed first signals and a number of delayed second signals and provide a time increment signal.
• the time increment signal corresponds to separation of the selected source from the noise source.
• the processor generates an output signal as a function of the time increment signal, and an output device responds to the output signal to provide an output representative of the desired signal.
• An additional form includes positioning a first and second sensor relative to a first signal source with the first and second sensor being spaced apart from each other and a second signal source being spaced apart from the first signal source.
• a first signal is provided from the first sensor and a second signal is provided from the second sensor.
• the first and second signals each represents a composite acoustic signal including a desired signal from the first signal source and unwanted signals from other sound sources.
• a number of spectral signals are established from the first and second signals as functions of a number of frequencies.
• a member of the spectral signals representative of position of the second signal source is determined, and an output signal is generated from the member which is representative of the first signal source.
• This feature facilitates extraction of a desired signal from a spectral signal determined as part of the localization of the interfering source. This approach can avoid the extensive post-localization computations required by many binaural systems to extract a desired signal.
• An additional object is to provide a system for the localization and extraction of acoustic signals by detecting a combination of these signals with two differently located sensors.
• FIG. 1 is a diagrammatic view of a system of one embodiment of the present invention.
• FIG. 2 is a signal flow diagram further depicting selected aspects of the system of FIG.
• FIG. 3 is schematic representation of the dual delay line of FIG. 2.
• FIGS. 4A and 4B depict other embodiments of the present invention corresponding to hearing aid and computer voice recognition applications, respectively.
• FIG. 5 is a graph of a speech signal in the form of a sentence about 2 seconds long.
• FIG. 6 is a graph of a composite signal including babble noise and the speech signal of FIG. 5 at a 0 dB signal-to-noise ratio with the babble noise source at about a 60 azimuth relative to the speech signal source.
• FIG. 7 is a graph of a signal representative of the speech signal of FIG. 5 after extraction from the composite signal of FIG. 6.
• FIG. 8 is a graph of a composite signal including babble noise and the speech signal of FIG. 5 at a -30 dB signal-to-noise ratio with the babble noise source at a 2 degree azimuth relative to the speech signal source.
• FIG. 9 is a graphic depiction of a signal representative of the sample speech signal of FIG. 5 after extraction from the composite signal of FIG. 8.
• FIG. 10 is a signal flow diagram of another embodiment of the present invention.
• FIG. 11 is a partial, signal flow diagram illustrating selected aspects of the dual delay lines of FIG. 10 in greater detail.
• FIG. 12 is a diagram illustrating selected geometric features of the embodiment illustrated in FIG. 10 for a representative example of one of a number of sound sources.
• FIG. 13 is a signal flow diagram illustrating selected aspects of the localization operator of FIG. 10 in greater detail.
• FIG. 14 is a diagram illustrating yet another embodiment of the present invention.
• FIG. 15 is a signal flow diagram further illustrating selected aspects of the embodiment of FIG. 14.
• FIG. 16 is a signal flow diagram illustrating selected aspects of the localization operator of FIG. 15 in greater detail.
• FIG. 17 is a graph illustrating a plot of coincidence loci for two sources.
• FIG. 18 is a graph illustrating coincidence patterns for azimuth positions corresponding to -75°, 0°, 20°, and 75°.
• FIGs. 19-22 are tables depicting experimental results obtained with the present invention.
• Fig. 1 illustrates an acoustic signal processing system 10 of one embodiment of the present invention.
• System 10 is configured to extract a desired acoustic signal from source 12 despite interference or noise emanating from nearby source 14.
• System 10 includes a pair of acoustic sensors 22, 24 configured to detect acoustic excitation that includes signals from sources 12, 14. Sensors 22, 24 are operatively coupled to processor 30 to process signals received therefrom. Also, processor 30 is operatively coupled to output device 90 to provide a signal representative of a desired signal from source 12 with reduced interference from source 14 as compared to composite acoustic signals presented to sensors 22, 24 from sources 12, 14.
• Sensors 22, 24 are spaced apart from one another by distance D along lateral axis T.
• Midpoint M represents the halfway point along distance D from sensor 22 to sensor 24.
• Reference axis RI is aligned with source 12 and intersects axis T perpendicularly through midpoint M.
• Axis N is aligned with source 14 and also intersects midpoint M.
• Axis N is positioned to form angle A with reference axis RI.
• Fig. 1 depicts an angle A of about 20 degrees.
• reference axis RI may be selected to define a reference azimuthal position of zero degrees in .an azimuthal plane intersecting sources 12, 14; sensors 22, 24; and containing axes T, N, RI.
• source 12 is "on-axis” and source 14, as aligned with axis N, is “off-axis.”
• Source 14 is illustrated at about a 20 degree azimuth relative to source 12.
• sensors 22, 24 are fixed relative to each other and configured to move in tandem to selectively position reference axis RI relative to a desired acoustic signal source. It is also preferred that sensors 22, 24 be microphones of a conventional variety, such as omnidirectional dynamic microphones. In other embodiments, a different sensor type may be utilized as would occur to one skilled in the art.
• FIG. 2 a signal flow diagram illustrates various processing stages for the embodiment shown in FIG. 1. Sensors 22, 24 provide analog signals Lp(t) and Rp(t) corresponding to the left sensor 22, and right sensor 24, respectively. Signals Lp(t) and Rp(t) are initially input to processor 30 in separate processing channels L and R.
• signals Lp(t) and Rp(t) are conditioned and filtered in stages 32a, 32b to reduce aliasing, respectively.
• the conditioned signals Lp(t), Rp(t) are input to corresponding Analog to Digital (A/D) converters 34a, 34b to provide discrete signals Lp(k), Rp(k), where k indexes discrete sampling events.
• A/D stages 34a, 34b sample signals Lp(t) and Rp(t) at a rate of at least twice the frequency of the upper end of the audio frequency range to assure a high fidelity representation of the input signals.
• Discrete signals Lp(k) and Rp(k) are transformed from the time domain to the frequency domain by a short-term Discrete Fourier Transform (DFT) algorithm in stages 36a, 36b to provide complex-valued signals XLp(m) and XRp(m).
• DFT Discrete Fourier Transform
• frequencies M encompass the audible frequency range and the number of samples employed in the short- term analysis is selected to strike .an optimum balance between processing speed limitations and desired resolution of resulting output signals.
• an audio range of 0.1 to 6 kHz is sampled in A/D stages 34a, 34b at a rate of at least 12.5 kHz with 512 samples per short-term spectral analysis time frame.
• the frequency domain analysis may be provided by an analog filter bank employed before A/D stages 34a, 34b. It should be understood that the spectral signals XLp(m) and XRp(m) may be represented as arrays each having a lxM dimension corresponding to the different frequencies f m .
• FIG. 3 depicts two delay lines 42, 44 each having N number of delay stages. Each delay line 42, 44 is sequentially configured with delay stages O ⁇ through D ⁇ .
• Delay lines 42, 44 are configured to delay corresponding input signals in opposing directions from one delay stage to the next, and generally correspond to the dual hearing channels associated with a natural binaural hearing process.
• Delay stages D ⁇ , D2, D3, . . ., E>N-2 > Ojsf.i, and Djvj each delay an input signal by corresponding time delay increments ⁇ , T2, T3, . .
• XLp(m) is alternatively designated XLpl(m).
• XLpl(m) is sequentially delayed by time delay increments ⁇ i , T2, 13, . . ., ⁇ .2, ⁇ . , and ⁇ jsj to produce delayed outputs at the taps of delay line 42 which are respectively designated XLp (m), XLp3(m), Xlp 4 (m), . .
• XLpN- ⁇ m XLp N (m), and XLp N+1 (m); (collectively designated XLp'(m)).
• XRp(m) is alternatively designated XRp N+1 (m).
• XRp N+1 (m) is sequentially delayed by time delay increments increments ⁇ , 12, 13, . . ., TN_2, ⁇ N-l. and ⁇ jsT to produce delayed outputs at the taps of delay line 44 which are respectively designated: XRpN(m), XRpN-l( m ), XR p N-2( m ) 5 . .
• the input spectral signals and the signals from delay line 42, 44 taps are arranged as input pairs to operation array 46.
• a pair of taps from delay lines 42, 44 is illustrated as input pair P in FIG. 3.
• Operation array 46 has operation units (OP) numbered from 1 to N+l, depicted as OPI, OP2, OP3, OP4,..., OPN-2, OPN-1, OPN, OPN+1 and collectively designated operations OPi.
• Input pairs from delay lines 42, 44 correspond to the operations of array 46 as follows: OPlfXLpHm), XRp!( )], OP2[XLp 2 (m), XRp 2 (m)], OP3[XLp 3 (m), XRp3(m)J, OP4[XLp4(m), XRp4(m)],..., OPN-2 [XLp(N-2)( m ), XR p (N-2)( m) ] j OPN-1 [XLp(N-l)(m), XRp( -l)( m )j, OPN[XLp N (m), XRpN( m )], and OPN+l[
• Xp N (m), and Xp(N +1 )(m) (collectively designated Xp J (m)).
• each OPi of operation array 46 is defined to be representative of a different azimuthal position relative to reference axis R.
• This arrangement is analogous to the different interaural time differences associated with a natural binaural hearing system. In these natural systems, there is a relative position in each sound passageway within the ear that corresponds to a maximum "in phase" peak for a given sound source.
• each operation of array 46 represents a position corresponding to a potential azimuthal or angular position range for a sound source, with the center operation representing a source at the zero azimuth — a source aligned with reference axis R.
• the center operation representing a source at the zero azimuth — a source aligned with reference axis R.
• dual delay line 40 provides a two dimensional matrix of outputs with N+1 columns corresponding to Xp ! (m), and M rows corresponding to each discrete frequency f m of Xp x (m). This (N+l)xM matrix is determined for each short-term spectral analysis interval p. Furthermore, by subtracting XRp'(m) from XLp'(m), the denominator of each expression CEl, CE2 is arranged to provide a minimum value of Xp ! (m) when the signal pair is "in-phase" at the given frequency f m . Localization stage 70 uses this aspect of expressions CEl, CE2 to evaluate the location of source 14 relative to source 12.
• Xavgp 1 The resulting averages, Xavgp 1 are then time averaged over the P most recent spectral- analysis time frames indexed by p in accordance with:
• ⁇ p are empirically determined weighting factors.
• the ⁇ p factors are preferably between 0.85 p and 0.90 p , where p is the short-term spectral analysis time frame index.
• the X 1 are analyzed to determine the minimum value, minp 1 ).
• the index i of min(X 1 ), designated "I,” estimates the column representing the azimuthal location of source 14 relative to source 12.
• the spectral content of a desired signal from source 12 when approximately aligned with reference axis RI, can be estimated from In other words, the spectral signal output by array 46 which most closely corresponds to the relative location of the "off-axis" source 14 contemporaneously provides a spectral representation of a signal emanating from source 12.
• the signal processing of dual delay line 40 not only facilitates localization of source 14, but also provides a spectral estimate of the desired signal with only minimal post-localization processing to produce a representative output.
• Post-localization processing includes provision of a designation signal by localization stage 70 to conceptual "switch" 80 to select the output column Xp*(m) of the dual delay line
• the Xpl(m) is routed by switch 80 to an inverse Discrete Fourier Transform algorithm (Inverse DFT) in stage 82 for conversion from a frequency domain signal representation to a discrete time domain signal representation denoted as s(k).
• the signal estimate s(k) is then converted by Digital to Analog (D/A) converter 84 to provide an output signal to output device 90.
• Output device 90 amplifies the output signal from processor 30 with amplifier 92 and supplies the amplified signal to speaker 94 to provide the extracted signal from a source 12.
• the present invention provides for the extraction of desired signals even when the interfering or noise signal is of equal or greater relative intensity. By moving sensors 22, 24 in tandem the signal selected to be extracted may correspondingly be changed.
• the present invention may be employed in an environment having many sound sources in addition to sources 12, 14.
• the localization algorithm is configured to dynamically respond to relative positioning as well as relative strength, using automated learning techniques.
• the present invention is adapted for use with highly directional microphones, more than two sensors to simultaneously extract multiple signals, and various adaptive amplification and filtering techniques known to those skilled in the art.
• the present invention greatly improves computational efficiency compared to conventional systems by determining a spectral signal representative of the desired signal as part of the localization processing.
• an output signal characteristic of a desired signal from source 12 is determined as a function of the signal pair XLp (m), XRp ⁇ (m) corresponding to the separation of source 14 from source 12.
• the exponents in the denominator of CEl, CE2 correspond to phase difference of frequencies f m resulting from the separation of source 12 from 14.
• processor 30 implements dual delay line 40 and corresponding operational relationships CEl, CE2 to provide a means for generating a desired-signal by locating the position of an interfering signal source relative to the source of the desired signal.
• ⁇ j be selected to provide generally equal azimuthal positions relative to reference axis R. In one embodiment, this arrangement corresponds to the values of ⁇ j changing about 20% from the smallest to the largest value. In other embodiments, ⁇ are all generally equal to one another, simplifying the operations of array 46. Notably, the pair of time increments in the numerator of CEl, CE2 corresponding to the separation of the sources 12 and 14 become approximately equal when all values ⁇ j are generally the same.
• Processor 30 may be comprised of one or more components or pieces of equipment.
• the processor may include digital circuits, analog circuits, or a combination of these circuit types.
• Processor 30 may be programmable, an integrated state machine, or utilize a combination of these techniques.
• processor 30 is a solid state integrated digital signal processor circuit customized to perform the process of the present invention with a rmriimum of external components and connections.
• the extraction process of the present invention may be performed on variously arranged processing equipment configured to provide the corresponding functionality with one or more hardware modules, firmware modules, software modules, or a combination thereof.
• signal includes, but is not limited to, software, firmware, hardware, programming variable, communication channel, and memory location representations. Referring to FIG.
• System 110 includes eyeglasses G with microphones 122 and 124 fixed to glasses G and displaced from one another.
• Microphones 122, 124 are operatively coupled to hearing aid processor 130.
• Processor 130 is operatively coupled to output device 190.
• Output device 190 is positioned in ear E to provide an audio signal to the wearer.
• Microphones 122, 124 are utilized in a manner similar to sensors 22, 24 of the embodiment depicted by FIGS 1-3.
• processor 130 is configured with the signal extraction process depicted in of FIGS. 1-3.
• Processor 130 provides the extracted signal to output device 190 to provide an audio output to the wearer.
• the wearer of system 110 may position glasses G to align with a desired sound source, such as a speech signal, to reduce interference from a nearby noise source off axis from the midpoint between microphones 122, 124. Moreover, the wearer may select a different signal by realigning with another desired sound source to reduce interference from a noisy environment.
• Processor 130 and output device 190 may be separate units (as depicted) or included in a common unit worn in the ear.
• the coupling between processor 130 and output device 190 may be an electrical cable or a wireless transmission.
• sensors 122, 124 and processor 130 are remotely located and are configured to broadcast to one or more output devices 190 situated in the ear E via a radio frequency transmission or other conventional telecommunication method.
• FIG. 4B shows a voice recognition system 210 employing the present invention as a front end speech enhancement device.
• System 210 includes personal computer C with two microphones 222, 224 spaced apart from each other in a predetermined relationship. Microphones 222, 224 are operatively coupled to a processor 230 within computer C. Processor 230 provides an output signal for internal use or responsive reply via speakers 294a, 294b or visual display 296. An operator aligns in a predetermined relationship with microphones 222, 224 of computer C to deliver voice commands. Computer C is configured to receive these voice commands, extracting the desired voice command from a noisy environment in accordance with the process system of FIGS. 1-3.
• FIG. 10 depicts left “L” and right “R" input channels for signal processor 330 of system 310.
• Channels L, R each include an acoustic sensor 22, 24 that provides an input signal xmft), X Rn (t), respectively.
• Input signals XL ⁇ ( andx R ⁇ (t) correspond to composites of sounds from multiple acoustic sources located within the detection range of sensors 22, 24.
• sensors 22, 24 be standard microphones spaced apart from each other at a predetermined distance D. In other embodiments a different sensor type or arrangement may be employed as would occur to those skilled in the art.
• Sensors 22, 24 are operatively coupled to processor 330 of system 310 to provide input signals x ⁇ meet(t) .and X R réelle(t) to A D converters 34a, 34b.
• A/D converters 34a, 34b of processor 330 convert input signals xm( and XR mast(t) from an analog form to a discrete form as represented as x ⁇ till(k) and X R réelle(k), respectively; where "t" is the familiar continuous time domain variable and "k” is the familiar discrete sample index variable.
• a corresponding pair of preconditioning filters may also be included in processor 330 as described in connection with system 10.
• Delay operator 340 receives spectral signals X ⁇ personally(m) and X R n(m) from stages 36a, 36b, respectively.
• delay operator 340 may be described as a single dual delay line that simultaneously operates on M frequencies like dual delay line 40 of system 10.
• the pair of frequency components from DFT stages 36a, 36b corresponding to a given value of m are inputs into a corresponding one of dual delay lines 342.
• Each dual delay line 342 includes a left channel delay line 342a receiving a corresponding frequency component input from DFT stage 36a and right channel delay line 342b receiving a corresponding frequency component input from DFT stage 36b.
• Delay lines 342a, 342b each include an odd number I of delay stages 344 indexed by i- , 2, ..., I.
• the I number of delayed signal pairs are provided on outputs 345 of delay stages 344 and are correspondingly sent to complex multipliers 346.
• Multipliers 346 provide equalization weighting for the corresponding outputs of delay stages 344.
• Each delayed signal pair from corresponding outputs 345 has one member from a delay stage 344 of left delay line 342a and the other member from a delay stage 344 of right delay line 342b.
• Complex multipliers 346 of each dual delay line 342 output corresponding products of the I number of delayed signal pairs along taps 347.
• the I number of signal pairs from taps 347 for each dual delay line 342 of operator 340 are input to signal operator 350.
• the I number of pairs of multiplier taps 347 are each input to a different Operation Array (OA) 352 of operator 350.
• Each pair of taps 347 is provided to a different operation stage 354 within a corresponding operation array 352.
• OA Operation Array
• delay stages 344, multipliers 346, and operation stages 354 are shown corresponding to the two stages at either end of delay lines 342a, 342b and the middle stages of delay lines 342a, 342b.
• the intervening stages follow the pattern of the illustrated stages and are represented by ellipses to preserve clarity.
• D is the distance between sensors 22, 24; and
• c is the speed of sound.
• the azimuthal plane may be uniformly divided into I sectors with the azimuth position of each resulting sector being given by equation (3) as follows:
• the dual delay-line structure is similar to the embodiment of system 10, except that a different dual delay line is represented for each value of m and multipliers 346 have been included to multiply each corresponding delay stage 344 by an appropriate one of equalization factors ⁇ , (m); where i is the delay stage index previously described.
• elements ⁇ ,- (m) are selected to compensate for differences in the noise intensity at sensors 22, 24 as a function of both azimuth and frequency.
• Fig. 12 depicts sensors 22, 24 and a representative acoustic source SI within the range of reception to provide input signals x ,(t) and xjo,(t).
• equations (A3) and (A4) are given by equations (A3) and (A4), as follows:
• equation (A6) results as follows:
• Equation (A8) where, K is in units of inverse length and is chosen to provide a convenient amplitude level, the value of ⁇ / -I+ ; (m) is given by equation (A8) as follows:
• equations (7) and (8) further define certain terms of equations (5) and (6) as follows:
• Equation (9) is comparable to the expressions CEl and CE2 of system 10; however, equation (9) includes equalization elements ⁇ ,- (m) and is organized into a single expression. With the outputs from operation array 352, the simultaneous localization and identification of the spectral content of the desired signal may be performed with system 310. Localization and extraction with system 310 are further described by the signal flow diagram of Fig. 13 and the following mathematical model. By substituting equations (5) and (6) into equation (9), equation (10) results as follows:
• equation (11) further defines:
• equation (12) results as follows:
• Equation (14) is a double summation over time and frequency that approximates a double integration in a continuous time domain representation. Further defining the following vectors:
• g c (G 1 (l) ⁇ (l), ⁇ ⁇ (2) ⁇ (2) ⁇ ⁇ ( ) ⁇ ( ⁇ f), ⁇ 2 (l) ⁇ (l),..., ⁇ 2 ( ⁇ f) ⁇ ( ⁇ f) ⁇ w (l) ⁇ 2(l),..., ⁇ w (Af) ⁇ «(Af)) r .
• r l 7
• the localization procedure includes finding the position t n01se along the operation array 352 for each of the delay lines 342 that produces the minimum value of
• the azimuth position of the noise source may be determined with equation (3).
• the estimated noise location i n0 ⁇ se may be utilized for noise cancellation or extraction of the desired signal as further described hereinafter.
• Localization operator 360 embodies the localization technique of system 310.
• Fig. 13 further depicts operator 360 with coupled pairs of summation operators 362 and 364 for each value of integer index i; where z-l,...,I.
• summation operators 362 and 364 perform the operation corresponding to equation (14) to generate
• stage 366 The I number of values of
• Stage 366 compares the I number of
• the index value i—g is sent by stage 366 of localization unit 360 to extraction operator 380.
• LFT Inverse Fourier Transform
• extraction operator 380 preferably includes a multiplexer or matrix switch that has IxM complex inputs and M complex outputs; where a different set of M inputs is routed to the outputs for each different value of the index 7 in response to the output from stage 366 of localization operator 360.
• Stage 82 converts the M spectral components received from extraction unit 380 to transform the spectral approximation of the desired signal, S aggression(m), from the frequency domain to the time domain as represented by signal S aggression(k).
• Stage 82 is operatively coupled to digital- to-analog (D/A) converter 84.
• D/A converter 84 receives signal S aggression(k) for conversion from a discrete form to an analog form represented by S aggression(t).
• Signal S aggression(t) is input to output device 90 to provide an auditory representation of the desired signal or other indicia as would occur to those skilled in the art.
• Stage 82, converter 84, and device 90 are further described in connection with system 10.
• equation (21) Another form of expression of equation (9) is given by equation (21) as follows:
• X ( ) w ⁇ m)X (m) + W ⁇ t (m) X% (m) .
• Equation (9) may be equivalently modeled as a beamforming procedure that places a null at the location corresponding to the predominant noise source, while steering to the desired output signal S notebook(t).
• Fig. 14 depicts system 410 of still another embodiment of the present invention.
• System 410 is depicted with several reference numerals that are the same as those used in connection with systems 10 and 310 and are intended to designate like features.
• a number of acoustic sources 412, 414, 416, 418 are depicted in Fig. 14 within the reception range of acoustic sensors 22, 24 of system 410.
• the positions of sources 412, 414, 416, 418 are also represented by the azimuth angles relative to axis AZ that are designated with reference numerals 412a, 414a, 416a, 418a.
• angles 412a, 414a, 416a, 418a correspond to about 0°, +20°, +75°, and -75°, respectively.
• Sensors 22, 24 are operatively coupled to signal processor 430 with axis AZ extending about midway therebetween.
• Processor 430 receives input signals x ⁇ blanket(t), x Rn (t) from sensors 22, 24 corresponding to left channel L and right channel R as described in connection with system 310.
• Processor 430 processes signals Xi n (t), X Rn (t) and provides corresponding output signals to output devices 90, 490 operatively coupled thereto.
• System 410 includes D/A converters 34a, 34b and DFT stages 36a, 36b to provide the same left and right channel processing as described in connection with system 310.
• System 410 includes delay operator 340 and signal operator 350 as described for system 310; however it is preferred that equalization factors a., ⁇ m) (z-1, ..., 7) be set to unity for the localization processes associated with localization operator 460 of system 410.
• localization operator 460 of system 410 directly receives the output signals of delay operator 340 instead of the output signals of signal operator 350, unlike system 310.
• the localization technique embodied in operator 460 begins by establishing two- dimensional (2-D) plots of coincidence loci in terms of frequency versus azimuth position.
• the coincidence points of each loci represent a minimum difference between the left and right ch.annels for each frequency as indexed by m. This minimum difference may be expressed as the minimum magnitude difference between the frequency domain representations Xi p (i) (m) at each discrete frequency m, yielding M/2 potentially different loci. If the acoustic sources are spatially coherent, then these loci will be the same across all frequencies. This operation is described in equations (22)-(25) as follows:
• Fig. 17 illustrates a 2-D coincidence plot 500 in terms of frequency in Hertz (Hz) along the vertical axis and azimuth position in degrees along the horizontal axis.
• Plot 500 indicates two sources corresponding to the generally vertically aligned locus 512a at about -20 degrees and the vertically aligned locus 512b at about + 40 degrees.
• Plot 500 also includes misidentified or phantom source points 514a, 514b, 514c, 514d, 514e at other azimuths positions that correspond to frequencies where both sources have significant energy. For more than two differently located competing acoustic sources, an even more complex plot generally results.
• localization operator 460 integrates over time and frequency.
• the signals are not correlated at each frequency, the mutual interference between the signals can be gradually attenuated by the temporal integration.
• This approach averages the locations of the coincidences, not the value of the function used to determine the minima, which is equivalent to applying a Rronecker delta function, ⁇ (i-i ⁇ (m)) to ⁇ X tract (i) (m) and averaging the h(i-i n (m)) over time.
• T > 0, is an empirically determined threshold. While this approach assumes the inter- sensor delays are independent of frequency, it has been found that departures from this assumption may generally be considered negligible.
• equation (30) approximates integration over time.
• the peaks in H n ( ⁇ _ / ) represent the source azimuth positions. If there are Q sources, Q peaks in H ⁇ O d ) may generally be expected. When compared with the patterns ⁇ (i-izie(m)) at each frequency, not only is the accuracy of localization enhanced when more than one sound source is present, but also almost immediate localization of multiple sources for the current frame is possible. Furthermore, although a dominant source usually has a higher peak in HN O) than do weaker sources, the height of a peak in H/v( ⁇ ⁇ ) only indirectly reflects the energy of the sound source.
• the height is influenced by several factors such as the energy of the signal component corresponding to Qj relative to the energy of the other signal components for each frequency band, the number of frequency bands, and the duration over which the signal is dominant.
• each frequency is weighted equally in equation (28).
• masking of weaker sources by a dominant source is reduced.
• existing time-domain cross-correlation methods incorporate the signal intensity, more heavily biasing sensitivity to the dominant source.
• the interaural time difference is ambiguous for high frequency sounds where the acoustic wavelengths are less than the separation distance D between sensors 22, 24.
• Equation (31) provides a means to determine a predictive coincidence pattern for a given azimuth that accounts for these secondary relationships as follows:
• the graph 600 of Fig. 18 illustrates a number of representative coincidence patterns 612, 614, 616, 618 determined in accordance with equations (31) and (32); where the vertical axis represents frequency in Hz and the horizontal axis represents azimuth position in degrees. Pattern 612 corresponds to the azimuth position of 0°. Pattern 612 has a primary relationship corresponding to the generally straight, solid vertical line 612a and a number of secondary relationships corresponding to curved solid line segments 612b.
• patterns 614, 616, 618 correspond to azimuth positions of -75°, 20°, and 75° and have primary relationships shown as straight vertical lines 614a, 616a, 618a and secondary relationships shown as curved line segments 614b, 616b, 618b, in correspondingly different broken line formats.
• the vertical lines are designated primary contours and the curved line segments are designated secondary contours.
• each stencil is a predictive pattern of the coincidence points attributable to an acoustic source at the azimuth position of the primary contour, including phantom loci corresponding to other azimuth positions as a factor of frequency.
• the stencil pattern may be used to filter the data at different values of m.
• equation (33) is used in place of equation (30) when the second technique of integration over frequency is desired.
• both variables ⁇ ,- and ⁇ ,- are equivalent and represent the position in the dual delay-line. The difference between these variables is that ⁇ ,- indicates location along the dual delay-line by using its corresponding spatial azimuth, whereas ⁇ ,- denotes location by using the corresponding time-delay unit of value ⁇ ,- . Therefore, the stencil pattern becomes much simpler if the stencil filter function is expressed with ⁇ ,- as defined in the following equation (34):
• ⁇ relates to ⁇ d through equation (4).
• ⁇ d the range of valid ⁇ OT, ⁇ / is given by equation (35) as follows: -(ITD ⁇ n penetrate / 2 + ⁇ ,)/ M ⁇ y ⁇ ⁇ (ITD ⁇ M- /2- r,)/ m .
• y m ⁇ is an integer.
• ⁇ ⁇ / only shifts the coincidence pattern (or stencil pattern) along the ⁇ ,-axis without changing its shape.
• the approach characterized by equations (34) and (35) may be utilized as an alternative to separate patterns for each azimuth position of interest; however, because the scaling of the delay units ⁇ ,- is uniform along the dual delay-line, azimuthal partitioning by the dual delay-line is not uniform, with the regions close to the median plane having higher azimuthal resolution. On the other hand, in order to obtain an equivalent resolution in azimuth, using a uniform ⁇ ,- would require a much larger 7 of delay units than using a uniform ⁇ ,-.
• the signal flow diagram of Fig. 16 further illustrates selected details concerning localization operator 460.
• the delayed signal of pairs of delay stages 344 are sent to coincidence detection operators 462 for each frequency indexed to m to determine the coincidence points.
• Detection operators 462 determine the minima in accordance with equation (22) or (26).
• Each coincidence detection operator 462 sends the results ikie(m) to a corresponding pattern generator 464 for the given m.
• Generators 464 build a 2-D coincidence plot for each frequency indexed to m and pass the results to a corresponding summation operator 466 to perform the operation expressed in equation (28) for that given frequency.
• Summation operators 466 approximate integration over time.
• Summation operators 466 pass results to summation operator 468 to approximate integration over frequency.
• Operators 468 may be configured in accordance with equation (30) if artifacts resulting from the secondary relationships at high frequencies are not present or may be ignored. Alternatively, stencil filtering with predictive coincidence patterns that include the secondary relationships may be performed by applying equation (33) with summation operator 468. Referring back to Fig. 15, operator 468 outputs H N ( ⁇ CI ) to output device 490 to map corresponding acoustic source positional information.
• Device 490 preferably includes a display or printer capable of providing a map representative of the spatial arrangement of the acoustic sources relative to the predetermined azimuth positions.
• the acoustic sources may be localized and tracked dynamically as they move in space. Movement trajectories may be estimated from the sets of locations ⁇ (i-izie(m)) computed at each sample window «.
• output device 490 is preferably not included. In still other embodiments, output device 90 may not be included.
• the localization techniques of localization operator 460 are particularly suited to localize more than two acoustic sources of comparable sound pressure levels and frequency ranges, and need not specify an on-axis desired source. As such, the localization techniques of system 410 provide independent capabilities to localize and map more than two acoustic sources relative to a number of positions as defined with respect to sensors 22, 24. However, in other embodiments, the localization capability of localization operator 460 may also be utilized in conjunction with a designated reference source to perform extraction and noise suppression. Indeed, extraction operator 480 of the illustrated embodiment incorporates such features as more fully described hereinafter.
• operator 480 receives the outputs of signal operator 350 as described in connection with system 310, that presents corresponding signals X " noiseU (m), ( OT ) f or eac h frequency m. These signals include a component of the desired signal at frequency m as well as components from sources other than the one to be canceled.
• the equalization factors ⁇ ,(/w) need not be set to unity once localization has taken place.
• I min ⁇
• the original signal a s (m) X f s) (m) is included.
• the resulting beam pattern may at times amplify other less intense noise sources.
• the amount of noise amplification is larger than the amount of cancellation of the most intense noise source, further conditions may be included in operator 480 to prevent changing the input signal for that frequency at that moment.
• Processors 30, 330, 430 include one or more components that embody the corresponding algorithms, stages, operators, converters, generators, arrays, procedures, processes, and techniques described in the respective equations and signal flow diagrams in software, hardware, or both utilizing techniques known to those skilled in the art.
• Processors 30, 330, 430 may be of any type as would occur to those skilled in the art; however, it is preferred that processors 30, 330, 430 each be based on a solid-state, integrated digital signal processor with dedicated hardware to perform the necessary operations with a minimum of other components.
• Systems 310, 410 may be sized and adapted for application as a hearing aide of the type described in connection with Fig. 4A.
• sensors application 22, 24 are sized and shaped to fit in the pinnae of a listener, and the processor algorithms are adjusted to account for shadowing caused by the head and torso. This adjustment may be provided by deriving a Head-Related-Transfer-Function (HRTF) specific to the listener or from a population average using techniques known to those skilled in the art. This function is then used to provide appropriate weightings of the dual delay stage output signals that compensate for shadowing.
• HRTF Head-Related-Transfer-Function
• system 310, 410 are adapted to voice recognition systems of the type described in connection with Fig. 4B.
• systems 310, 410 may be utilized in sound source mapping applications, or as would otherwise occur to those skilled in the art.
• a signal processing system includes a first sensor configured to provide a first signal corresponding to an acoustic excitation; where this excitation includes a first acoustic signal from a first source and a second acoustic signal from a second source displaced from the first source.
• the system also includes a second sensor displaced from the first sensor that is configured to provide a second signal corresponding to the excitation.
• a processor responsive to the first and second sensor signals that has means for generating a desired signal with a spectrum representative of the first acoustic signal.
• This means includes a first delay line having a number of first taps to provide a number of delayed first signals and a second delay line having a number of second taps to provide a number of delayed second signals.
• the system also includes output means for generating a sensory output representative of the desired signal.
• a method of signal processing includes detecting an acoustic excitation at both a first location to provide a corresponding first signal and at a second location to provide a corresponding second signal.
• the excitation is a composite of a desired acoustic signal from a first source and an interfering acoustic signal from a second source that is spaced apart from the first source.
• This method also includes spatially localizing the second source relative to the first source as a function of the first and second signals and generating a characteristic signal representative of the desired acoustic signal during performance of this localization.
• a Sun Sparc-20 workstation was programmed to emulate the signal extraction process of the present invention.
• One loudspeaker (LI) was used to emit a speech signal and another loudspeaker (L2) was used to emit babble noise in a semi-anechoic room.
• Two microphones of a conventional type were positioned in the room and operatively coupled to the workstation. The microphones had an inter-microphone distance of about 15 centimeters and were positioned about 3 feet from LI. LI was aligned with the midpoint between the microphones to define a zero degree azimuth.
• L2 was placed at different azimuths relative to LI approximately equidistant to the midpoint between LI and L2. Referring to FIG.
• FIG. 6 depicts a composite signal from LI and L2.
• the composite signal includes babble noise from L2 combined with the speech signal depicted in FIG. 5.
• the babble noise and speech signal are of generally equal intensity (OdB) with L2 placed at a 60 degree azimuth relative to LI.
• FIG. 7 depicts the signal recovered from the composite signal of FIG. 6. This signal is nearly the same as the signal of FIG. 5.
• FIG. 8 depicts another composite signal where the babble noise is 3 OdB more intense than the desired signal of FIG. 5. Furthermore, L2 is placed at only a 2 degree azimuth relative to LI.
• FIG. 9 depicts the signal recovered from the composite signal of FIG. 8, providing a clearly intelligible representation of the signal of FIG. 5 despite the greater intensity of the babble noise from L2 and the nearby location.
• EXAMPLE TWO Experiments corresponding to system 410 were conducted with two groups having four talkers (2 male, 2 female) in each group. Five different tests were conducted for each group with different spatial configurations of the sources in each test. The four talkers were arranged in correspondence with sources 412, 414, 416, 418 of Fig. 14 with different values for angles 412a, 414a, 416a, and 418a in each test.
• the illustration in Fig. 14 most closely corresponds to the first test with angle 418a being -75 degrees , angle 412a being 0 degrees, angle 414a being +20 degrees, and angle 416a being +75 degrees.
• the coincident patterns 612, 614, 616, and 618 of Fig. 18 also correspond to the azimuth positions of -75 degrees, 0 degrees, +20 degrees, and +75 degrees.
• the experimental set-up for the tests utilized two microphones for sensors 22, 24 with an inter-microphone distance of about 144mm. No diffraction or shadowing effect existed between the two microphones, and the inter-microphone intensity difference was set to zero for the tests.
• the signals were low-pass filtered at 6 kHz and sampled at a 12.8-kHz rate with 16-bit quantization.
• a Wintel-based computer was programmed to receive the quantized signals for processing in accordance with the present invention and output the test results described hereinafter.
• a 20-ms segment of signal was weighted by a Hamming window and then padded with zeros to 2048 points for DFT, and thus the frequency resolution was about 6Hz.
• the dual delay-line used in the tests was azimuth-uniform.
• the coincidence detection method was based on minimum magnitude differences.
• Each of the five tests consisted of four subtests in which a different talker was taken as the desired source.
• the speech materials four equally-intense spondaic words
• the speech material was presented in free-field.
• the localization of the talkers was done using both the equation (30) and equation (33) techniques.
• the experimental results are presented in Tables I, II, III, and IV of FIGs. 19-22, respectively.
• the five tests described in Table I of FIG. 19 approximate integration over frequency by utilizing equation (30); and includes two male speakers Ml, M2 and two female speakers FI, F2.
• the five tests described in Table II of FIG. 20 are the same as Table I, except that integration over frequency was approximated by equation (33).
• the five tests described in Table III of FIG. 21 approximate integration over frequency by utilizing equation (30); and includes two different male speakers M3, M4 and two different female speakers F3, F4.
• the five tests described in Table IN of FIG. 22 are the same as Table III, except that integration over frequency was approximated by equation (33).
• the data was arranged in a matrix with the numbers on the diagonal line representing the degree of noise cancellation in dB of the desired source (ideally 0 dB) and the numbers elsewhere representing the degree of noise cancellation for each noise source.
• the next to the last column shows a degree of cancellation of all the noise sources lumped together, while the last column gives the net intelligibility- weighted improvement (which considers both noise cancellation and loss in the desired signal).
• the results generally show cancellation in the intelligibility-weighted measure in a range of about 3 ⁇ 11 dB, while degradation of the desired source was generally less than about 0.1 dB).
• the total noise cancellation was in the range of about 8—12 dB.
• Comparison of the various Tables suggests very little dependence on the talker or the speech materials used in the tests. Similar results were obtained from six-talker experiments. Generally, a 7-10 dB enhancement in the intelligibility- weighted signal-to-noise ratio resulted when there were six equally loud, temporally aligned speech sounds originating from six different loudspeakers.

Landscapes

• Engineering & Computer Science (AREA)
• Acoustics & Sound (AREA)
• Neurosurgery (AREA)
• Otolaryngology (AREA)
• Physics & Mathematics (AREA)
• General Health & Medical Sciences (AREA)
• Health & Medical Sciences (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Circuit For Audible Band Transducer (AREA)
• Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
• Stereophonic System (AREA)
• Stereophonic Arrangements (AREA)
• Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)

Applications Claiming Priority (3)

Publications (3)

Family

ID=22712122

Family Applications (1)

Country Status (9)

Families Citing this family (23)

Citations (2)

Family Cites Families (4)

• 1999
  • 1999-11-16 DE DE69939272T patent/DE69939272D1/de not_active Expired - Lifetime
  • 1999-11-16 DK DK99958975T patent/DK1133899T3/da active
  • 1999-11-16 AU AU16240/00A patent/AU748113B2/en not_active Ceased
  • 1999-11-16 WO PCT/US1999/026965 patent/WO2000030404A1/fr active IP Right Grant
  • 1999-11-16 AT AT99958975T patent/ATE404028T1/de not_active IP Right Cessation
  • 1999-11-16 CN CN99815559.4A patent/CN1333994A/zh active Pending
  • 1999-11-16 EP EP99958975A patent/EP1133899B1/fr not_active Expired - Lifetime
  • 1999-11-16 JP JP2000583297A patent/JP3745227B2/ja not_active Expired - Fee Related
  • 1999-11-16 CA CA002348894A patent/CA2348894C/fr not_active Expired - Fee Related

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events