EP1469702B1 - Feedback suppression - Google Patents

Feedback suppression Download PDF

Info

Publication number
EP1469702B1
EP1469702B1 EP04006073.3A EP04006073A EP1469702B1 EP 1469702 B1 EP1469702 B1 EP 1469702B1 EP 04006073 A EP04006073 A EP 04006073A EP 1469702 B1 EP1469702 B1 EP 1469702B1
Authority
EP
European Patent Office
Prior art keywords
acoustical
feedback
input
output
signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP04006073.3A
Other languages
German (de)
French (fr)
Other versions
EP1469702A2 (en
EP1469702A3 (en
Inventor
Hans-Ueli Roeck
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sonova Holding AG
Original Assignee
Sonova AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sonova AG filed Critical Sonova AG
Priority to DK04006073.3T priority Critical patent/DK1469702T3/en
Priority to EP04006073.3A priority patent/EP1469702B1/en
Publication of EP1469702A2 publication Critical patent/EP1469702A2/en
Publication of EP1469702A3 publication Critical patent/EP1469702A3/en
Application granted granted Critical
Publication of EP1469702B1 publication Critical patent/EP1469702B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/45Prevention of acoustic reaction, i.e. acoustic oscillatory feedback
    • H04R25/453Prevention of acoustic reaction, i.e. acoustic oscillatory feedback electronically

Definitions

  • the present invention deals with a method for suppressing feedback between an acoustical output of an electrical/acoustical output converter arrangement- also called “output converter” - and an acoustical input of a acoustical/electrical input converter arrangement - also called “input converter” - of a hearing device, wherein acoustical signals impinging on the input converter arrangement are converted into a first electrical signal, by a controllably variable transfer characteristic and which is dependent on the angle at which said acoustical signals impinge on the input converter.
  • the first electrical signal is processed and a resulting signal is applied to the output converter.
  • an electrical feedback-compensating signal generated in dependency of the result signal which is applied via a feedback signal path upstream the processing.
  • a unit to which the output of the input converter is input and which provides a signal transfer characteristic to its output which has an amplification dependent on spatial angle at which acoustical signals impinge on the acoustic input of the input converter is called a beamformer unit.
  • the transfer characteristic in polar representation is called the beam.
  • An adaptive beamformer unit is a beamformer unit, the beam generated therefrom being controllably variable.
  • the feedback-compensation process Due to the complex task of estimating the acoustical feedback signal to be suppressed e.g. by correlation at the feedback-compensator, the feedback-compensation process has a relatively long adaptation time constant to adapt from one feedback situation to be suppressed to another by appropriately varying its gain.
  • Such an adaptation time constant is customarily in the range of hundreds of milliseconds.
  • a behind-the-ear hearing device 3 with an input converter 5 applied at the pinna 1 of an individual experiences acoustical feedback signals to be suppressed from a distinct direction as shown at d1.
  • An in-the-ear hearing device 7 according to Fig. 2 which has, as an example, a vent 9 and two acoustical ports 11 to the input converter, experiences acoustical feedback signals to be suppressed from the distinct directions d2.
  • a further approach for suppressing acoustical feedback is to install high signal attenuation between the input and the output converters of the device for signals which impinge on the input converter under such distinct spatial angles. This accords with applying a beamformer technique generating a beam having zero or minimum amplification at such angles.
  • Hearing devices which have adaptive beamformer ability are known e.g. from the WO 00/33634 .
  • feedback-compensation techniques as e.g. known from the EP 0 656 737 or the WO 00/19605 with adaptive beamformer technique as e.g. known from the WO 00/33634 and thereby to place minimum amplification of the beam at those angles which are specific for acoustical feedback signals to be suppressed impinging on the input converter. This especially because these angles are clearly different from the target direction range within which maximum amplification of the beam is to be variably set.
  • the adaptation time constant of an adaptive beamformer unit is considerably smaller, in the range of single to few dozen milliseconds, than the adaption time constant of a feedback-compensator which is, as mentioned above, in the range of hundreds of milliseconds.
  • a beamformer unit is provided, the input thereof being operationally connected to two mutually distant microphones of an input converter.
  • two feedback-compensators are provided with inputs operationally connected to the input of the output converter. The respective output signals are superimposed to the respective output signals of the two microphones.
  • a third approach is proposed in M. Brandenstein et al. as mentioned and in W. Herboldt et al. "Computationally efficient frequency domain combination of acoustic echo cancellation and robust adaptive beamforming ".
  • a generalised side lobe cancelling technique for the beamformer is used whereat only a not-adaptive beamformer is placed upstream the compensation feedback path, thus eliminating the adaptation time problem as well as double computational load. Nevertheless, by this approach placing minimum amplification of the beam in the direction of acoustical feedback signal arrival may not be realised.
  • a method for suppressing acoustical feedback between an acoustical output of an electrical/acoustical output converter arrangement and an acoustical input of an acoustical/electrical input converter arrangement of a hearing device wherein acoustical signals impinging on the input converter arrangement are converted into a first electric signal by a controllably variable transfer characteristic which is dependent on the angle at which said acoustical signals impinge on said input converter arrangement.
  • the first electric signal is processed and a resulting signal is applied to the output converter.
  • the acoustical feedback to be suppressed is compensated by a feedback-compensating signal which is generated in dependency of the resulting signal and is fed back by a feedback signal path to a location along the signal path upstream the processing.
  • the feedback-compensating signal is fed back to the first electric signal - thus downstream the beamformer - and the adaptation rate of converting to variations of the transfer characteristic - and thus of beamforming - is controlled in dependency of gain along the feedback-compensator feedback signal path.
  • the adaptation rate of the adaptive beamformer unit the speed with which the beamformer unit reacts on an adaptation command to change beamforming operation as e.g. changing target enhancement or noise suppression direction.
  • the adaptation rate accords with an adaptation time constant to change from one beamforming polar pattern to another.
  • the compensator thereby estimates the prevailing situation of feedback to be suppressed e.g. by a correlation technique between the signal applied to the output converter and the signal received from the input converter as e.g. described in the EP 0 656 737 .
  • the adaptation rate of the compensator accords with an adaptation time constant too. Whenever the loop gain along the feedback-compensating feedback signal path increases, this is caused by an increasing amount of acoustical feedback to be suppressed and thus to be compensated.
  • the adaptation rate of the beamformer unit is to be slowed down so that the feedback-compensator feedback signal may model the response of the beamformer unit too.
  • the adaptation rate of converting i.e. of beamforming, is slowed down with increasing loop gain along the feedback signal path.
  • amplification of the transfer characteristic representing beamforming is minimized at one or more than one specific angles which accord to angles at which the feedback to be suppressed predominantly impinges on the input converter arrangement.
  • the compensator may still model the beamformer without losing the established minimum or minima in the direction of the said specific angles.
  • the acoustical feedback to be suppressed is a narrow band acoustical signal, thus in a further improvement of the method according to the present invention, it is not necessary - so as to deal with an acoustical feedback to be suppressed - to control and especially to slow down the adaptation rate of beamforming conversion in the entire frequency range beamforming is effective at, but it suffices to controllably adapt the adaptation rate of the beamforming conversion at frequencies which are significant for the acoustical feedback signal to be suppressed. Therefore, in a further preferred embodiment of the present invention, controlling of the adaptation rate of the beamforming conversion is performed frequency selectively.
  • the principal according to the present invention may be applied at hearing devices where signal processing is performed in analog technique, it is preferred to perform the method in devices where signal processing is performed digitally.
  • at least signal processing in the beamforming conversion as well as along the feedback-compensation feedback signal path is performed in frequency domain, whereby time domain to frequency domain conversion may be realised in a known manner, be it by FFT, DCT, wavelet transform or other suitable transforms.
  • the respective re-conversion for the signal applied to the output converter is performed with the respective inverse processes.
  • the adaptation rate is controlled at selected frequencies in dependency of the feedback-compensator gain at these selected frequencies.
  • beamforming is only effective with respect to the acoustical feedback to be suppressed at specific frequencies or at a specific frequency band on the one hand the control of the adaptation rate of beamforming is in fact only to be performed at these specific frequencies or for the addressed frequency band. Further, selecting minimum amplification at the specific feedback impingement angles must be provided at the beamformer only for the specific frequencies or for the frequency band of the feedback to be suppressed too. Thus, this leads to the recognition that in fact beamforming may be subdivided in beamforming for frequencies which are not significant for the acoustical feedback to be suppressed and beamforming for frequencies or the frequency band which is specific for the acoustical feedback signal to be suppressed.
  • beamforming in the addressed specific frequencies may be performed and its adaptation rate controlled independently from tailoring beamforming at frequencies which are not specific for the acoustical feedback signal to be suppressed.
  • This beamforming may be performed at adaption rates which are independent from feedback-compensation and thus faster and which generates a beam which is not dealing with the specific impinging angles of the acoustical feedback signal to be suppressed.
  • performing controlling of beamforming is done selectively at frequencies which are significant for the acoustical feedback to be suppressed.
  • Further preferred minimalising the amplification of the beamforming transfer characteristic is only done at specific angles in a frequency selective manner.
  • two independent beamforming actions are superimposed, a first dealing with the generically desired beamforming behaviour, a second dealing with acoustical feedback-suppression as concerns frequencies and as concerns beamshaping. It becomes possible e.g. to switch off first beamforming, thereby maintaining the second and thereby preventing acoustical feedback to become effective.
  • the method according to the present invention may be applied to behind-the-ear hearing devices or to in-the-ear hearing devices, monaural or binaural systems, and further may be applied to such devices which are conceived as ear protection devices i.e. protecting the human ear from excess acoustical load, or to hearing improvement devices be it just to improve or facilitate hearing by an individual, or in the sense of a hearing aid, to improve hearing of a hearing impaired individual.
  • ear protection devices i.e. protecting the human ear from excess acoustical load
  • hearing improvement devices be it just to improve or facilitate hearing by an individual, or in the sense of a hearing aid, to improve hearing of a hearing impaired individual.
  • a hearing device which comprises:
  • Fig. 3 there is schematically shown, by means of a signal flow-/functional block-diagram a device according to the present invention, whereat the method according to the invention is realised.
  • the device comprises an acoustical/electrical input converter arrangement 10, which cooperates with a beamformer unit 12.
  • the conversion characteristics of the input converter 10 together with signal processing in beamformer unit 12 provides a beamformer characteristic between acoustical input E 10 to input converter 10 and electrical output A 12 of the beamformer unit 12.
  • the beamformer unit 12 has an adaptation control input C 12A and an adaptation rate control input C 12R .
  • the transfer characteristic between E 10 and A 12 has an amplification which is dependent on the angle ⁇ at which acoustical signals impinge on the acoustical port of input converter 10.
  • the angle at which acoustical signals impinge on the acoustical port of input converter 10.
  • the transfer characteristic in polar representation the beam B, may be varied with respect to its characteristics as e.g. with respect to target direction, maximum amplification etc. as shown in dotted line within block 12.
  • Variation of the beam characteristic B is controlled by control input C 12A which latter is, as shown in dotted line, normally connected to a processing unit 14 for adapting the beam characteristic B e.g. to prevailing acoustical situations automatically or program controlled or by an individual wearing the hearing device.
  • Beamforming units which may be adapted are known. One example thereof is described in the WO 00/33634 .
  • Variation of the beam characteristic B may also be, caused at the beamformer itself, i.e. by beamformer internal reasons.
  • the input C 12A and control signals applied thereto are merely a schematic representation of beam characteristic variation ability or occurrence.
  • the electrical output of beamforming unit 12, A 12 is operationally connected to an input E 14 , of the signal processing 14 unit whereat input signals are processed and output at an output A 14 operationally connected to an electric input E 16 of an output electrical/acoustical converter arrangement 16 so as to provide desired ear protection or hearing improvement to the individual carrying such device.
  • ear protecting ability the ability of reducing or even cancelling acoustical signals which impinge on the input converter arrangement 10, so as to protect individual's hearing or even provide the individual with silent perception in non-vanishing acoustical surrounding.
  • hearing improvement we understand the improvement of individual's hearing in an acoustical surrounding, be it for customary applications of normal hearing individual or be it in the sense of hearing aid to improve individual's impaired hearing.
  • acoustical feedback AFB between the acoustical output of the output converter 16 and acoustical input E 10 of the input converter 10.
  • a feedback-compensator 18 whereat the prevailing acoustical feedback AFB, which is to be suppressed, is estimated e.g. with a correlation technique, correlating the signal applied to output converter 16 with a signal dependent on the output of input converter 10 as shown in dashed line at A.
  • the gain G of feedback-compensator 18 is estimated so a to compensate for the AFB by negative feedback.
  • the feedback-compensator 18 By means of feedback-compensator 18, a signal as predicted is fed back to the input of processor unit 14 downstream the output of beamformer unit 12 so as to compensate for the feedback AFB. As shown in Fig. 3 , the feedback-compensator 18 has an input E18 which is operationally connected to the output A 14 of the processing unit 14 and has an output A 18 which is superimposed to the output E 12 of beamformer unit 12, the result of such superimposing at a unit 20 being input to input E 14 of processing unit 14.
  • the feedback-compensator 18 which computes estimation of the acoustical feedback to be suppressed, has an adaptation rate in the range of several hundred ms and is thus considerably slower than the adaptation rate of beamforer unit 12.
  • the feedback-compensator 18 will not be able to accurately rapidly deal with the varied situation with respect to acoustical feedback AFB.
  • a control of the adaptation rate of beamformer unit 12 which control is performed by the feedback-compensator 18, according to Fig. 3 at control input C 12R .
  • the adaptation rate or time constant of beamformer unit 12 is lowered to or below the adaptation rate of compensator unit 18.
  • the loop gain may at be least estimated e.g. by multiplying the linear gains along the loop, primarily consisting of the feedback-compensator 18 and the processing unit 14 in Fig. 3 or by adding these gains in dB.
  • adaptation rate control of beamformer unit 12 is performed in dependency of the loop gain along the feedback loop with feedback-compensator 18.
  • the rate control input C 12R to beamforming unit 12 is operationally connected to a loop gain output A G of feedback-compensator 18.
  • the direction with which acoustical feedback signals AFB to be suppressed impinge on the acoustical port of the input converter 10 is specific. Therefore, at the beamformer unit 12, there is generated a beam characteristic B AFB , as shown in Fig. 4 , which has minimum amplification for these specific angles or, as shown e.g. for an in-the-ear hearing device, at two specific angles ⁇ AFB .
  • beamforming is realised with minimum amplification for those spatial angles ⁇ AFB with which the acoustical feedback AFB to be suppressed impinges on the input converter 10.
  • acoustical feedback AFB to be suppressed occurs substantially within a specific frequency band.
  • This frequency band is dependent, among others, on the specific output converter 16 used, the type of device e.g. in-the-ear or outside-the-ear device. Therefore, in a further improved embodiment, overall feedback-suppression may be performed within that specific frequency band, thereby leaving beamforming in frequencies not within this specific frequency band unaffected and tailored according to needs different from acoustic feedback-suppression.
  • beamforming B AFB for minimum amplification of acoustical feedback AFB to be suppressed is performed frequency selectively for frequencies f AFB of the acoustical feedback signal AFB.
  • Beamforming for frequencies f AFB which are not significantly present in the acoustical feedback AFB is performed by a second beamforming B AFB which may be selected independently from B AFB , according to fig. 5(b) .
  • Frequency selective feedback-compensation and adaptation beamforming may easily be realised, if at least beamforming in unit 12 as well as feedback-compensation in feedback-compensator 18 are performed in frequency domain or, respectively, in sub-bands. Beamforming is then realised at the frequencies f AFB with minimum amplification at the specific angles ⁇ AFB , whereas beamforming at other frequencies f AFB is performed according to other needs. Consequently the adaptation rate of beamforming in unit 12 is only controlled by the gain of feedback-compensator 18 at the frequencies f AFB .
  • beamforming B AFB may be maintained active to suppress acoustical feedback also in such "quiet" mode.
  • the loop gain as estimated in feedback-compensator 18, may be compared with a threshold value and adaptation rate control at C 12R is only established, if the instantaneous loop gain at least reaches such threshold.
  • the control of the adaptation rate may then be lowered to practically zero, which means that beamforming is switched off for frequencies f AFB .
  • such switching may be performed steadily which may be realised on the one hand by lowering the adaptation rate of B AFB steadily and/or by reducing beamforming amplification of B AFB steadily.

Description

  • The present invention deals with a method for suppressing feedback between an acoustical output of an electrical/acoustical output converter arrangement- also called "output converter" - and an acoustical input of a acoustical/electrical input converter arrangement - also called "input converter" - of a hearing device, wherein acoustical signals impinging on the input converter arrangement are converted into a first electrical signal, by a controllably variable transfer characteristic and which is dependent on the angle at which said acoustical signals impinge on the input converter. The first electrical signal is processed and a resulting signal is applied to the output converter. There is further provided an electrical feedback-compensating signal, generated in dependency of the result signal which is applied via a feedback signal path upstream the processing.
  • Definition
  • A unit to which the output of the input converter is input and which provides a signal transfer characteristic to its output which has an amplification dependent on spatial angle at which acoustical signals impinge on the acoustic input of the input converter is called a beamformer unit. The transfer characteristic in polar representation is called the beam.
  • An adaptive beamformer unit is a beamformer unit, the beam generated therefrom being controllably variable.
  • From the EP 0 656 737 and the WO 00/19605 there is known such a method which nevertheless does not apply beamforming. The input of a feedback-compensator is operationally connected to the input of the output converter of the device, the output of the feedback-compensator is operationally connected to the output of the input converter, thereby forming a feedback signal path.
  • Due to the complex task of estimating the acoustical feedback signal to be suppressed e.g. by correlation at the feedback-compensator, the feedback-compensation process has a relatively long adaptation time constant to adapt from one feedback situation to be suppressed to another by appropriately varying its gain. Such an adaptation time constant is customarily in the range of hundreds of milliseconds.
  • Acoustical feedback signals to be suppressed impinge upon the input converter substantially from distinct spatial angles. As schematically shown in Fig. 1, a behind-the-ear hearing device 3 with an input converter 5 applied at the pinna 1 of an individual, experiences acoustical feedback signals to be suppressed from a distinct direction as shown at d1. An in-the-ear hearing device 7 according to Fig. 2 which has, as an example, a vent 9 and two acoustical ports 11 to the input converter, experiences acoustical feedback signals to be suppressed from the distinct directions d2.
  • Therefore, a further approach for suppressing acoustical feedback is to install high signal attenuation between the input and the output converters of the device for signals which impinge on the input converter under such distinct spatial angles. This accords with applying a beamformer technique generating a beam having zero or minimum amplification at such angles.
  • Hearing devices which have adaptive beamformer ability are known e.g. from the WO 00/33634 . For acoustical feedback suppression at a hearing device with adaptive beamforming ability, it seems, at first, quite straight forward to combine on the one hand feedback-compensation techniques as e.g. known from the EP 0 656 737 or the WO 00/19605 with adaptive beamformer technique as e.g. known from the WO 00/33634 and thereby to place minimum amplification of the beam at those angles which are specific for acoustical feedback signals to be suppressed impinging on the input converter. This especially because these angles are clearly different from the target direction range within which maximum amplification of the beam is to be variably set.
  • Thereby, it has to be noted that the adaptation time constant of an adaptive beamformer unit is considerably smaller, in the range of single to few dozen milliseconds, than the adaption time constant of a feedback-compensator which is, as mentioned above, in the range of hundreds of milliseconds.
  • One approach is known where a beamformer unit is provided, the input thereof being operationally connected to two mutually distant microphones of an input converter. As both spaced apart microphones experience the feedback signal to be suppressed differently, two feedback-compensators are provided with inputs operationally connected to the input of the output converter. The respective output signals are superimposed to the respective output signals of the two microphones.
  • The fact that the adaptation time constant of the beamformer unit is much shorter than the adaptation time constant of the compensators does not pose a problem in this configuration, because the fast adapting beamformer unit is placed within the closed feedback loop formed by the feedback-compensation feedback paths.
  • Nevertheless, this known approach has the serious drawback that for each of the microphones one compensator feedback path must be provided which unacceptably raises computational load.
  • A further approach for beamformer/feedback-compensation combination is known from M. Brandenstein et al. "Microphone arrays", Springer Verlag 2001. Here the feedback-compensation path is fed back to the output of the beamformer unit. By this approach only one compensation path is necessary and thus computational load is reduced. Nevertheless, here the fast adapting beamformer is outside the negative feedback loop. Thus, whenever the adaptive beamformer is controlled to rapidly change its beam pattern, the compensator will not be able to adequately rapidly deal with the new situation of feedback to be suppressed.
  • Therefore, M. Brandenstein et al. "Microphone arrays" considers this approach as, at least, very difficult to realise.
  • A third approach is proposed in M. Brandenstein et al. as mentioned and in W. Herboldt et al. "Computationally efficient frequency domain combination of acoustic echo cancellation and robust adaptive beamforming". A generalised side lobe cancelling technique for the beamformer is used whereat only a not-adaptive beamformer is placed upstream the compensation feedback path, thus eliminating the adaptation time problem as well as double computational load. Nevertheless, by this approach placing minimum amplification of the beam in the direction of acoustical feedback signal arrival may not be realised.
  • It is an object of the present invention to provide a method for suppressing acoustical feedback as addressed above at a hearing device which has an adaptive beamformer on the one hand, and a feedback-compensator on the other hand, thereby avoiding the drawbacks as addressed above.
  • This is achieved on the one hand by superimposing the fed back feedback-compensating signal to the signal downstream the beamforming, and, on the other hand, by controlling the adaptation rate of beamforming in dependency of the gain along feedback signal path with the compensator.
  • Thus, there is proposed a method for suppressing acoustical feedback between an acoustical output of an electrical/acoustical output converter arrangement and an acoustical input of an acoustical/electrical input converter arrangement of a hearing device, wherein acoustical signals impinging on the input converter arrangement are converted into a first electric signal by a controllably variable transfer characteristic which is dependent on the angle at which said acoustical signals impinge on said input converter arrangement. The first electric signal is processed and a resulting signal is applied to the output converter. The acoustical feedback to be suppressed is compensated by a feedback-compensating signal which is generated in dependency of the resulting signal and is fed back by a feedback signal path to a location along the signal path upstream the processing. Thereby, the feedback-compensating signal is fed back to the first electric signal - thus downstream the beamformer - and the adaptation rate of converting to variations of the transfer characteristic - and thus of beamforming - is controlled in dependency of gain along the feedback-compensator feedback signal path.
  • Definition
  • We understand by the adaptation rate of the adaptive beamformer unit the speed with which the beamformer unit reacts on an adaptation command to change beamforming operation as e.g. changing target enhancement or noise suppression direction. The adaptation rate accords with an adaptation time constant to change from one beamforming polar pattern to another.
  • We understand by the adaptation rate of feedback-compensating the rate with which the respective feedback-compensator reacts on a detected change of acoustical feedback situation until the compensator has settled to a new setting. The compensator thereby estimates the prevailing situation of feedback to be suppressed e.g. by a correlation technique between the signal applied to the output converter and the signal received from the input converter as e.g. described in the EP 0 656 737 . The adaptation rate of the compensator accords with an adaptation time constant too. Whenever the loop gain along the feedback-compensating feedback signal path increases, this is caused by an increasing amount of acoustical feedback to be suppressed and thus to be compensated. This means that the adaptation rate of the beamformer unit is to be slowed down so that the feedback-compensator feedback signal may model the response of the beamformer unit too. Thus, in a preferred embodiment, the adaptation rate of converting, i.e. of beamforming, is slowed down with increasing loop gain along the feedback signal path.
  • As was addressed above, feedback signals, which are acoustical and which have to be suppressed, impinge on the acoustical input of the input converter arrangement substantially and dependent on the specific device at specific angles. Thus, in a most preferred embodiment of the method according to the present invention, amplification of the transfer characteristic representing beamforming is minimized at one or more than one specific angles which accord to angles at which the feedback to be suppressed predominantly impinges on the input converter arrangement.
  • Thus, and considered in combination with slowing down the adaptation rate of beamforming with increasing gain along feedback-compensation feedback signal path, it becomes apparent that the compensator may still model the beamformer without losing the established minimum or minima in the direction of the said specific angles.
  • Further, it has to be noted that the acoustical feedback to be suppressed is a narrow band acoustical signal, thus in a further improvement of the method according to the present invention, it is not necessary - so as to deal with an acoustical feedback to be suppressed - to control and especially to slow down the adaptation rate of beamforming conversion in the entire frequency range beamforming is effective at, but it suffices to controllably adapt the adaptation rate of the beamforming conversion at frequencies which are significant for the acoustical feedback signal to be suppressed. Therefore, in a further preferred embodiment of the present invention, controlling of the adaptation rate of the beamforming conversion is performed frequency selectively.
  • In spite of the fact that the principal according to the present invention may be applied at hearing devices where signal processing is performed in analog technique, it is preferred to perform the method in devices where signal processing is performed digitally. Thereby, and in view of the addressed preferred frequency selective control, in a most preferred embodiment, at least signal processing in the beamforming conversion as well as along the feedback-compensation feedback signal path, is performed in frequency domain, whereby time domain to frequency domain conversion may be realised in a known manner, be it by FFT, DCT, wavelet transform or other suitable transforms. The respective re-conversion for the signal applied to the output converter is performed with the respective inverse processes. The adaptation rate is controlled at selected frequencies in dependency of the feedback-compensator gain at these selected frequencies. Thereby the following approach is achieved:
  • As beamforming is only effective with respect to the acoustical feedback to be suppressed at specific frequencies or at a specific frequency band on the one hand the control of the adaptation rate of beamforming is in fact only to be performed at these specific frequencies or for the addressed frequency band. Further, selecting minimum amplification at the specific feedback impingement angles must be provided at the beamformer only for the specific frequencies or for the frequency band of the feedback to be suppressed too. Thus, this leads to the recognition that in fact beamforming may be subdivided in beamforming for frequencies which are not significant for the acoustical feedback to be suppressed and beamforming for frequencies or the frequency band which is specific for the acoustical feedback signal to be suppressed. Thus, beamforming in the addressed specific frequencies may be performed and its adaptation rate controlled independently from tailoring beamforming at frequencies which are not specific for the acoustical feedback signal to be suppressed. This beamforming may be performed at adaption rates which are independent from feedback-compensation and thus faster and which generates a beam which is not dealing with the specific impinging angles of the acoustical feedback signal to be suppressed.
  • Therefore, in a further preferred embodiment of the method according to the present invention, performing controlling of beamforming is done selectively at frequencies which are significant for the acoustical feedback to be suppressed. Further preferred minimalising the amplification of the beamforming transfer characteristic is only done at specific angles in a frequency selective manner. In fact two independent beamforming actions are superimposed, a first dealing with the generically desired beamforming behaviour, a second dealing with acoustical feedback-suppression as concerns frequencies and as concerns beamshaping. It becomes possible e.g. to switch off first beamforming, thereby maintaining the second and thereby preventing acoustical feedback to become effective. The method according to the present invention may be applied to behind-the-ear hearing devices or to in-the-ear hearing devices, monaural or binaural systems, and further may be applied to such devices which are conceived as ear protection devices i.e. protecting the human ear from excess acoustical load, or to hearing improvement devices be it just to improve or facilitate hearing by an individual, or in the sense of a hearing aid, to improve hearing of a hearing impaired individual.
  • It is to be noted that feedback caused not by acoustical but by electrical or mechanical reasons is often fed into the microphones of the input converter with equal gains and phases, thus appearing to originate from a direction perpendicular to the port axis of the input converter arrangement. In an endfire array, as typically used in hearing instruments, this conforms to a 90° direction or arrival, and may be suppressed by a technique according to the present invention as well.
  • Thus, when we speak of "acoustical feedback" to be suppressed we understand also such other feedback fed into the microphones.
  • To resolve the object as mentioned above, there is further, and according to the present invention, provided a hearing device which comprises:
    • an acoustical/electrical input converter arrangement and a adaptive beamformer unit generating at an output an electric output signal dependent on acoustical signals impinging on said acoustical/electrical input converter arrangement and in dependency of angle at which said acoustical signals impinge, said beamformer unit having a first control input for varying beamforming characteristics and a second control input for controllably adjusting adaptation rate;
    • a processing unit with an input operationally connected to the output of said beamformer unit with an output operationally connected to an input of an electrical/acoustical output converter arrangement;
    • a feedback-compensator unit, the input thereof being operationally connected to said input of said electrical/acoustical output converter arrangement, the output thereof being operationally connected to the input of said processing unit and having a loop gain output, said loop gain output being operationally connected to said second control input of said beamformer unit.
  • Preferred embodiments of the method according to the present invention, as well as of a hearing device according to the present invention, shall additionally become apparent from the following detailed description of preferred embodiments with the help of further figures and from the claims. The figures show:
  • Figs. 1 & 2:
    as discussed above, schematically specific angles at which acoustical feedback signals impinge on the acoustical input port of outside-the-ear (Fig. 1) and in-the-ear (Fig. 2) hearing devices.
    Fig. 3:
    by means of a simplified functional block/signal flow-diagram, a device according to the present invention operated according to the method of the present invention.
    Fig. 4:
    in polar diagram representation preferred beamforming at the device according to Fig. 3 taking into account specific angles with which the acoustical feedback to be suppressed impinges on the acoustic input as exemplified in the Figs. 1 or 2.
    Fig. 5a:
    as an example and quantitatively, beamforming by the device of Fig. 3 at specific frequencies which are significantly present in the feedback signal to be suppressed.
    Fig. 5b:
    beamforming at the device of Fig. 3 for frequencies which are not significantly present in the feedback signal to be suppressed.
  • In Fig. 3 there is schematically shown, by means of a signal flow-/functional block-diagram a device according to the present invention, whereat the method according to the invention is realised. The device comprises an acoustical/electrical input converter arrangement 10, which cooperates with a beamformer unit 12. The conversion characteristics of the input converter 10 together with signal processing in beamformer unit 12 provides a beamformer characteristic between acoustical input E10 to input converter 10 and electrical output A12 of the beamformer unit 12. The beamformer unit 12 has an adaptation control input C12A and an adaptation rate control input C12R.
  • The transfer characteristic between E10 and A12 has an amplification which is dependent on the angle α at which acoustical signals impinge on the acoustical port of input converter 10. Thus, there is generated by the combined units 10 and 12 a beam characteristic as exemplified with B in unit 12.
  • As further schematically shown by the variation arrow V within block 12, the transfer characteristic, in polar representation the beam B, may be varied with respect to its characteristics as e.g. with respect to target direction, maximum amplification etc. as shown in dotted line within block 12. Variation of the beam characteristic B is controlled by control input C12A which latter is, as shown in dotted line, normally connected to a processing unit 14 for adapting the beam characteristic B e.g. to prevailing acoustical situations automatically or program controlled or by an individual wearing the hearing device. Beamforming units which may be adapted are known. One example thereof is described in the WO 00/33634 .
  • Variation of the beam characteristic B may also be, caused at the beamformer itself, i.e. by beamformer internal reasons.
  • Therefore, it must be emphasised that the input C12A and control signals applied thereto are merely a schematic representation of beam characteristic variation ability or occurrence.
  • The electrical output of beamforming unit 12, A12, is operationally connected to an input E14, of the signal processing 14 unit whereat input signals are processed and output at an output A14 operationally connected to an electric input E16 of an output electrical/acoustical converter arrangement 16 so as to provide desired ear protection or hearing improvement to the individual carrying such device. We understand under ear protecting ability the ability of reducing or even cancelling acoustical signals which impinge on the input converter arrangement 10, so as to protect individual's hearing or even provide the individual with silent perception in non-vanishing acoustical surrounding. Under hearing improvement, we understand the improvement of individual's hearing in an acoustical surrounding, be it for customary applications of normal hearing individual or be it in the sense of hearing aid to improve individual's impaired hearing.
  • As perfectly known to the skilled artisan, one ongoing problem in context with such hearing devices is the acoustical feedback AFB between the acoustical output of the output converter 16 and acoustical input E10 of the input converter 10. As principally known e.g. from the EP 0 656 737 , there is provided a feedback-compensator 18 whereat the prevailing acoustical feedback AFB, which is to be suppressed, is estimated e.g. with a correlation technique, correlating the signal applied to output converter 16 with a signal dependent on the output of input converter 10 as shown in dashed line at A. Thereby the gain G of feedback-compensator 18 is estimated so a to compensate for the AFB by negative feedback.
  • By means of feedback-compensator 18, a signal as predicted is fed back to the input of processor unit 14 downstream the output of beamformer unit 12 so as to compensate for the feedback AFB. As shown in Fig. 3, the feedback-compensator 18 has an input E18 which is operationally connected to the output A14 of the processing unit 14 and has an output A18 which is superimposed to the output E12 of beamformer unit 12, the result of such superimposing at a unit 20 being input to input E14 of processing unit 14.
  • Customarily, the feedback-compensator 18, which computes estimation of the acoustical feedback to be suppressed, has an adaptation rate in the range of several hundred ms and is thus considerably slower than the adaptation rate of beamforer unit 12. Thus without additional measures according to the present invention, whenever the beamformer unit 12 is controlled or caused to vary its beamforming characteristic B as schematically represented by a control at input C12A, the feedback-compensator 18 will not be able to accurately rapidly deal with the varied situation with respect to acoustical feedback AFB.
  • Therefore, there is provided a control of the adaptation rate of beamformer unit 12 which control is performed by the feedback-compensator 18, according to Fig. 3 at control input C12R. Whenever the feedback signal loop gain via feedback-compensator 18 rises, indicating the increase in acoustical feedback AFB to be suppressed, the adaptation rate or time constant of beamformer unit 12 is lowered to or below the adaptation rate of compensator unit 18.
  • The loop gain may at be least estimated e.g. by multiplying the linear gains along the loop, primarily consisting of the feedback-compensator 18 and the processing unit 14 in Fig. 3 or by adding these gains in dB.
  • Thereby, it is prevented that an adjustment of the beamformer unit 12 with respect to its beamforming characteristic B may not be dealt with by compensator unit 18.
  • Thus, in fact, adaptation rate control of beamformer unit 12 is performed in dependency of the loop gain along the feedback loop with feedback-compensator 18. The rate control input C12R to beamforming unit 12 is operationally connected to a loop gain output AG of feedback-compensator 18. With the embodiment according to the present invention as shown in Fig. 3, it becomes possible to slow down the adaptation rate of the beamformer unit 12 at least down to the adaptation rate of the feedback-compensator 18 in dependency of prevailing feedback thereof.
  • Thereby, combination of adaptive beamforming and feedback compensating becomes feasible.
  • As has already been mentioned, the direction with which acoustical feedback signals AFB to be suppressed impinge on the acoustical port of the input converter 10 is specific. Therefore, at the beamformer unit 12, there is generated a beam characteristic BAFB, as shown in Fig. 4, which has minimum amplification for these specific angles or, as shown e.g. for an in-the-ear hearing device, at two specific angles αAFB. Thus and in addition to compensation of AFB by feedback-compensator 18, beamforming is realised with minimum amplification for those spatial angles αAFB with which the acoustical feedback AFB to be suppressed impinges on the input converter 10.
  • Further, it has to be noticed that acoustical feedback AFB to be suppressed occurs substantially within a specific frequency band. This frequency band is dependent, among others, on the specific output converter 16 used, the type of device e.g. in-the-ear or outside-the-ear device. Therefore, in a further improved embodiment, overall feedback-suppression may be performed within that specific frequency band, thereby leaving beamforming in frequencies not within this specific frequency band unaffected and tailored according to needs different from acoustic feedback-suppression. According to Fig. 5 (a), beamforming BAFB for minimum amplification of acoustical feedback AFB to be suppressed, is performed frequency selectively for frequencies fAFB of the acoustical feedback signal AFB.
  • Beamforming for frequencies f AFB which are not significantly present in the acoustical feedback AFB is performed by a second beamforming B AFB which may be selected independently from BAFB, according to fig. 5(b).
  • In fact, two independent beam forms are superimposed each operating in respective, distinct frequency-bands. Frequency selective feedback-compensation and adaptation beamforming may easily be realised, if at least beamforming in unit 12 as well as feedback-compensation in feedback-compensator 18 are performed in frequency domain or, respectively, in sub-bands. Beamforming is then realised at the frequencies fAFB with minimum amplification at the specific angles αAFB, whereas beamforming at other frequencies f AFB is performed according to other needs. Consequently the adaptation rate of beamforming in unit 12 is only controlled by the gain of feedback-compensator 18 at the frequencies fAFB.
  • Thus, even when beamforming B AFB is switched off to minimum overall amplification, beamforming BAFB may be maintained active to suppress acoustical feedback also in such "quiet" mode. Thereby, and with an eye on processing in frequency domain, in each sub-band, which is significant for AFB, the loop gain, as estimated in feedback-compensator 18, may be compared with a threshold value and adaptation rate control at C12R is only established, if the instantaneous loop gain at least reaches such threshold. The control of the adaptation rate may then be lowered to practically zero, which means that beamforming is switched off for frequencies fAFB. This establishes a hard on/off-switching of beamforming in the fAFB frequency-range. In a further approach, such switching may be performed steadily which may be realised on the one hand by lowering the adaptation rate of BAFB steadily and/or by reducing beamforming amplification of BAFB steadily.
  • Due to the inventively improved suppression of acoustical feedback from the output of the output converter to the input of the input converter, there is reached additional stability of the device. The inter dependencies of vent tailoring at in-the-ear hearing devices and acoustical feedback problems is resolved to a significantly higher degree than was possible up to now when the device had the ability of adaptive beamforming.

Claims (14)

  1. A method for suppressing acoustical feedback between an acoustical output of an electrical/acoustical output converter arrangement and an acoustical input of an acoustical/electrical input converter arrangement of a hearing device, by
    • converting acoustical signals impinging on the acoustical/electrical input converter arrangement into a first electric signal by a controllably variable transfer characteristic which is dependent on the angle at which said acoustical signals impinge on said acoustical/electrical input converter arrangement; wherein
    • said first electric signal is processed and a resulting signal is applied to the electrical/acoustical output converter arrangement;
    • said acoustical feedback to be suppressed is compensated by an electrical feedback-compensating signal which is generated in dependency of the resulting signal and is fed back by a feedback signal path upstream said processing;
    • said electrical feedback-compensating signal is fed back to and superimposed upon the first electric signal and wherein further
    • the adaptation rate of said converting to variations of said transfer characteristic is controlled in dependency of the loop gain along said feedback signal path.
  2. The method of claim 1, further comprising slowing down the adaptation rate of said converting with increasing loop gain along said feedback signal path.
  3. The method of claims 1 or 2, further comprising minimising amplification of said transfer characteristic at one or more specific angles which accord to angles at which said acoustical feedback to be suppressed predominantly impinges on said input converter arrangement.
  4. The method of one of claims 1 to 3, further comprising frequency selectively controlling said adaptation rate.
  5. The method of one of claims 1 to 4, further comprising performing said converting in said first electric signal, and said processing along said feedback signal path in frequency domain and controlling said adaptation rate at selected frequencies in dependency of said loop gain at said selected frequencies.
  6. The method of one of claims 1 to 5, further comprising minimizing amplification of said transfer characteristic at specific angles frequency selectively.
  7. The method of one of claims 1 to 6, further comprising performing said converting into said first electric signal independently for frequencies present in said acoustical feedback to be suppressed and for frequencies substantially not present in said acoustical feedback to be suppressed.
  8. The method of one of claims 1 to 7, further comprising performing said control of said adaptation rate selectively for frequencies present in said acoustical feedback to be suppressed, said control comprising switching said converting on and off.
  9. The method of claim 8, further comprising performing switching from on to off and/or vice versa steadily during a predetermined timespan.
  10. The method of one of claims 1 to 9, said hearing device being a behind-the-ear or a in-the-ear hearing device.
  11. The method of one of claims 1 to 10, said hearing device being a ear protection or a hearing improvement device.
  12. A hearing device, comprising:
    • an acoustical/electrical input converter arrangement and an adaptive beamformer unit, generating at an output an electric output signal dependent on acoustical signals impinging on said acoustical/electrical input converter arrangement and in dependency of an angle at which said acoustical signals impinge, said beamformer unit having a first control input for varying beamforming characteristics
    • a processing unit with an input operationally connected to the output of said beamformer unit and with an output operationally connected to an input of an electrical/acoustical output converter arrangement
    • a feedback-compensator unit, the input thereof being operationally connected to said input of said electrical/acoustical output converter arrangement, an output thereof being operationally connected to the input of said processing unit, wherein
    • said output of said feedback-compensator unit is operationally superimposed with the output of said beamformer unit,
    and Wherein further
    • said beamformer unit has a second control input for adjusting its adaptation rate, and
    • said feedback-compensator unit has an output for a loop gain indicative signal, being operationally connected to said second control input of said beamformer unit.
  13. The device of claim 12 being a behind-the-ear hearing device or an in-the-ear hearing device.
  14. The device of one of claims 12 or 13, being a hearing protection device or a hearing improvement device.
EP04006073.3A 2004-03-15 2004-03-15 Feedback suppression Expired - Lifetime EP1469702B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DK04006073.3T DK1469702T3 (en) 2004-03-15 2004-03-15 Feedback suppression
EP04006073.3A EP1469702B1 (en) 2004-03-15 2004-03-15 Feedback suppression

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP04006073.3A EP1469702B1 (en) 2004-03-15 2004-03-15 Feedback suppression

Publications (3)

Publication Number Publication Date
EP1469702A2 EP1469702A2 (en) 2004-10-20
EP1469702A3 EP1469702A3 (en) 2004-11-17
EP1469702B1 true EP1469702B1 (en) 2016-11-23

Family

ID=32893043

Family Applications (1)

Application Number Title Priority Date Filing Date
EP04006073.3A Expired - Lifetime EP1469702B1 (en) 2004-03-15 2004-03-15 Feedback suppression

Country Status (2)

Country Link
EP (1) EP1469702B1 (en)
DK (1) DK1469702T3 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110139200A (en) * 2018-02-09 2019-08-16 奥迪康有限公司 Hearing devices including the Beam-former filter unit for reducing feedback

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DK1949755T3 (en) * 2005-10-11 2010-08-23 Widex As Hearing aid and method of processing input signals in a hearing aid
DE102008049086B4 (en) * 2008-09-26 2011-12-15 Siemens Medical Instruments Pte. Ltd. Hearing aid with a directional microphone system and method for operating such a hearing aid
DE102009060094B4 (en) * 2009-12-22 2013-03-14 Siemens Medical Instruments Pte. Ltd. Method and hearing aid for feedback detection and suppression with a directional microphone
EP2439958B1 (en) 2010-10-06 2013-06-05 Oticon A/S A method of determining parameters in an adaptive audio processing algorithm and an audio processing system
WO2011027005A2 (en) * 2010-12-20 2011-03-10 Phonak Ag Method and system for speech enhancement in a room
WO2013007309A1 (en) * 2011-07-14 2013-01-17 Phonak Ag Speech enhancement system and method
DK3249955T3 (en) 2016-05-23 2019-11-18 Oticon As CONFIGURABLE HEARING, INCLUDING A RADIATION FORM FILTER UNIT AND AMPLIFIER

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0585976A3 (en) * 1993-11-10 1994-06-01 Phonak Ag Hearing aid with cancellation of acoustic feedback
US6219427B1 (en) * 1997-11-18 2001-04-17 Gn Resound As Feedback cancellation improvements
EP1118247A2 (en) * 1998-09-30 2001-07-25 House Ear Institute Band-limited adaptive feedback canceller for hearing aids
CA2404863C (en) * 2000-03-31 2009-08-04 Phonak Ag Method for providing the transmission characteristics of a microphone arrangement and microphone arrangement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110139200A (en) * 2018-02-09 2019-08-16 奥迪康有限公司 Hearing devices including the Beam-former filter unit for reducing feedback
CN110139200B (en) * 2018-02-09 2022-05-31 奥迪康有限公司 Hearing device comprising a beamformer filtering unit for reducing feedback

Also Published As

Publication number Publication date
EP1469702A2 (en) 2004-10-20
DK1469702T3 (en) 2017-02-13
EP1469702A3 (en) 2004-11-17

Similar Documents

Publication Publication Date Title
EP2217007B1 (en) Hearing device with adaptive feedback suppression
US7492916B2 (en) Method for manufacturing acoustical devices and for reducing especially wind disturbances
EP1380187B1 (en) Directional controller and a method of controlling a hearing aid
EP2002690B2 (en) Hearing aid, and a method for control of adaptation rate in anti-feedback systems for hearing aids
EP1665882B1 (en) Hearing aid with acoustic feedback suppression
WO2007099420A1 (en) Adaptive control system for a hearing aid
EP2317778A2 (en) Hearing aid and method of utilizing gain limitation in a hearing aid
KR20120035216A (en) Method and processing unit for adaptive wind noise suppression in a hearing aid system and a hearing aid system
AU2002338610A1 (en) Directional controller and a method of controlling a hearing aid
US7688985B2 (en) Automatic microphone matching
US10362413B2 (en) Hearing device with suppression of sound impulses
CN109121055B (en) Hearing device with comb filter effect suppression
EP1469702B1 (en) Feedback suppression
WO2012099518A1 (en) Method and device for microphone selection
US10659890B2 (en) Method for operating a hearing device and a hearing device
US7324651B2 (en) Feedback suppression
EP2148526B1 (en) Spectral content modification for robust feedback channel estimation
EP1339256B1 (en) Method for manufacturing acoustical devices and for reducing wind disturbances
EP2890154B1 (en) Hearing aid with feedback suppression
CN109326301A (en) Self-adaptive post-filtering
EP1489883A2 (en) Automatic microphone matching
KR100363252B1 (en) Adaptive feedback cancellation apparatus and method for multi-band compression hearing aids

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

17P Request for examination filed

Effective date: 20041229

AKX Designation fees paid

Designated state(s): CH DE DK LI

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SONOVA AG

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160629

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): CH DE DK LI

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602004050347

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20170209

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602004050347

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20170824

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DK

Payment date: 20180326

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20180328

Year of fee payment: 15

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602004050347

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20190331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191001

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190331