EP2869600A1 - Adaptive Restrückkopplungsunterdrückung - Google Patents

Adaptive Restrückkopplungsunterdrückung Download PDF

Info

Publication number
EP2869600A1
EP2869600A1 EP20130191660 EP13191660A EP2869600A1 EP 2869600 A1 EP2869600 A1 EP 2869600A1 EP 20130191660 EP20130191660 EP 20130191660 EP 13191660 A EP13191660 A EP 13191660A EP 2869600 A1 EP2869600 A1 EP 2869600A1
Authority
EP
European Patent Office
Prior art keywords
feedback
signal
hearing aid
audio signal
suppression circuit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP20130191660
Other languages
English (en)
French (fr)
Other versions
EP2869600B1 (de
Inventor
Erik Cornelis Diederik Van Der Werf
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GN Hearing AS
Original Assignee
GN Resound AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GN Resound AS filed Critical GN Resound AS
Priority to DK13191660.3T priority Critical patent/DK2869600T3/en
Priority to EP13191660.3A priority patent/EP2869600B1/de
Priority to US14/074,152 priority patent/US9712908B2/en
Priority to JP2016528128A priority patent/JP6283413B2/ja
Priority to PCT/EP2014/073711 priority patent/WO2015067606A1/en
Priority to CN201480066150.0A priority patent/CN105794228B/zh
Publication of EP2869600A1 publication Critical patent/EP2869600A1/de
Application granted granted Critical
Publication of EP2869600B1 publication Critical patent/EP2869600B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/45—Prevention of acoustic reaction, i.e. acoustic oscillatory feedback
    • H04R25/453—Prevention of acoustic reaction, i.e. acoustic oscillatory feedback electronically
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/30—Monitoring or testing of hearing aids, e.g. functioning, settings, battery power
    • H04R25/305—Self-monitoring or self-testing

Definitions

  • a new method for performing adaptive feedback suppression in a hearing aid and a hearing aid utilizing the method are provided. According to the method, residual feedback is estimated and reduced. The estimate of residual feedback is based on features of an input signal of the hearing aid.
  • acoustical signals arriving at a microphone of the hearing aid are amplified and output with a small loudspeaker to restore audibility.
  • the small distance between the microphone and the loudspeaker may cause feedback.
  • Feedback is generated when a part of the amplified acoustic output signal propagates back to the microphone for repeated amplification.
  • the feedback loop becomes unstable, possibly leading to audible distortions or howling. To stop the feedback, the gain has to be turned down.
  • the risk of feedback limits the maximum gain that can be used with a hearing aid.
  • the feedback suppression circuit configured to model the feedback path of propagation along which an output signal of the hearing aid propagates back to an input of the hearing aid for repeated amplification.
  • the transfer function of the receiver in the art of hearing aids, the loudspeaker of the hearing aid is usually denoted the receiver
  • the transfer function of the microphone are included in the model of the feedback path of propagation.
  • the feedback suppression circuit adapts its transfer function to match the corresponding transfer function of the feedback path as closely as possible.
  • the digital feedback suppression circuit may include one or more digital adaptive filters to model the feedback path. An output of the feedback suppression circuit is subtracted from the audio signal of the microphone to remove the feedback signal part of the audio signal.
  • the hearing aid may comprise separate digital feedback suppression circuits for individual microphones and groups of microphones.
  • the feedback part of the audio signal is removed completely so that only an external signal generated in the surroundings of the hearing aid is amplified in the hearing aid.
  • the feedback suppression circuit cannot model the feedback path perfectly; leaving an undesired residual feedback signal for amplification. Near instability, the residual feedback signal may cause the hearing aid output level to exceed the desired output level.
  • EP 2 203 000 A1 discloses a hearing aid with suppression of residual feedback utilizing an adaptive feedback gain circuit wherein the level of residual feedback is estimated based on the hearing aid gain and a feedback path model as determined during power up or during fitting of the hearing aid.
  • a new method and a new hearing aid are provided in which residual feedback is suppressed based on another estimate of residual feedback.
  • residual feedback is reduced by gain adjustments based on an estimate of the residual feedback signal, wherein the estimate is based on an input signal of the hearing aid, such as a power spectrum of the input signal.
  • a new method of suppressing residual feedback comprising converting an acoustic signal into an audio signal, modelling a feedback path with a feedback suppression circuit receiving an input signal based on the audio signal, and generating an output signal, subtracting the output signal of the feedback suppression circuit from the audio signal to form a feedback compensated audio signal, determining an estimate of a residual feedback signal part of the feedback compensated audio signal based at least on the audio signal, and applying a gain to the feedback compensated audio signal based at least on the estimate.
  • the method may further comprise monitoring the feedback path, wherein the estimate of the residual feedback signal part is based on a result from the act of monitoring.
  • a new hearing aid comprising an input transducer for generating an audio signal, a feedback suppression circuit configured for modelling a feedback path of the hearing aid, a subtractor for subtracting an output signal of the feedback suppression circuit from the audio signal to form a feedback compensated audio signal, a signal processor that is connected to an output of the subtractor for processing the feedback compensated audio signal to perform hearing loss compensation, and a receiver that is connected to an output of the signal processor for converting the processed feedback compensated audio signal into a sound signal, the hearing aid further comprising:
  • a transducer is a device that converts a signal in one form of energy to a corresponding signal in another form of energy.
  • the input transducer may comprise a microphone that converts an acoustic signal arriving at the microphone into a corresponding analogue audio signal in which the instantaneous voltage of the audio signal varies continuously with the sound pressure of the acoustic signal.
  • the input transducer comprises a microphone.
  • the input transducer may also comprise a telecoil that converts a magnetic field at the telecoil into a corresponding analogue audio signal in which the instantaneous voltage of the audio signal varies continuously with the magnetic field strength at the telecoil.
  • Telecoils may be used to increase the signal to noise ratio of speech from a speaker addressing a number of people in a public place, e.g. in a church, an auditorium, a theatre, a cinema, etc., or through a public address systems, such as in a railway station, an airport, a shopping mall, etc.
  • Speech from the speaker is converted to a magnetic field with an induction loop system (also called "hearing loop"), and the telecoil is used to magnetically pick up the magnetically transmitted speech signal.
  • an induction loop system also called "hearing loop
  • the input transducer may further comprise at least two spaced apart microphones, and a beamformer configured for combining microphone output signals of the at least two spaced apart microphones into a directional microphone signal, e.g. as is well-known in the art.
  • the input transducer may comprise one or more microphones and a telecoil and a switch, e.g. for selection of an omni-directional microphone signal, or a directional microphone signal, or a telecoil signal, either alone or in any combination, as the audio signal.
  • a switch e.g. for selection of an omni-directional microphone signal, or a directional microphone signal, or a telecoil signal, either alone or in any combination, as the audio signal.
  • the output transducer preferably comprises a receiver, i.e. a small loudspeaker, which converts an analogue audio signal into a corresponding acoustic sound signal in which the instantaneous sound pressure varies continuously in accordance with the amplitude of the analogue audio signal.
  • a receiver i.e. a small loudspeaker
  • the analogue audio signal may be made suitable for digital signal processing by conversion into a corresponding digital audio signal in an analogue-to-digital converter whereby the amplitude of the analogue audio signal is represented by a binary number.
  • a discrete-time and discrete-amplitude digital audio signal in the form of a sequence of digital values represents the continuous-time and continuous-amplitude analogue audio signal.
  • a part of the output signal may propagate from the output transducer back to the input transducer both along an external signal path outside the hearing aid housing and along an internal signal path inside the hearing aid housing.
  • Acoustical feedback occurs, e.g., when a hearing aid ear mould does not completely fit the wearer's ear, or in the case of an ear mould comprising a canal or opening for e.g. ventilation purposes. In both examples, sound may "leak" from the receiver back to the microphone and thereby cause feedback.
  • Mechanical feedback may be caused by mechanical vibrations in the hearing aid housing and in components inside the hearing aid housing. Mechanical vibrations may be generated by the receiver and are transmitted to other parts of the hearing aid, e.g. through receiver mounting(s). In some hearing aids, the receiver is flexibly mounted in the housing, whereby transmission of vibrations from the receiver to other parts of the hearing aid is reduced.
  • Internal feedback may also be caused by propagation of an electromagnetic field generated by coils in the receiver to the telecoil.
  • the feedback signal part of the audio signal a part of the audio signal generated by the hearing aid itself, e.g., in response to sound, mechanical vibration, and electromagnetic fields is termed the feedback signal part of the audio signal; or in short, the feedback signal.
  • a difference between the feedback signal part of the audio signal and the output signal of the feedback suppression circuit is termed the residual feedback signal part of the audio signal; or in short, the residual feedback signal.
  • An external feedback path extends "around" the hearing aid and is therefore usually longer than an internal feedback path, i.e. sound has to propagate a longer distance along the external feedback path than along the internal feedback path to get from the receiver to the microphone. Accordingly, when sound is emitted from the receiver, the part of it propagating along the external feedback path will arrive at the microphone with a delay in comparison to the part propagating along the internal feedback path. Therefore, separate digital feedback suppression circuits may operate on first and second time windows, respectively, wherein at least a part of the first time window precedes the second time window. Whether the first and second time windows overlap or not, depends on the length of the impulse response of the internal feedback path.
  • Open solutions may lead to feedback paths with long impulse responses, since the receiver output is not separated from the microphone input by a tight seal in the ear canal.
  • a hearing aid with a housing that does not obstruct the ear canal when the housing is positioned in its intended operational position in the ear canal; is categorized "an open solution”.
  • the term "open solution” is used because of a passageway is formed between a part of the ear canal wall and a part of the housing allowing sound waves to escape from behind the housing between the ear drum and the housing through the passageway to the surroundings of the user. With an open solution, the occlusion effect is diminished and preferably substantially eliminated.
  • a standard sized hearing aid housing which fits a large number of users with a high level of comfort may represent an open solution.
  • the risk of feedback limits the maximum gain that can be achieved with a hearing aid.
  • a feedback suppression circuit is provided in the hearing aid, configured for modelling the feedback path, i.e. desirably the feedback suppression circuit has the same transfer function as the feedback path itself so that an output signal of the feedback suppression circuit matches the feedback signal part of the audio signal as closely as possible.
  • a subtractor is provided for subtraction of the output signal of the feedback suppression circuit from the audio signal to form a feedback compensated audio signal in which the feedback signal has been removed or at least reduced.
  • the feedback suppression circuit may comprise an adaptive filter that tracks the current transfer function of the feedback path.
  • limitations in the tracking performance of the feedback suppression circuit may leave a residual feedback signal part in the audio signal formed by a difference between the estimated feedback signal and the actual feedback signal.
  • a gain processor for improved feedback suppression.
  • the gain processor is configured for compensating for the residual feedback signal by applying a gain to the feedback compensated audio signal based on an improved estimate of the residual feedback signal based at least on the audio signal, e.g. a power spectrum of the audio signal.
  • the gain processor desirably applies a gain to the feedback compensated audio signal so that the resulting loudness of the output signal of the hearing aid substantially equals the loudness that would have been obtained with no residual feedback signal.
  • the estimate of the residual feedback signal part of the audio signal on the input signal may include an analysis of the input spectrum of the audio signal for detection of high risk of feedback, or feedback, e.g. in the event that the feedback suppression circuit provides insufficient information to prevent feedback.
  • the feedback suppression circuit may be configured during an initialization of the hearing aid, and the estimate of the residual feedback signal may further be based on a configuration of the feedback suppression circuit achieved during the initialization of the hearing aid.
  • Initialization may be performed during turn-on of the hearing aid and/or during fitting as disclosed in EP 2 203 000 A1 .
  • the feedback suppression circuit may have a configuration that is variable, and the estimate of the residual feedback signal may further be based on a configuration of the feedback suppression circuit as determined during a current operation of the hearing aid.
  • the estimate of the residual feedback signal may thus be based on an updated feedback suppression circuit as determined during current operation of the hearing aid modelling the feedback path, e.g. following slow variations of the feedback path as for example resulting from a re-insertion of the hearing aid in the ear canal of the user, build-up of ear wax, aging of electronic components, etc.
  • the estimate of the residual feedback signal may further be based on a gain value of the hearing aid.
  • the feedback suppression circuit may comprise one or more adaptive filters.
  • the estimate of the residual feedback signal may be based on filter coefficients of the one or more adaptive filters.
  • the gain adjustment may be performed separate from hearing loss compensation, preferably before bearing loss compensation.
  • the estimate of the residual feedback signal may include an estimate of an adaptive broad-band contribution ⁇ .
  • the signal processor may be configured to perform multi-band hearing loss compensation in a set of frequency bands k, wherein the estimate of the residual feedback signal comprises individual estimates of the residual feedback signal in respective frequency bands k.
  • h emp may be equal to one.
  • the hearing aid may further comprise attack and release filters configured for smoothing process parameters in the gain processor.
  • the estimate of the residual feedback signal part of the audio signal, based on the input signal may include an analysis of the input spectrum of the audio signal for detection of feedback, e.g. in the event that the feedback suppression circuit provides insufficient information to prevent feedback.
  • Monitoring the feedback suppression circuit improves the estimate of the residual feedback signal part of the audio signal, especially upon detection of a significant change of the feedback suppression circuit modelling the feedback path, such as bringing a phone to the ear with the hearing aid. Such a feedback path change may cause a significant increase of the magnitude of the residual feedback signal until the feedback suppression circuit has had time to adjust to the change. Such an increase may be adequately estimated due to the monitoring.
  • the hearing aid may be a multi-band hearing aid performing hearing loss compensation differently in different frequency bands, thus accounting for the frequency dependence of the hearing loss of the intended user.
  • the audio signal from the input transducer is divided into two or more frequency channels or bands; and the audio signal may be amplified differently in each frequency band.
  • a compressor may be utilized to compress the dynamic range of the audio signal in accordance with the hearing loss of the intended user.
  • the compressor performs compression differently in each of the frequency bands varying not only the compression ratio, but also the time constants associated with each band.
  • the time constants refer to compressor attack and release time constants.
  • the compressor attack time is the time required for the compressor to lower the gain at the onset of a loud sound.
  • the release time is the time required for the compressor to increase the gain after the cessation of the loud sound.
  • the feedback suppression circuit e.g. including one or more adaptive filters, may be a broad band circuit, i.e. the circuit may operate substantially in the entire frequency range of the hearing aid, or in a significant part of the frequency range of the hearing aid, without being divided into a set of frequency bands.
  • the feedback suppression circuit may be divided into a set of frequency bands for individual modelling of the feedback path in each frequency band.
  • the estimate of the residual feedback signal may be provided individually in each frequency band m of the feedback suppression circuit.
  • the frequency bands m of the feedback suppression circuit and the frequency bands k of the hearing loss compensation may be identical, but preferably, they are different, and preferably the number of frequency bands m of the feedback suppression circuit is less than the number of frequency bands of the hearing loss compensation.
  • audio signal is used to identify any analogue or digital signal forming part of the signal path from an output of the microphone to an input of the processor.
  • the feedback suppression circuit may be implemented as a dedicated electronic hardware circuit or may form part of a signal processor in combination with suitable signal processing software, or may be a combination of dedicated hardware and one or more signal processors with suitable signal processing software.
  • Signal processing in the new hearing aid may be performed by dedicated hardware or may be performed in a signal processor, or performed in a combination of dedicated hardware and one or more signal processors.
  • processor As used herein, the terms "processor”, “signal processor”, “controller”, “system”, etc., are intended to refer to CPU-related entities, either hardware, a combination of hardware and software, software, or software in execution.
  • a "processor”, “signal processor”, “controller”, “system”, etc. may be, but is not limited to being, a process running on a processor, a processor, an object, an executable file, a thread of execution, and/or a program.
  • processor designate both an application running on a processor and a hardware processor.
  • processors may reside within a process and/or thread of execution, and one or more "processors”, “signal processors”, “controllers”, “systems”, etc., or any combination hereof, may be localized on one hardware processor, possibly in combination with other hardware circuitry, and/or distributed between two or more hardware processors, possibly in combination with other hardware circuitry.
  • a processor may be any component or any combination of components that is capable of performing signal processing.
  • the signal processor may be an ASIC processor, a FPGA processor, a general purpose processor, a microprocessor, a circuit component, or an integrated circuit.
  • Fig. 1 schematically illustrates a hearing aid 10 and a feedback path 12 along which signals generated by the hearing aid 10 propagates back to an input of the hearing aid 10.
  • an acoustical signal 14 is received at a microphone 16 that converts the acoustical signal 14 into an audio signal 18 that is input to the signal processor 20 for hearing loss compensation.
  • the audio signal 18 is amplified in accordance with the hearing loss of the user.
  • the signal processor 20 may for example comprise a multi-band compressor.
  • the output signal 22 of the signal processor 20 is converted into an acoustical output signal 24 by the receiver 26 that directs the acoustical signal towards the eardrum of the user when the hearing aid is worn in its proper operational position at an ear of the user.
  • a part of the acoustical signal 24 from the receiver 26 propagates back to the microphone 16 as indicated by feedback path 12 in Fig. 1 .
  • the feedback signal level at the microphone 16 may exceed the level of the original acoustical signal thereby causing audible distortion and possibly howling.
  • Fig. 2 schematically illustrates a hearing aid 10 with a feedback suppression circuit 28.
  • the feedback suppression circuit 28 models the feedback path 12, i.e. the feedback suppression circuit 28 seeks to generate a signal that is identical to the signal having propagated along the feedback path 12, i.e. the feedback suppression circuit 28 adapts its transfer function to match the corresponding transfer function of the feedback path as closely as possible. It is noted that the feedback suppression circuit 28 includes models of the receiver 26 and the microphone 16.
  • the feedback suppression circuit 28 may be an adaptive digital filter which adapts to changes in the feedback path 12.
  • the feedback suppression circuit 28 generates an output signal 30 to the subtractor 32 in order to suppress or cancel the feedback signal part of the audio signal 18 before processing takes place in the signal processor 20.
  • the feedback suppression circuit 28 does not model the feedback path 12 accurately, a fraction of the feedback signal, the residual feedback signal, remains in the feedback compensated audio signal 34.
  • Fig. 3 schematically illustrates a linear model of signal processing and signals in a hearing aid.
  • the feedback suppression circuit 28 models the transfer functions of the real feedback path 12, including the receiver (not shown), microphone (not shown), and possible other analogue components (not shown).
  • the feedback suppression circuit 28 is configured to output a signal c 30 to be subtracted from the audio signal x 18 thereby eliminating, or at least substantially reducing, the feedback signal f.
  • the feedback suppression circuit 28 cannot exactly model the real feedback path 12, whereby a residual feedback signal part remains in the feedback compensated audio signal e 34.
  • lower case characters will be used for time domain signals and functions, while upper case characters will be used for their z-transforms.
  • the effective gain provided by the hearing aid approximates G, G being the gain of the hearing aid, when
  • G being the gain of the hearing aid
  • ⁇ 1 i.e. when the residual feedback signal level is very small.
  • the GR term cannot be neglected, and
  • Fig. 4 schematically illustrates an exemplary new hearing aid 10 with a gain processor 38 that is configured for applying a gain ⁇ to the feedback compensated audio signal 34 so that the effect on the residual feedback signal is reduced.
  • the signals x , r , and f are not present individually in the hearing aid circuitry, while the signals e , c , y , and z are present individually in the hearing aid circuitry.
  • a worst case value for the feedback compensated signal e could be obtained by summing amplitude values of signals x and r , however it is presently preferred to use equation (4).
  • Fig. 5 schematically illustrates an exemplary new hearing aid with a gain processor 38.
  • the hearing aid 10 illustrated in Fig. 5 corresponds to the known hearing aid illustrated in Fig. 5 of EP 2 203 000 A1 ; however the new hearing aid provides an improved estimate of the residual feedback signal R as explained below in more detail.
  • the hearing aid 10 of Fig. 5 has a compressor that performs dynamic range compression using digital frequency warping of the kind disclosed in more detail in WO 03/015468 , in particular the basic operating principles of the warped compressor are illustrated in Fig. 10 and the corresponding parts of the description of WO 03/015468 .
  • the hearing aid 10 illustrated in Fig. 5 corresponds to the hearing aid of Fig. 10 of WO 03/015468 ; however feedback suppression and gain processing and noise reduction have been added in the signal processing of the hearing aid 10. Other processing circuitry may be added as well.
  • the gain processor 38 may be employed with non-warped frequency bands.
  • the hearing aid schematically illustrated in Fig. 5 has a single microphone 16.
  • the hearing aid 10 may comprise two or more microphones, possibly with a beamformer. These components are not shown for simplicity.
  • possible A/D and D/A converters, buffer structures, optional additional channels, etc, are not shown for simplicity.
  • An incoming acoustical signal received by the microphone 16 is passed through a DC filter 42 which ensures that the signals have a mean value of zero; this is convenient for calculating the statistics as discussed previously.
  • the signal received by the microphone 16 may be passed directly to the subtractor 32.
  • feedback suppression may be applied by subtracting an estimated feedback signal c from the audio signal s.
  • the feedback signal estimate 30 is provided by the feedback suppression circuit 28.
  • the feedback suppression circuit 28 comprises a series connection of a delay 44, a slow adaptive or fixed filter 46, and a fast adaptive filter 48 operating on the output signal z of the hearing aid 10.
  • a fixed or slow adaptive filter 46 may be an all-pole or general infinite impulse response (IIR) filter initialized at a certain point in time, for example upon turn on in the ear of the hearing aid, or, during fitting, while a slow adaptive filter 46 and the fast adaptive filter 48 are preferably finite impulse response (FIR) filters, but in principle any other adaptive filter structure (lattice, adaptive IIR, etc.) may be used.
  • IIR infinite impulse response
  • FIR finite impulse response
  • the fast adaptive filter 48 is an all zero filter.
  • the feedback suppression circuit 28 is a broad-band system, i.e. the feedback suppression circuit 28 operates in the entire frequency range of the multi-band hearing aid 10.
  • the input signal 22 to the feedback suppression circuit 28 may also be divided into a number of frequency bands m for individual feedback suppression in each frequency band m of the feedback suppression circuit 28.
  • the frequency bands k of the audio signal and the frequency bands m of the feedback suppression circuit 28 may be identical, but they may be different, and preferably, the feedback suppression circuit 28 has a fewer number of frequency bands m than the frequency divided audio signal.
  • the output signal 30 of the feedback suppression circuit 28 is subtracted from the audio signal 18 and transformed to the frequency domain.
  • the hearing aid 10 illustrated in Fig. 5 has a side-branch structure 52 where the analysis of the signal is performed outside a main signal path 50; and signal shaping is performed using a time domain-filter constructed from outputs of the side-branch 52.
  • a warped side-branch system 52 has advantages for high quality low-delay signal processing, but in principle any textbook FFT-system, a multi-rate filter bank, or a non-warped side-branch system may be used. Thus, although it is convenient to use frequency warping, it is not at all necessary in order to exercise the new method of estimating the residual feedback signal.
  • a warped FIR filter 50 is provided for generation of warped frequency bands.
  • the warped FIR filter 50 is obtained by substitution of the unit delays of a tapped delay line of a FIR filter with all pass filters as is well-known in the art and e.g. as explained in WO 03/015468 .
  • a power estimate is formed in each warped frequency band with an FFT operation 51.
  • a side branch 52 is formed having a chain of so-called gain agents 38, 54, 56 that analyze the respective power estimates and adjust gains applied individually to the respective signals in each of the warped frequency bands in a specific order.
  • the order of the gain agents is: gain processor 38, noise reduction 54, and loudness restoration 56.
  • the order of the gain agents 38, 54, 56 may be different.
  • the first gain agent i.e. the gain processor 38
  • the gain processor 38 receives input from the feedback suppression circuit 28, and finally, the gain vector in the frequency domain output by loudness restoration processor 56 as calculated in the previous iteration (representing the current gains as applied by the warped FIR filter 50) is also input to the gain processor 38.
  • the second gain agent 54 shown here providing noise reduction, is optional. Noise reduction is a comfort feature which is often used in modern hearing aids. Together, the first two gain agents 38, 54 seek to shape the audio signal in such a way that the envelope of the original signal is restored without undesired noise or feedback.
  • the third gain agent 56 adjusts loudness in order to compensate for the hearing loss of the intended user. A significant difference should be noted between restoring the loudness to loudness of the original signal without feedback performed by the gain processor 38, and restoring normal loudness perception for the hearing impaired listener performed by the loudness restoration processor 56 and including dynamic range compression in accordance with the hearing loss of the intended user of the hearing aid 10.
  • the agents 38, 54 and 56 in the gain-chain may be re-ordered, e.g., the gain processor 38 may be moved to the end of the chain.
  • the illustrated order so that the signal envelope is corrected before hearing loss dependent adjustments are performed, which may be non-linear and sound pressure dependent.
  • the output gain vector 58 in the frequency domain is transformed back to the time domain using an Inverse Fast Fourier Transform (IFFT) 60 and used as the coefficient vector of the warped FIR filter.
  • IFFT Inverse Fast Fourier Transform
  • the gain vector 58 is also propagated back to the gain processor 38 to be used in the next gain determination.
  • the signal that has passed through the warped FIR filter 50 is output limited in an output limiter 62 to ensure that (possibly unknown) receiver 16 and/or microphone 16 non-linearities do not propagate along the feedback path. Otherwise the feedback suppression circuit 28 may fail to model large signal levels adequately.
  • the output limiter 62 may be omitted.
  • output limiting may be provided by the dynamic range compressor or by other parts of the digital signal processing circuitry.
  • the residual feedback signal is estimated by the gain processor 38 in a way different from the estimation scheme disclosed in EP 2 203 000 A1 .
  • a k is the feedback reference gain obtained from the feedback suppression circuit
  • B k is a potential band offset ⁇ 1 obtained from monitoring the input power spectrum
  • the fractional residual error ⁇ is a scaling term which relates the residual feedback signal to the feedback reference level.
  • ⁇ and A k relate to the feedback suppression circuit 28 and they provide a proactive good estimate of the residual feedback signal so that residual feedback compensating gains are applied to the feedback compensated audio signal before instability occurs.
  • the feedback suppression circuit 28 may adapt too slowly leading to significant residual feedback and possible instability.
  • the band offsets B k relating to the audio signal provide a significant contribution to the estimate of residual feedback so that feedback compensating gains are applied to overcome emerging instability.
  • Feedback reference gains A k are obtained from the transfer function of the feedback suppression circuit 28. In EP 2 203 000 A1 , this was performed only at initialization, i.e. during fitting and/or at hearing aid turn on. The same method of obtaining the feedback reference gains A k may be used here.
  • the feedback reference gains A k are updated at regular time intervals during operation, e.g. following slow changes of the feedback suppression circuit 28, e.g. resulting from repeated insertion of the hearing aid in the ear canal of the user.
  • the transfer function of the feedback suppression circuit 28 is calculated for the warped frequency bands k, i.e. a Fourier transform is performed for the frequencies in question.
  • a k is the value calculated at the centre frequency of the band in question, while for high frequency bands, the resolution is doubled by also calculating the Fourier transform at the border frequencies.
  • the transfer function is calculated for a number of bins, e.g. 22 bins, and the value A k is determined for each warped frequency band k by setting A k to the maximum value of the three nearest frequency bins, whereby the risk of underestimation is suppressed.
  • sudden changes are reduced by applying a first order low pass filter (not shown) to the transformed magnitudes in the log domain.
  • the Fourier transform may not be performed for all frequencies for each block of samples, e.g. the Fourier transform may be performed for one frequency only for each block of samples.
  • ⁇ is calculated for every block of samples and is used for all frequency bands k as a scaling factor determining the magnitude of the residual feedback signal
  • ⁇ was the only adaptive mechanism while the reference gains A k were fixed between determinations at fitting or at hearing aid turn on.
  • ⁇ takes care of fast changes in the feedback path, while changes of longer duration will eventually be absorbed in the adaptive feedback reference gains A k .
  • ⁇ is calculated from two orthogonal contributions, namely a static contribution representing an accuracy of the feedback suppression circuit under ideal conditions, e.g. due to limited precision; and a dynamic contribution representing inaccuracy due to changes in the feedback path which the feedback suppression circuit cannot track accurately.
  • w is the weight coefficient vector of the fast adaptive filter of the feedback suppression circuit
  • h e is an optional frequency emphasis filter
  • * denotes convolution
  • c s is a constant related to the expected static performance.
  • w ref is the reference weight coefficient vector of the fast adaptive filter of the feedback suppression circuit.
  • the response of the feedback suppression circuit equals the response of the fixed or slowly adaptive filter.
  • ⁇ d c d ⁇ ⁇ h e ⁇ * w ⁇ - w ref ⁇ ⁇ 1
  • c d is a constant related to the expected dynamic performance
  • ⁇ c s ⁇ h emp ⁇ * w ⁇ ⁇ 2 + c d ⁇ h emp ⁇ * w ⁇ - w ref ⁇ ⁇ 2 ⁇ norm
  • ⁇ may be further simplified by elimination of the frequency emphasis, i.e. h emp is set equal to the 1.
  • c s and c d may be determined empirically, e.g. based on system performance, such as tracking accuracy in various situations.
  • the static part of the fractional residual error is determined by c s , the other part accounts for the adapting feedback reference gains A k .
  • c s and c d may range from 0.1 to 0.4, depending on a tradeoff between speed and accuracy of the feedback suppression circuit and assuming that the feedback reference gains A k are scaled to match the feedback level. For example, in a slow adapting system c s may be set to a small value due to expected better static performance while c d is set to a larger value larger due to larger expected deviations when a change occurs.
  • the feedback suppression circuit may be unable to adapt sufficiently to avoid feedback in response to changes in the feedback path.
  • underestimates the residual feedback signal, and this may lead to instability.
  • instability may be clearly audible and may be detected in the input power spectrum. Therefore, the new method includes provision of offsets B k in equation (9) in order to restore stability. Frequency bands k with persistent peaks are detected and corresponding offsets B k to the residual feedback signal estimate R k are provided in order to suppress the feedback signal.
  • all frequency bands are classified as either a peak, valley, or slope for each block of samples.
  • a peak is a frequency band where the input power in neighboring bands is lower than the input power of the frequency band in question.
  • a valley is a frequency band where the input power in neighboring bands is larger than the input power of the frequency band in question.
  • a frequency band is not a peak or a valley, it is a slope, which is ignored.
  • the band offset B k is incremented or decremented, respectively, in dB. Values are confined between 0 dB and a maximum value.
  • the ratio between increment and decrement step sizes is determined by a peak probability threshold, whereby the peak probability threshold determines an upper limit on how often feedback peaks are allowed to occur in the input power spectrum, since by increasing band offset B k the probability of observing more peaks in band k will be reduced when the peak is caused by feedback.
  • this probability threshold is only used implicitly to determine the magnitude ratio between increments (for peaks) and decrements (for valleys). E.g., if a decrement is twice the size of an increment, gain reduction does not occur until at least twice as many peaks than valleys occur.
  • Step sizes, peak probability thresholds and maximum offset values can all be changed adaptively to make the algorithm more aggressive depending on the situation.
  • the probability of detecting a peak is equal to the probability of detecting a valley. Since slopes are ignored the expected peak probability is 50%.
  • the valid range of possible values for the peak probability threshold is therefore somewhere between 50% and 100%.
  • the decrements are always greater than the increments, so for average signals the band offsets remain close to the lower bound of 0 dB. When audible feedback occurs and dominates a specific band, the band offsets will increase until either the observed peak probability is reduced to the peak probability threshold, or the max band offset is reached.
  • Detection of peaks and valleys is sensitive to systematic offsets in the input power spectrum, which may, e.g., be caused by inaccuracies in the input calibration, unexpected peaks in transducer responses, specifically shaped background noises, uneven bandwidths caused by the frequency warping, etc.
  • the input spectrum therefore has to be normalized adaptively.
  • the normalization values are updated using a conditional attack and release filter that attempts to identify the non-tonal ambient noise level.
  • the normalization slowly leaks to a flat response.
  • the risk of feedback instability can be determined from various features available in the system., for example: (1) the feedback level, determined by combining the forward path gain with the feedback path gain (to roughly determine the distance to the maximum stable gain value), (2) the distance to the reference, which accounts for all changes since the device was first fitted, and (3) the tonal signal power, which represents how predictable the input signal is (externally generated pure tones & feedback squealing are both highly predictable yet difficult to discriminate).
  • the three features are combined into one value in a range between 0 and 1 denoted Peak Suppression Aggressiveness (PSA).
  • PSA Peak Suppression Aggressiveness
  • a high peak probability threshold is combined with small step sizes.
  • a lower peak probability threshold is combined with larger step sizes. Between 0 and 1, a weighted combination is used.
  • the output level does not go to infinity (as one expects for the theoretical linear system). Instead it converges to a steady state level determined by the (non-linear) compression and limiting of the Adaptive Gain Controls (AGC's). Since for this steady state level the total loop gain is unity (i.e.,
  • 1) an upper bound on the residual feedback gain can be inferred by monitoring the lowest observed gain in the forward path. Using this bound to restrict the maximum band offset, taking care to distinguish between PPS' own contribution and that of other gain agents, ensures that PPS cannot react excessively to tonal input.
  • ⁇ g k is updated recursively based on the actual hearing aid gains provided at the output of the gain-chain, i.e. the output of loudness restoration processor 56, which includes the contribution of all gain agents, previous gains, and the feedback reference gains.
  • the attack and release filters are applied in two stages.
  • a feedback suppression circuit 28 broadband scaling factor ⁇ is smoothed with configurable attack and release rates.
  • an instantaneous attack is combined with a slow fixed-step release.
  • Fig. 6 is a flowchart of the new method 100 of suppressing residual feedback, comprising the steps of:
  • Figs. 7 and 8 show plots 200, 300, respectively, of various feedback paths related transfer functions for performance comparison. The simulation is performed with Matlab.
  • the plot 200 of Fig. 7 shows feedback related transfer functions for a hearing aid as disclosed in EP 2 203 000 A1 with a fixed filter 46.
  • the plot 300 of Fig. 8 shows feedback related transfer functions for the hearing aid illustrated in Fig. 5 with a slow adaptive filter 46.
  • the lower dashed curves 210, 310 show the feedback path transfer functions with the hearing aids in their intended operating positions at the ear of the user, while the solid curves 220, 320 show the respective feedback path transfer functions when a telephone has been brought to the ear. A significant increase in the magnitudes of the transfer functions is noted.
  • the solid curves 230, 330 show the transfer functions of the feedback suppression circuit with the phone at the ear
  • solid curves 240, 340 show the residual feedback path transfer functions with the phone at the ear.
  • the dashed curves with squares 250, 350 show the estimated residual feedback path transfer functions with the phone at the ear.
  • the estimate 350 of the new hearing aid is significantly improved over the prior art.
EP13191660.3A 2013-11-05 2013-11-05 Adaptive Restrückkopplungsunterdrückung Active EP2869600B1 (de)

Priority Applications (6)

Application Number Priority Date Filing Date Title
DK13191660.3T DK2869600T3 (en) 2013-11-05 2013-11-05 Adaptive suppression of residual feedback
EP13191660.3A EP2869600B1 (de) 2013-11-05 2013-11-05 Adaptive Restrückkopplungsunterdrückung
US14/074,152 US9712908B2 (en) 2013-11-05 2013-11-07 Adaptive residual feedback suppression
JP2016528128A JP6283413B2 (ja) 2013-11-05 2014-11-04 適応型残留フィードバック抑制
PCT/EP2014/073711 WO2015067606A1 (en) 2013-11-05 2014-11-04 Adaptive residual feedback suppression
CN201480066150.0A CN105794228B (zh) 2013-11-05 2014-11-04 自适应残余反馈抑制

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP13191660.3A EP2869600B1 (de) 2013-11-05 2013-11-05 Adaptive Restrückkopplungsunterdrückung

Publications (2)

Publication Number Publication Date
EP2869600A1 true EP2869600A1 (de) 2015-05-06
EP2869600B1 EP2869600B1 (de) 2016-12-28

Family

ID=49515296

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13191660.3A Active EP2869600B1 (de) 2013-11-05 2013-11-05 Adaptive Restrückkopplungsunterdrückung

Country Status (2)

Country Link
EP (1) EP2869600B1 (de)
DK (1) DK2869600T3 (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106911992A (zh) * 2015-12-22 2017-06-30 奥迪康有限公司 包括反馈检测器的听力装置
EP3448064A1 (de) * 2017-08-25 2019-02-27 Oticon A/s Hörgerät mit selbstprüfungseinheit zur bestimmung des status einer oder mehrerer eigenschaften des hörgeräts basierend auf der rückkopplungsreaktion
EP3576433A1 (de) * 2018-05-30 2019-12-04 Sivantos Pte. Ltd. Verfahren zur verringerung eines auftretens einer akustischen rückkopplung in einem hörgerät

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5259033A (en) * 1989-08-30 1993-11-02 Gn Danavox As Hearing aid having compensation for acoustic feedback
WO2003015468A1 (en) 2001-08-08 2003-02-20 Gn Resound A/S Dynamic range compression using digital frequency warping
WO2006063624A1 (en) * 2004-12-16 2006-06-22 Widex A/S Hearing aid with feedback model gain estimation
EP2136575A2 (de) * 2008-06-20 2009-12-23 Starkey Laboratories, Inc. System zum Messen der maximalen stabilen Verstärkung in Hörgeräten
EP2203000A1 (de) 2008-12-23 2010-06-30 GN Resound A/S Adaptive Verstärkungsrückkopplungskorrektur
EP2217007A1 (de) * 2009-02-06 2010-08-11 Oticon A/S Hörgerät mit adaptiver Rückkopplungsunterdrückung

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5259033A (en) * 1989-08-30 1993-11-02 Gn Danavox As Hearing aid having compensation for acoustic feedback
WO2003015468A1 (en) 2001-08-08 2003-02-20 Gn Resound A/S Dynamic range compression using digital frequency warping
WO2006063624A1 (en) * 2004-12-16 2006-06-22 Widex A/S Hearing aid with feedback model gain estimation
EP2136575A2 (de) * 2008-06-20 2009-12-23 Starkey Laboratories, Inc. System zum Messen der maximalen stabilen Verstärkung in Hörgeräten
EP2203000A1 (de) 2008-12-23 2010-06-30 GN Resound A/S Adaptive Verstärkungsrückkopplungskorrektur
EP2217007A1 (de) * 2009-02-06 2010-08-11 Oticon A/S Hörgerät mit adaptiver Rückkopplungsunterdrückung

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106911992A (zh) * 2015-12-22 2017-06-30 奥迪康有限公司 包括反馈检测器的听力装置
EP3448064A1 (de) * 2017-08-25 2019-02-27 Oticon A/s Hörgerät mit selbstprüfungseinheit zur bestimmung des status einer oder mehrerer eigenschaften des hörgeräts basierend auf der rückkopplungsreaktion
US10687151B2 (en) 2017-08-25 2020-06-16 Oticon A/S Hearing aid device including a self-checking unit for determine status of one or more features of the hearing aid device based on feedback response
EP3576433A1 (de) * 2018-05-30 2019-12-04 Sivantos Pte. Ltd. Verfahren zur verringerung eines auftretens einer akustischen rückkopplung in einem hörgerät
CN110557708A (zh) * 2018-05-30 2019-12-10 西万拓私人有限公司 用于减少助听器中出现声反馈的方法
US10873817B2 (en) 2018-05-30 2020-12-22 Sivantos Pte. Ltd. Method for reducing the occurrence of acoustic feedback in a hearing device and hearing device

Also Published As

Publication number Publication date
DK2869600T3 (en) 2017-02-06
EP2869600B1 (de) 2016-12-28

Similar Documents

Publication Publication Date Title
EP2203000B1 (de) Adaptive Verstärkungsrückkopplungskorrektur
US9712908B2 (en) Adaptive residual feedback suppression
EP3563562B1 (de) Unterdrückung von akustischem echo
EP1068773B1 (de) Vorrichtung und verfahren zur kombinierung von audiokompression und rückkopplungsunterdrückung in einem hörgerät
US8737655B2 (en) System for measuring maximum stable gain in hearing assistance devices
JP5143121B2 (ja) 補聴器，および補聴器内における動的利得限界推定方法
US7974428B2 (en) Hearing aid with acoustic feedback suppression
US8019104B2 (en) Hearing aid with feedback model gain estimation
JP2014232331A (ja) アダプティブ・インテリジェント・ノイズ抑制システム及び方法
EP2234105A1 (de) Hintergrundgeräuschschätzung
WO2008104446A2 (en) Method for reducing noise in an input signal of a hearing device as well as a hearing device
EP1992196A2 (de) Hörgerät und verfahren zur kompensation von direktschall in hörgeräten
US9628923B2 (en) Feedback suppression
KR101253708B1 (ko) 보청장치의 외부 소음을 차폐하는 방법
EP2869600B1 (de) Adaptive Restrückkopplungsunterdrückung
EP2890154B1 (de) Hörgerät mit Rückkopplungsunterdrückung
EP3641337A1 (de) Signalverarbeitungsvorrichtung, telekonferenzvorrichtung und signalverarbeitungsverfahren
JP5606731B6 (ja) 適応型帰還利得補正
Pandey et al. Adaptive gain processing to improve feedback cancellation in digital hearing aids
DK1068773T4 (en) Apparatus and method for combining audio compression and feedback suppression in a hearing aid
JP2007060427A (ja) 騒音抑圧装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131105

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

R17P Request for examination filed (corrected)

Effective date: 20151104

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

17Q First examination report despatched

Effective date: 20160128

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160708

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 858194

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20170201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013015823

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170329

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170328

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20161228

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 858194

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170428

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170328

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170428

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013015823

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

26N No opposition filed

Effective date: 20170929

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171105

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20131105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161228

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20221117

Year of fee payment: 10

Ref country code: FR

Payment date: 20221115

Year of fee payment: 10

Ref country code: DK

Payment date: 20221115

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20221121

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231121

Year of fee payment: 11