EP3506659B1 - Dispositif d'aide auditive avec localisation de source acoustique et procédé associé - Google Patents

Dispositif d'aide auditive avec localisation de source acoustique et procédé associé Download PDF

Info

Publication number
EP3506659B1
EP3506659B1 EP19158537.1A EP19158537A EP3506659B1 EP 3506659 B1 EP3506659 B1 EP 3506659B1 EP 19158537 A EP19158537 A EP 19158537A EP 3506659 B1 EP3506659 B1 EP 3506659B1
Authority
EP
European Patent Office
Prior art keywords
signal
hearing device
processing unit
summing
input signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP19158537.1A
Other languages
German (de)
English (en)
Other versions
EP3506659A1 (fr
Inventor
Jesper UDESEN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GN Hearing AS
Original Assignee
GN Hearing AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GN Hearing AS filed Critical GN Hearing AS
Priority to EP19158537.1A priority Critical patent/EP3506659B1/fr
Priority to DK19158537.1T priority patent/DK3506659T3/da
Publication of EP3506659A1 publication Critical patent/EP3506659A1/fr
Application granted granted Critical
Publication of EP3506659B1 publication Critical patent/EP3506659B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
    • H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/40—Arrangements for obtaining a desired directivity characteristic
    • H04R25/407—Circuits for combining signals of a plurality of transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
    • H04R25/552—Binaural
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
    • H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils

Definitions

  • the present disclosure relates to a hearing device and related method and in particular to a hearing device for provision of sound source localization of wireless sound sources, such as spouse microphones, and method for provision of sound source localization of wireless sound sources.
  • EP2584794 A1 discloses a binaural listening system comprising first and second listening devices adapted for being located at or in left and right ears, respectively, of a user, the binaural listening system being adapted for receiving a wirelessly transmitted signal comprising a target signal and an acoustically propagated signal comprising the target signal as modified by respective first and second acoustic propagation paths from an audio source to the first and second listening devices.
  • a hearing device comprising a processing unit, a transceiver connected to the processing unit and being configured for outputting a transceiver output signal representative of a first audio signal to form a first input signal for the processing unit, and a microphone or microphone unit connected to the processing unit for converting a second audio signal into a microphone output signal to form a second input signal for the processing unit.
  • the processing unit is configured to estimate or measure a time shift or time lag between the microphone output signal and the transceiver output signal, the time shift being a property between the microphone output signal and the transceiver output signal.
  • the processing unit is configured to determine or select a time delay based on the time shift.
  • the processing unit is configured to apply the time delay to the first input signal to form a first summing signal.
  • the processing unit may be configured to combine the first summing signal and a second summing signal.
  • the second summing signal is based on the second input signal to the processing unit.
  • a method of operating a hearing device comprising a processing unit, a transceiver connected to the processing unit, and a microphone connected to the processing unit.
  • the method comprises estimating a time shift between a microphone output signal and a transceiver output signal.
  • the method comprises determining a time delay based on the time shift.
  • the method comprises applying the time delay to the first input signal to the processing unit to form a first summing signal.
  • the method comprises combining the first summing signal and a second summing signal, where the second summing signal is based on the second input signal to the processing unit.
  • a hearing device system comprising a first and a second hearing device, wherein the first hearing device is a hearing device according to the above and following exemplary hearing devices.
  • the hearing device and method is able to compensate for possible hardware delays in both the hearing device and in the spouse microphone device.
  • a time shift between the first and the second summing signals is anywhere from 2 ms to 50 ms.
  • the processing unit is configured to estimate the time shift between the microphone output signal and the transceiver output signal using cross correlation, summing of squared differences, summing of absolute differences, or any combination of the foregoing.
  • the processing unit is configured to output the time shift between the microphone output signal and the transceiver output signal and/or the time delay for transmission to another hearing device.
  • the processing unit is configured to estimate a first correlation coefficient corresponding to the time shift, and output the first correlation coefficient for transmission to another hearing device.
  • the processing unit is configured to receive a time shift between input signals in another hearing device, and a second correlation coefficient corresponding to the time shift between the input signals in the other hearing device.
  • the processing unit is configured to determine the time delay based on the time shift between the input signals in the other hearing device and the second correlation coefficient.
  • a hearing device system includes a first hearing device and a second hearing device, wherein the first hearing device is any of the hearing devices described herein.
  • a method of operating a hearing device comprising a processing unit, a transceiver connected to the processing unit, and a microphone connected to the processing unit, the method includes: estimating a time shift between a microphone output signal and a transceiver output signal; determining a time delay based on the time shift; and using the time delay to obtain a summing signal.
  • the disclosed hearing device may comprise a receiver connected to the processing unit for converting an output signal from the processing unit to an audio output signal.
  • the hearing device may be a hearing aid, e.g. the processing unit may be configured for hearing loss compensation of a user's hearing loss.
  • pre-processing of one or more of the audio signals may be performed, and/or post-processing of one or more of the audio signals may be performed, such as e.g. applying a gain, performing filtering by filters, providing A/D conversion by an A/D converter etc.
  • an apparatus and method that provides localization of spouse microphones and other wireless sound sources and at the same time provides a high signal-to-noise ratio of speech from the person wearing the spouse microphone.
  • the hearing device comprises a transceiver that is configured for outputting a transceiver output signal representative of a first audio signal.
  • the first audio signal may be a spouse microphone signal or other sound source signal.
  • the transceiver may be configured to operate in the 2.4-2.5 GHz band.
  • the transceiver may be a Bluetooth or Bluetooth Low Energy transceiver.
  • the hearing device comprises a microphone or microphone unit configured for converting a second audio signal into a microphone output signal which is the hearing aid microphone signal.
  • the microphone output signal (second input signal) may comprise the natural intra-aural time difference (ITD) and intra-aural level difference (ILD) cues from the head geometry of the hearing aid user.
  • ITD intra-aural time difference
  • ILD intra-aural level difference
  • the signal-to-noise ratio (SNR) in the microphone output signal is typically much lower than in the transceiver output signal, i.e. the spouse microphone signal.
  • the microphone output signal is typically delayed, e.g. in the range from 0-30 ms due to a distance, d, between the hearing device worn by the user and the spouse microphone, or other wireless sound sources, providing the transceiver output signal.
  • the microphone unit may comprise a plurality of microphones and/or a directional microphone. Accordingly, the second input signal may be a beamformed signal.
  • the processing unit is configured to estimate a time shift or time lag between the transceiver output signal (first input signal) and the microphone output signal (second input signal).
  • the processing unit may comprise a time shift estimator.
  • the time shift between the transceiver output signal (first input signal) and the microphone output signal (second input signal) is caused by the distance between the hearing device user and the spouse carrying the spouse microphone.
  • Hardware delays in the spouse microphone and in the hearing device also contribute to or affect the time shift between the transceiver output signal (first input signal) and the microphone output signal (second input signal).
  • the processing unit is configured to determine or select a time delay based on the time shift ⁇ t between the transceiver output signal (first input signal) and the microphone output signal (second input signal).
  • the processing unit may comprise a time delay selector.
  • the time delay is a function of the time shift.
  • the time delay(s) is/are determined such that a time shift ⁇ t' between the first and the second summing signals is within a desired range, such as in the range from 2 to 50 ms or in the range from 5 to 30 ms.
  • To determine or select a time delay D based on the time shift ⁇ t between the transceiver output signal (first input signal) and the microphone output signal (second input signal) may comprise to determine or select a first time delay D1 for the first input signal and/or a second time delay D2 for the second input signal based on the time shift ⁇ t between the transceiver output signal (first input signal) and the microphone output signal (second input signal).
  • the term “summing signal” refers to any signal, which may be obtained based on one or more input signals. The term “summing signal” does not imply or require that the summing signal is the result of or used for a summation unless otherwise stated.
  • the time delay is applied to the first input signal.
  • the input signal which the time delay is applied to is provided to the processing unit to form a first summing signal.
  • the delay of the first input signal is performed such that the hearing device user will hear the second audio signals from the hearing aid microphones in some appropriate time before hearing the first audio signals from e.g. the spouse microphone. Thereby, the hearing device user will be able to locate the person using the spouse microphone using the localization cues in the second audio signal and at the same time obtain a high signal-to-noise ratio of the speech signal due to the so-called precedence effect.
  • the processing unit may be configured to determine or select one or more time delays (and to which input signal the time delay(s) is/are applied to) such that a time shift between the first and second summing signals is within a desired range, such as in the range from 2 ms to 50 ms, and such that the summing signal based on the second input signal comes first, i.e. before the summing signal based on the first input signal, in order to take advantage of the precedence effect.
  • the second summing signal is based on the second input signal to the processing unit.
  • the processing unit may comprise at least one delay unit, such as a first delay unit and/or a second delay unit. A single delay unit may be used.
  • the first summing signal and the second summing signal are combined, e.g. by addition or weighted sum.
  • s1 is the transceiver output signal from the spouse microphone (first input signal)
  • s2 is the microphone output signal from the hearing device (second input signal)
  • d is the distance between the hearing device worn by the user and the spouse microphone
  • c is the speed of sound e.g. the speed of sound in air at sea level i.e. approximately 340 m/s.
  • the combined signal s3(t) is used for hearing device processing and the processed signal is presented to the user in the hearing device.
  • the result is that the user will hear the audio signals from the hearing aid microphone(s) some millisecond before, such as 2-50 ms before, the audio signals from the spouse microphone(s), thus maintaining localization with an improved signal-to-noise ratio.
  • the method is based on the precedence effect, also called the law of the first wavefront or Haas effect and is a binaural psychoacoustic effect.
  • the listener perceive a single fused auditory image; its perceived spatial location is dominated by the location of the first-arriving sound, i.e. the first wave front.
  • the lagging sound also affects the perceived location. However, its effect is suppressed by the first-arriving sound.
  • the hearing device user's brain uses the ILD and ITD cues in the first arriving wave front, which is from the hearing aid microphones (second input signal), to localize the sound.
  • the part of the signal from the spouse microphones i.e. the transceiver output signal (first input signal)
  • the brain has the ability not to use the lack of localization cues from the first input signal when it is delayed compared to the second input signal.
  • the improved SNR in the transceiver output signal from the spouse microphone is still used by the brain.
  • the hearing device user receives the sound, typically talk or speech, from the spouse both from the hearing device microphone(s) (second input signal) and from the spouse microphone (first input signal) in the form of the transceiver connected to the processing unit and being configured for outputting a transceiver output signal representative of a first audio signal to form a first input signal.
  • the spouse microphone providing the first input signal is configured to be arranged on, such as worn by the spouse, or at, such as placed on a table in front of the spouse.
  • the spouse microphone providing the first input signal may be a directional microphone, and thus the spouse microphone will typically receive and transmit the sound mainly from the spouse and substantially not transmit sound from the surroundings.
  • the hearing device microphone (second input signal) will typically capture sounds from the surroundings, including the sound from the spouse, which is also transmitted to the hearing device as the first audio signal.
  • spouse microphone is used here as a term to refer to a microphone, typically a directional microphone, which is arranged distant from the hearing device worn by the hearing device user, i.e. the spouse microphone is not arranged in the hearing device.
  • the spouse microphone will typically be worn by a spouse to the hearing device user, or be placed on a surface, such as table in front of the spouse for capturing the sounds, e.g. voice, sounds, speech or talk from the spouse, to be transmitted wirelessly to the hearing device of the hearing device user.
  • the person whose sounds or voice is captured by the spouse microphone may be a spouse to the hearing device user, a family member, a friend, a colleague, a business partner etc., i.e. any person which the hearing device user wishes to clearly hear the sounds e.g. talk from, e.g. in an environment with many sounds, such as in a restaurant, in an office facility, at home, in a garden etc..
  • the transceiver output signal representative of a first audio signal is not limited to a spouse microphone signal, but may be any, e.g. wireless, microphone signal, thus the first audio signal may be any audio signal.
  • the first input signal or the second input signal may be received first in the processing unit of the hearing device.
  • the other input signal i.e. the first input signal or the second input signal, may be received subsequently, such as 1 ms, 2 ms, 5 ms, 10 ms, 15 ms, 20 ms, 25 ms, 30 ms, or 35 ms later depending on the distance to the spouse microphone and hardware delays.
  • the first input signal (transceiver output signal) is received first in the hearing device (processing unit), i.e. before the second input signal, as the second input signal typically is delayed, such as delayed 0-30 ms, due to the distance between the hearing device user and the spouse wearing the spouse microphone.
  • a processing unit configured to apply time delay to both the first and second input signal may be preferred to provide a high degree of freedom in order to provide a combined signal with localization cues and a large signal to noise ratio.
  • a processing unit configured to apply time delay to both the first and second input signal may be preferred for a binaural hearing device system in order to provide and maintain localization in all listening situations.
  • a first hearing device may require that the microphone signal (second input signal) is delayed with a second delay.
  • ITD interaural time difference
  • the processing unit is configured to apply the determined time delay to the first input signal.
  • the processing unit may be configured to apply a first time delay to the first input signal and a second time delay to the second input signal.
  • the processing unit may comprise a first time delay unit and/or a second time delay unit.
  • the first time delay unit and the second time delay unit may be embedded as a time delay unit in a combined time delay selector and time delay unit.
  • the processing unit is configured to combine, such as adding or linearly combining, the first summing signal and a second summing signal.
  • the processing unit may comprise a combiner, such as an adder or weighted sum unit.
  • the second summing signal is based on the second input signal to the processing unit.
  • the second summing signal may be the second input signal, e.g. when only the first input signal is delayed.
  • the second summing signal may be the first input signal, e.g. when only the second input signal is delayed.
  • the second input signal may be delayed when the time shift between the first and second input signals is too large to take suitable advantage of the precedence effect, e.g. if the speaker is close to the hearing device wearer and the hardware delays in the spouse microphone and the transceiver are significant, such that the second input signal is received too early.
  • the time delay may be selected such that a time shift between the first and the second summing signals is between 2 ms and 50 ms, such as between 5-10ms, such as between 5-15 ms, such as between 5-20 ms, such as between 10-20ms, such as between 15-25ms, such as between 20-30ms, such as between 30-40ms, such as between 40-50 ms.
  • This time shift or time lag between the two summing signals is performed such that the hearing device user will hear the second audio signals from the hearing aid microphones in some appropriate time before hearing the first audio signals from e.g. the spouse microphone, i.e. for providing the precedence effect.
  • the time delay may be selected such that a time shift between the first and the second summing signals is between 5 ms and 30 ms.
  • the time shift between the transceiver output signal (first input signal) and the microphone output signal (second input signal) may be estimated using at least one of cross correlation, summing of squared differences and summing of absolute differences.
  • the cross correlation, the summing of squared differences and the summing of absolute differences may be termed time shift estimators.
  • signals may be multiplied, added, subtracted, divided etc. If the difference in amplitude between two input signals in e.g. the cross correlation is large, then the cross correlation may not be optimal.
  • the processing unit may therefore be configured to normalise one or more of the input signals before and/or after the cross correlation in order to obtain signals with a similar amplitude, thereby providing an optimal cross correlation.
  • the processing unit may be configured to transmit time shift between the transceiver output signal (first input signal) and the microphone output signal (second input signal).
  • the processing unit may be configured to transmit time delay(s) based on the time shift between the microphone output signal (second input signal) and the transceiver output signal (first input signal) and/or which input signal(s) the time delay(s) is/are applied to another hearing device.
  • the other hearing device may be a hearing device in the other ear of the user, i.e. when the user wears a hearing device in both ears, i.e. a binaural hearing device system. Thereby an effective binaural processing is facilitated by enabling the hearing devices of the binaural hearing device system to maintain localization cues, in particular if the second input signal is delayed.
  • the processing unit may be configured to estimate a first correlation coefficient corresponding to the time shift.
  • the first correlation coefficient may be the correlation value at the time shift.
  • the processing unit may be configured to transmit the first correlation coefficient to another hearing device.
  • the first correlation coefficient may be estimated in the hearing device and then transmitted to the other hearing device, e.g. the hearing device in the user's other ear.
  • the processing unit may be configured to receive a time shift between input signals in another hearing device and/or a second correlation coefficient corresponding to the time shift between input signals in the other hearing device.
  • the second correlation coefficient may be the correlation value at the time shift between input signals in the other hearing device.
  • the processing unit may be configured to receive one or more time delays from another hearing device.
  • the processing unit may receive the time shift between input signals in the other hearing device and/or the second correlation coefficient from the other hearing device, e.g. from a hearing device in the user's other ear. Alternatively, and/or additionally, the processing unit may receive the time shift between input signals in the other hearing device and/or the second correlation coefficient from a smart phone and/or the like.
  • the smart phone may be wirelessly connected with both hearing devices in a binaural hearing device system, and the smart phone may be configured for transmitting and/or receiving time shifts, e.g. a first and second time shift, and/or correlation coefficients, e.g. a first and a second correlation coefficient, to and/or from the two hearing devices in a binaural hearing device system.
  • the processing unit may be configured to determine or select time delay(s) based on the time shift of the other hearing device, time delay(s) of the other hearing device, and/or second correlation coefficient from the other hearing device, and apply the determined time delay(s) to the input signal(s).
  • the time delay may correspond to the estimated time shift corresponding to the hearing device with the highest correlation coefficient. Applying this time delay may provide the best signal-to-noise ratio and thus the best final audio signal in the binaural hearing device system to the user.
  • the processing unit may be configured to determine time delay(s) based on the time shift between input signals in the other hearing device and/or the second correlation coefficient. The time delay(s) of the hearing device may then be applied to input signal(s) to the processing unit, e.g. to form the first summing signal.
  • the processing unit in the hearing device may be configured to determine and apply time delay(s) at a fixed or variable frequency.
  • the time delays may be determined and/or updated at an update frequency of 1 Hz or less.
  • a suitable update frequency may be in the range from 0.1 Hz -10 Hz, such as in the range from 0.5 Hz to 2 Hz, e.g. 1 Hz, e.g. in order to balance the requirement to adapt to changing input signals (change in position) and the limited power resources of a hearing device.
  • the time delays may be adjusted based on a user input or an update criterion and/or an update event e.g. from another hearing device.
  • the processing unit may be configured to determine whether the time delay changes more than a threshold value, and optionally apply updated time delay(s), if the change(s) in time delay(s) meets an update criterion, e.g. if the change in the time delay applied is larger than the threshold value.
  • An algorithmic processing of the processing unit may overrule the change and/or update in time delay, e.g. if the time delay changes more than the predetermined threshold value, as this would, possibly erroneously be perceived by the hearing device user as, if the spouse wearing the spouse microphone providing the transceiver output signal (first input signal) moves too much too fast. Thus the first audio signal and the second audio signal would then be out of synchronization and this is not optimal for the hearing device user to listen to.
  • Fig. 1 schematically illustrates an exemplary hearing device for provision of sound source localization of wireless sound sources.
  • the hearing device is configured to add localization cues to a first audio signal, such as a spouse microphone signal by application of suitable delay(s) to one or more input signals.
  • the hearing device 2 comprises a processing unit 4, 4', 4" a transceiver 6 connected to the processing unit 4, 4', 4" and being configured for outputting a transceiver output signal 8 representative of a first audio signal 10 to form a first input signal to the processing unit 4, 4', 4".
  • the first audio signal 10 is a wireless audio signal.
  • the hearing device 2 comprises a microphone 12 connected to the processing unit 4 for converting a second audio signal 14 into a microphone output signal 16 to form a second input signal to the processing unit 4.
  • the second audio signal 14 is an acoustic signal.
  • the processing unit 4, 4', 4" is configured to estimate a time shift between the microphone output signal 16 and the transceiver output signal 8 and/or which of the microphone output signal 16 and the transceiver output signal is first in time.
  • the processing unit 4, 4', 4" is configured to determine one or more time delays, e.g.
  • a first time delay and/or a second time delay based on the time shift
  • the processing unit 4, 4', 4" is configured to apply the determined time delay(s) to the first input signal and optionally to the second input signal to the processing unit 4, 4', 4".
  • the processing unit 4, 4', 4" is configured to form a first summing signal and a second summing signal and to combine the first summing signal and the second summing signal.
  • the second summing signal is based on the second input signal.
  • the hearing device 2 comprises a receiver 18, such as an output transducer, a loudspeaker or speaker, connected to the processing unit 4, 4', 4" for outputting an output signal 19 of the processing unit (processed first and second audio signals) into the ear of the hearing device user.
  • the combined signal comprising the first summing signal and the second summing signal may be processed further by the processing unit 4, 4', 4" before being outputted or transmitted through the transducer 18, such as processed to apply a gain to the combined signal, such as processed to compensate the combined signal for the hearing loss specific for the hearing device user etc.
  • Fig. 2 schematically illustrates an exemplary processing unit 4 of a hearing device for provision of sound source localization of wireless sound sources.
  • a transceiver output signal 8 representative of a first audio signal forms a first input signal 20 to the processing unit 4.
  • a microphone output signal 16 representative of a second audio signal forms a second input signal 22 to the processing unit 4.
  • a time shift estimator 24 is configured to estimate a time shift between the microphone output signal 16/second input signal 22 and the transceiver output signal 8/first input signal 20. Estimating the time shift with the time shift estimator 24 comprises using at least one of cross correlation, summing of squared differences and summing of absolute differences.
  • the processing unit comprises a time delay selector 25 connected to the time shift estimator and being configured to select or determine one or more time delays to be used in delay unit(s) 28, 30 of the processing unit.
  • a first time delay unit 28 is configured to apply a first time delay to the first input signal depending on a control signal 26 indicative of first time delay from the time delay selector.
  • a second time delay unit 30 is configured to apply a second time delay to the second input signal depending on a control signal 27 indicative of second time delay from the time delay selector.
  • the first time delay unit 28 outputs a first summing signal 32 based on the first input signal 20 and the control signal 26.
  • the second time delay unit 30 outputs a second summing signal 34 based on the second input signal 22 and the control signal 27.
  • the second input signal 22 may constitute the second summing signal 34.
  • the processing unit 4 comprises a combiner 36 in the form of an adder or weighted sum unit.
  • the combiner 36 adds the first summing signal 32 and the second summing signal 34 to form a combined signal 38.
  • the combined signal 38 is then fed to the hearing compensator 40 for hearing aid compensation of the combined signal 38.
  • the hearing compensator 40 may compensate the combined signal 38 according to a hearing loss of a user of the hearing device 2.
  • Fig. 3 schematically illustrates an exemplary processing unit 4' of a hearing device for provision of sound source localization of wireless sound sources.
  • the processing unit 4' feeds the second input signal 22 to the combiner 36. Accordingly, the second input signal is the second summing signal 34.
  • the first time delay applied in the first time delay unit 28 may be zero.
  • Fig. 4 schematically illustrates an exemplary processing unit 4" of a hearing device for provision of sound source localization of wireless sound sources.
  • the processing unit 4" comprises a combined time delay selector and time delay unit 42.
  • the combined time delay selector and time delay unit 42 selectively applies a time delay to the first input signal or the second input signal based on time delay control signal 43 from the time shift estimator 24 to form the first summing signal 32.
  • the input signal which is not to be delayed forms the second summing signal 34.
  • Fig. 5 shows an exemplary flowchart of a method 100 of operating a hearing device.
  • the method comprises estimating 101 a time shift between a microphone output signal (second input signal) and a transceiver output signal (first input signal).
  • Estimating 101 a time shift may comprise determining which of the microphone output signal and the transceiver output signal is first in time.
  • the method 100 proceeds to determining 102 a time delay based on the time shift and optionally based on which of the microphone output signal (second input signal) and the transceiver output signal (first input signal) is first in time.
  • the time delay may take both negative and positive values, where the sign of the time delay indicates which of the microphone output signal and the transceiver output signal is first in time.
  • the method 100 comprises applying 103 the determined time delay to the first input signal to the processing unit to form a first summing signal and combining 104 the first summing signal and a second summing signal, the second summing signal being based on the second input signal.
  • the combined signal may be fed to a hearing compensator for further processing.
  • the exemplary processing units 4, 4', 4" may be used for performing the method 100.
  • Fig. 6a-d shows exemplary graphs illustrating a first and a second input signal, time shift estimation in form of e.g. a cross correlation between the first and second input signals, and the combined first and second summing signals.
  • time shift estimation in form of e.g. a cross correlation between the first and second input signals
  • a transceiver is connected to the processing unit of the hearing device, the transceiver is configured for outputting a transceiver output signal representative of a first audio signal to form the first input signal 20 to the processing unit.
  • a microphone is connected to the processing unit for converting a second audio signal into a microphone output signal to form the second input signal 22 to the processing unit.
  • Fig. 6c shows an example of a time shift or time lag estimation, e.g. in form of a cross correlation, between the first input signal (the transceiver output signal), see fig. 6a ) and the second input signal (microphone output signal), see fig. 6b ).
  • the time shift ⁇ t between the first input signal and the second input signal may be selected as the time t where the cross-correlation between the first input signal and the second input signal has maximum. Accordingly as taken from Fig. 6c , the time shift ⁇ t is 3 ms.
  • Fig. 6d shows an example where the determined time delay D1 is applied to the first input signal, see fig.
  • a first summing signal 32 to form a first summing signal 32.
  • a second summing signal 34 is based on the second input signal, see fig. 6b ).
  • the first summing signal 32 and the second summing signal 34 are combined and transmitted to the hearing compensator of the hearing device.
  • the determined time delay between the summing signals is 20 ms as seen from Fig. 6d , as the time between the first summing signal 32 and the second summing signal is 20 ms.
  • Fig. 7 shows exemplary graphs of combined summing signals in the left and right hearing devices of a binaural hearing device system.
  • the x-axis shows time in milliseconds (ms).
  • the y-axis shows the amplitude of the audio signals.
  • the top graph shows first summing signal 32 and second summing signal 34 in a first hearing device (right hearing device in right ear).
  • the bottom graph shows first summing signal 32' and second summing signal 34' in a secoond hearing device (left hearing device in left ear).
  • the first summing signal 32 is based on a first input signal to the processing unit of the first hearing device
  • the second summing signal 34 is based on a second input signal to the processing unit of the first hearing device.
  • the first summing signal 32' is based on a first input signal to the processing unit of the second hearing device
  • the second summing signal 34' is based on a second input signal to the processing unit of the second hearing device.
  • the first and second input signals may be speech from a spouse to the hearing device user.
  • the first delay D1 and the second delay D2 applied to respective first and second input signals of the first hearing device, and the first delay D1' and the second delay D2' applied to respective first and second input signals of the second hearing device are determined (D1 and D2 in first hearing device and D1' and D2' in second hearing device, or D1, D2, D1' and D2' in one of the first and/or second hearing device) such that ITD between the second input signal of the first hearing and the second input signal of the second hearing aid are maintained between the second summing signals and such that the time shift ⁇ t 1 ' between summing signals 32 and 34 is in the range from 2-50 ms and the time shift ⁇ t 2 ' between summing signals 32' and 34' is in the range from 2-50 ms.
  • the second input signal is received first in the right hearing device and subsequently received in the left hearing device.
  • the difference in time between receiving the second input signal in the right hearing device and subsequently in the left hearing device is maintained or substantially maintained (e.g. ⁇ 5%) in the summing signals and marked in the graph of the left ear as an arrow named ITD, i.e. intra-aural time difference, indicating that this is the difference in time between the left and the right ear receiving the second input signal.
  • ITD intra-aural time difference
  • the reason for the time difference, ITD may be that the hearing device user is positioned such that his/her right ear points towards the source of the second input signal, and his/her left ear points away from the source of the second input signal 20.
  • the hearing device e.g. in the right ear
  • may communicate with another hearing device e.g. in the left ear, in a binaural hearing device system.
  • the hearing devices of a binaural hearing device system may be hearing devices with processing unit 4' of Fig. 3 , reducing the need for communication between the hearing devices and ensuring that ITD is maintained.
  • the correlation value or correlation coefficient of the hearing device in e.g. the right ear may be determined as explained above and termed a first correlation coefficient, and this first correlation coefficient may be compared with a correlation coefficient of the other hearing device, e.g.
  • the processing unit of the hearing device in e.g. the right ear or another processing unit, e.g. in the hearing device in the left ear, may determine which of the first and the second correlation coefficient is the highest.
  • the processing unit may then be configured to apply a determined time delay to the input signal, the time delay corresponding to the estimated time shift corresponding to the hearing device with the highest correlation coefficient. Applying this time delay will provide the best signal-to-noise ratio and thus the best final audio signal in both hearing devices, i.e. in the right and left ear, to the user.
  • the hearing devices of a binaural hearing device system may be hearing devices with processing unit 4' of Fig. 3 , reducing the need for communication between the hearing devices and at the same time ensuring that ITD is maintained.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Circuit For Audible Band Transducer (AREA)

Claims (12)

  1. Dispositif auditif (2) comprenant :
    - une unité de traitement (4, 4', 4") ;
    - un émetteur-récepteur (6) connecté à l'unité de traitement (4, 4', 4") et configuré pour faire sortir un signal de sortie d'émetteur-récepteur (8) représentatif d'un premier signal audio (10) pour former un premier signal d'entrée (20) pour l'unité de traitement (4, 4', 4") ; et
    - un microphone (12) connecté à l'unité de traitement (4, 4', 4") pour convertir un deuxième signal audio (14) en un signal de sortie de microphone (16) pour former un deuxième signal d'entrée (22) pour l'unité de traitement (4, 4', 4") ; l'unité de traitement (4, 4', 4") étant configurée pour :
    - estimer un décalage temporel entre le signal de sortie de microphone (16) et le signal de sortie d'émetteur-récepteur (8) ;
    - déterminer une temporisation en fonction du décalage temporel ;
    - appliquer la temporisation au premier signal d'entrée (20) à l'unité de traitement (4, 4', 4") pour former un premier signal de sommation (32, 32') ; et
    - combiner le premier signal de sommation (32, 32') et un deuxième signal de sommation (34, 34') pour générer un signal combiné, le deuxième signal de sommation (34, 34') étant basé sur le deuxième signal d'entrée (22) vers l'unité de traitement (4, 4', 4") ;
    caractérisé en ce que la temporisation est déterminée de telle sorte que si le signal combiné est présenté à un utilisateur du dispositif auditif (2), ledit utilisateur entende le deuxième signal d'entrée (22) avant d'entendre le premier signal d'entrée (20).
  2. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel le dispositif auditif (2) comprend un récepteur (18) connecté à l'unité de traitement (4, 4', 4"), et dans lequel le récepteur (18) est configuré pour faire sortir le signal combiné de l'unité de traitement (4, 4', 4"), le signal combiné comprenant le premier signal de sommation (32, 32') et le deuxième signal de sommation (34, 34').
  3. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel le dispositif auditif (2) est un dispositif auditif d'un système auditif binaural comprenant un dispositif auditif gauche et un dispositif auditif droit.
  4. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel la temporisation appliquée au premier signal d'entrée (20) est une première temporisation (D1) basée sur le décalage temporel, et dans laquelle une deuxième temporisation (D2) est appliqué au deuxième signal d'entrée (22) sur la base du décalage temporel.
  5. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel un décalage temporel entre les premier (32, 32') et deuxième signaux de sommation (34, 34') est dans la plage de 2 à 50 ms.
  6. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel le décalage temporel entre le signal de sortie de microphone (16) et le signal de sortie d'émetteur-récepteur (8) est estimé en utilisant au moins l'un parmi la corrélation croisée, la sommation de différences au carré et la sommation de différences absolues.
  7. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel l'unité de traitement (4, 4', 4") est configurée pour transmettre le décalage temporel entre le signal de sortie de microphone (16) et le signal de sortie d'émetteur-récepteur (8) et / ou la temporisation basée sur le décalage temporel entre le signal de sortie de microphone (16) et le signal de sortie d'émetteur-récepteur (8) vers un autre dispositif auditif.
  8. Dispositif auditif (2) selon l'une quelconque des revendications précédentes, dans lequel l'unité de traitement (4, 4', 4") est configurée pour estimer un premier coefficient de corrélation correspondant au décalage temporel et transmettre le premier coefficient de corrélation à un autre dispositif auditif.
  9. Dispositif auditif (2) selon la revendication 5 ou la revendication 6, dans lequel l'unité de traitement (4, 4', 4") est configurée pour recevoir un décalage temporel entre les signaux d'entrée dans un autre dispositif auditif et un deuxième coefficient de corrélation correspondant au décalage temporel entre les signaux d'entrée dans l'autre dispositif auditif.
  10. Dispositif auditif (2) selon la revendication 9, dans lequel l'unité de traitement (4, 4', 4") est configurée pour déterminer la temporisation sur la base du décalage temporel entre les signaux d'entrée dans l'autre dispositif auditif et le deuxième coefficient de corrélation.
  11. Système de dispositif auditif (2) comprenant un premier et un deuxième dispositif auditif, dans lequel le premier dispositif auditif est un dispositif auditif (2) selon l'une quelconque des revendications 1 à 10.
  12. Procédé (100) de fonctionnement d'un dispositif auditif (2), le dispositif auditif (2) comprenant une unité de traitement (4, 4', 4"), un émetteur-récepteur (6) connecté à l'unité de traitement (4, 4', 4"), et un microphone (12) connecté à l'unité de traitement (4, 4', 4"), dans lequel le procédé (100) comprend les étapes consistant à :
    - estimer (101) un décalage temporel entre le signal de sortie de microphone (16) et le signal de sortie d'émetteur-récepteur (8) ;
    - déterminer (102) une temporisation en fonction du décalage temporel ;
    - appliquer (103) la temporisation au premier signal d'entrée (20) à l'unité de traitement (4, 4', 4") pour former un premier signal de sommation (32, 32') ; et
    - combiner (104) le premier signal de sommation (32, 32') et un deuxième signal de sommation (34, 34') pour générer un signal combiné, le deuxième signal de sommation (34, 34') étant basé sur le deuxième signal d'entrée (22) vers l'unité de traitement (4, 4', 4") ;
    caractérisé en ce que la temporisation est déterminée de telle sorte que si le signal combiné est présenté à un utilisateur du dispositif auditif (2), ledit utilisateur entende le deuxième signal d'entrée (22) avant d'entendre le premier signal d'entrée (20).
EP19158537.1A 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé Active EP3506659B1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19158537.1A EP3506659B1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé
DK19158537.1T DK3506659T3 (da) 2014-12-29 2014-12-29 Høreapparat med lydkildelokalisering og relateret fremgangsmåde

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP19158537.1A EP3506659B1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé
EP14200405.0A EP3041269A1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
EP14200405.0A Division EP3041269A1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé

Publications (2)

Publication Number Publication Date
EP3506659A1 EP3506659A1 (fr) 2019-07-03
EP3506659B1 true EP3506659B1 (fr) 2021-10-20

Family

ID=52134059

Family Applications (2)

Application Number Title Priority Date Filing Date
EP19158537.1A Active EP3506659B1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé
EP14200405.0A Ceased EP3041269A1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP14200405.0A Ceased EP3041269A1 (fr) 2014-12-29 2014-12-29 Dispositif d'aide auditive avec localisation de source acoustique et procédé associé

Country Status (2)

Country Link
EP (2) EP3506659B1 (fr)
DK (1) DK3506659T3 (fr)

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2103179A1 (fr) * 2007-01-10 2009-09-23 Phonak AG Système et procédé pour fournir une aide auditive à un utilisateur
EP2584794A1 (fr) * 2011-10-17 2013-04-24 Oticon A/S Système d'écoute adapté à la communication en temps réel fournissant des informations spatiales dans un flux audio

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP3041269A1 (fr) 2016-07-06
DK3506659T3 (da) 2022-01-17
EP3506659A1 (fr) 2019-07-03

Similar Documents

Publication Publication Date Title
US10123134B2 (en) Binaural hearing assistance system comprising binaural noise reduction
US9949040B2 (en) Peer to peer hearing system
US9930456B2 (en) Method and apparatus for localization of streaming sources in hearing assistance system
EP3051844B1 (fr) Système auditif binauriculaire
EP3407627B1 (fr) Système d'assistance auditive incorporant une personnalisation de microphone directionnel
US20100002886A1 (en) Hearing system and method implementing binaural noise reduction preserving interaural transfer functions
US9432778B2 (en) Hearing aid with improved localization of a monaural signal source
EP2961199B1 (fr) Perception omnidirectionnelle dans un système de prothèse auditive
EP3386216B1 (fr) Système auditif comprenant un estimateur de niveau binaural et/ou de gain, et une méthode correspondante
US11805364B2 (en) Hearing device providing virtual sound
CN109845296B (zh) 双耳助听器系统和操作双耳助听器系统的方法
EP3998779A3 (fr) Procédé et système d'aide auditive binaurale et aide auditive comprenant une estimation de la propre voix d'un utilisateur
US9749755B2 (en) Hearing device with sound source localization and related method
EP2928213B1 (fr) Prothèse auditive à localisation améliorée d'une source de signal monophonique
EP3506659B1 (fr) Dispositif d'aide auditive avec localisation de source acoustique et procédé associé
DK201470832A1 (en) Hearing device with sound source localization and related method

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AC Divisional application: reference to earlier application

Ref document number: 3041269

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20191206

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20200430

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210517

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: GN HEARING A/S

AC Divisional application: reference to earlier application

Ref document number: 3041269

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014080838

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1440915

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211115

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20220113

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20211215

Year of fee payment: 8

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20211020

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1440915

Country of ref document: AT

Kind code of ref document: T

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220120

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20220121

Year of fee payment: 8

Ref country code: DK

Payment date: 20220112

Year of fee payment: 8

Ref country code: CH

Payment date: 20220310

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220220

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220221

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220120

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220121

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014080838

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20211231

26N No opposition filed

Effective date: 20220721

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211229

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211229

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20141229

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20221229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221229

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221231

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231218

Year of fee payment: 10

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211020