EP3041270A1 - Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon - Google Patents
Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon Download PDFInfo
- Publication number
- EP3041270A1 EP3041270A1 EP14200593.3A EP14200593A EP3041270A1 EP 3041270 A1 EP3041270 A1 EP 3041270A1 EP 14200593 A EP14200593 A EP 14200593A EP 3041270 A1 EP3041270 A1 EP 3041270A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- microphone signal
- signal
- hearing
- hearing aid
- external microphone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 55
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 55
- 230000004044 response Effects 0.000 claims abstract description 47
- 238000004891 communication Methods 0.000 claims abstract description 29
- 230000005236 sound signal Effects 0.000 claims abstract description 29
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 230000003044 adaptive effect Effects 0.000 claims description 32
- 208000016354 hearing loss disease Diseases 0.000 claims description 28
- 206010011878 Deafness Diseases 0.000 claims description 27
- 230000010370 hearing loss Effects 0.000 claims description 27
- 231100000888 hearing loss Toxicity 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 17
- 230000010255 response to auditory stimulus Effects 0.000 claims description 4
- 208000032041 Hearing impaired Diseases 0.000 description 27
- 230000006870 function Effects 0.000 description 17
- 238000012546 transfer Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 230000002452 interceptive effect Effects 0.000 description 7
- 210000005069 ears Anatomy 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000002411 adverse Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 210000003128 head Anatomy 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 230000008054 signal transmission Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 210000000613 ear canal Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000003477 cochlea Anatomy 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 210000000883 ear external Anatomy 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 210000003454 tympanic membrane Anatomy 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/40—Arrangements for obtaining a desired directivity characteristic
- H04R25/407—Circuits for combining signals of a plurality of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/552—Binaural
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/43—Electronic input selection or mixing based on input signal analysis, e.g. mixing or selection between microphone and telecoil or between microphones with different directivity characteristics
Definitions
- the present invention relates in a first aspect to a method of superimposing spatial auditory cues to an externally picked-up sound signal in a hearing instrument.
- the method comprises steps of a generating an external microphone signal by an external microphone arrangement and transmitting the external microphone signal to a wireless receiver of a first hearing instrument via a first wireless communication link. Further steps of the methodology comprise determining response characteristics of a first spatial synthesis filter by correlating the external microphone signal and a first hearing aid microphone signal of the first hearing instrument and filtering the external microphone signal by the first spatial synthesis filter to produce a first synthesized microphone signal comprising first spatial auditory cues.
- Hearing instruments or aids typically comprise a microphone arrangement which includes one or more microphones for receipt of incoming sound such as speech and music signals.
- the incoming sound is converted to an electric microphone signal or signals that are amplified and processed in a control and processing circuit of the hearing instrument in accordance with parameter settings of one or more preset listening program(s).
- the parameter settings for each listening program have typically been computed from the hearing impaired individual's specific hearing deficit or loss for example expressed in an audiogram.
- An output amplifier of the hearing instrument delivers the processed, i.e. hearing loss compensated, microphone signal to the user's ear canal via an output transducer such as a miniature speaker, receiver or possibly electrode array.
- the miniature speaker or receiver may be arranged inside housing or shell of the hearing instrument together with the microphone arrangement or arranged separately in an ear plug or earpiece of the hearing instrument.
- a hearing impaired person typically suffers from a loss of hearing sensitivity which loss is dependent upon both frequency and the level of the sound in question.
- a hearing impaired person may be able to hear certain frequencies (e.g., low frequencies) as well as a normal hearing person, but unable to hear sounds with the same sensitivity as a normal hearing individual at other frequencies (e.g., high frequencies).
- the hearing impaired person may perceive loud sounds, e.g. above 90 dB SPL, with the same intensity as the normal hearing person, but still unable to hear soft sounds with the same sensitivity as the normal hearing person.
- the hearing impaired person suffers from a loss of dynamic range at certain frequencies or frequency bands.
- the healthy hearing system relies on the well-known cocktail party effect to discriminate between the competing or interfering sound sources under such adverse listening conditions.
- the signal-to-noise ratio (SNR) of sound at the listener's ears may be very low for example around 0 dB.
- the cocktail party effect relies inter alia on spatial auditory cues in the competing or interfering sound sources to perform the discrimination based on spatial localization of the competing sound sources.
- the SNR of sound received at the hearing impaired individual's ears may be so low that the hearing impaired individual is unable to detect and use the spatial auditory cues to discriminate between different sound streams from the competing sound sources. This leads to a severe worsened ability to hearing and understanding speech in noisy sound environments for many hearing impaired persons compared to normal hearing subjects.
- the external microphone signal is transmitted to a wireless receiver of the left ear and/or right hearing instrument(s) via a suitable wireless communication link or links.
- the wireless communication link or links may be based proprietary or industry standard wireless technologies such as Bluetooth.
- the hearing instrument or instruments thereafter reproduces the external microphone signal with the SNR improved target sound signal to the hearing aid user's ear or ears via a suitable processor and output transducer.
- the external microphone signal generated by such prior art external microphone arrangements lacks spatial auditory cues because of its distant or remote position in the sound field. This distant or remote position typically lies far away from the hearing aid user's head and ears for example more than 5 meters or 10 meters away.
- the lack of these spatial auditory cues during reproduction of the external microphone signal in the hearing instrument or instruments leads to an artificial and unpleasant internalized perception of the target sound source.
- the sound source appears to be placed inside the hearing aid user's head.
- This problem has been addressed and solved by the present invention by generating and superimposing appropriate spatial auditory cues on a remotely recorded or picked-up microphone signal in connection with reproduction of the remotely picked-up microphone signal in the hearing instrument.
- a first aspect of the invention relates to a method of superimposing spatial auditory cues to an externally picked-up sound signal in a hearing instrument, comprising steps of:
- the present invention addresses and solves the above discussed prior art problems with artificial and unpleasant internalized perception of the target sound source when reproduced via the remotely placed external microphone arrangement instead of through the microphone arrangement of the first hearing aid or instrument.
- the determination of frequency response characteristics, or equivalently impulse response characteristics of the first spatial synthesis filter in accordance with the invention allows appropriate spatial auditory cues to be added or superimposed to the received external microphone signal. These spatial auditory cues correspond largely to the auditory cues that would be generated by sound propagating from the true spatial position of the target sound source relative to the hearing user's head where the first hearing instrument is arranged.
- the microphone arrangement of the first hearing instrument is preferably housed within a housing or shell of the first hearing instrument such that this microphone arrangement is arranged at, or in, the hearing aid user's left or right ear as the case may be.
- the skilled person will understand that the first hearing instrument may comprise different types of hearing instruments such as so-called BTE types, ITE types, CIC types, RIC types etc.
- the microphone arrangement of the first hearing instrument may be located at various locations at, or in, the user's ear such as behind the user's pinnae, or inside the user's outer ear or inside the user's ear canal.
- the first spatial synthesis filter may be determined solely from the first hearing aid microphone signal and the external microphone signal without involving a second hearing aid microphone signal picked-up at the user's other ear.
- This type of direct communication between the first and second hearing instruments would require the presence of a wireless transmitter in at least one of the first and second hearing instruments leading to increased power consumption and complexity of the hearing instruments in question.
- the present methodology preferably comprises further steps of:
- Another embodiment of the present methodology comprises superimposing respective spatial auditory cues to the remotely picked-up sound signal for a left ear, or first, hearing instrument and a right ear, or second, hearing instrument.
- This embodiment is capable of generating binaural spatial auditory cues to the hearing impaired individual to exploit the advantages associated with binaural processing of acoustic signals propagating in the sound field such as the target sound of the target sound source.
- This binaural methodology of superimposing spatial auditory cues to the remotely picked-up sound signal comprises further steps of:
- the step of processing the first synthesized microphone signal comprises:
- the mixing of the first synthesized microphone signal and the first hearing aid microphone signal comprises varying the ratio between the first synthesized microphone signal and the first hearing aid microphone signal in dependence of a signal to noise ratio of the first microphone signal.
- the skilled person will understand that there exist numerous way of correlating the external microphone signal and the first hearing aid microphone signal to determine of the response characteristics of the first spatial synthesis filter according to step d) and/or step d1) above.
- the external microphone signal and the first hearing aid microphone signal are cross-correlated to determine a time delay between these signals.
- This embodiment additionally comprises a step of determining a level difference between the external microphone signal and the first hearing aid microphone signal based on the cross-correlation of the external microphone signal and the first hearing aid microphone signal, determining the response characteristics of the first spatial synthesis filter by multiplying the determined time delay and the determined level difference,
- the first synthesized microphone signal may be generated in the time domain from the impulse response g L ( t ) of the first spatial synthesis filter by a further step of:
- the correlation of the external microphone signal and the first hearing aid microphone signal to determine of the response characteristics of the first spatial synthesis filter according to step d) and/or step d1) above comprises:
- the impulse response g L (t) of the first spatial synthesis filter can be computed in real-time as a corresponding adaptive filter by a suitably configured or programmed signal processor of the first hearing instrument and/or the second hearing instrument for the second spatial synthesis filter.
- the solution of g L (t) may comprise adaptively filtering the external microphone signal by a first adaptive filter to produce the first synthesized microphone signal as an output of the adaptive filter and subtracting the first synthesized microphone signal outputted by the first adaptive filter from the first hearing aid microphone signal to produce an error signal, adapting filter coefficients of the first adaptive filter according to a predetermined adaptive algorithm to minimize the error signal.
- a second aspect of the invention relates to a hearing aid system comprising a first hearing instrument and a portable external microphone unit.
- the portable external microphone unit comprises:
- the hearing aid system may be configured for binaural use and processing of the external microphone signal such that the first hearing instrument is arranged at, or in, the user's left or right ear and the second hearing instrument placed at, or in, the user's other ear.
- the hearing aid system may comprise the second hearing instrument which comprises:
- Signal processing functions of the each of the first and/or second signal processors may be executed or implemented by dedicated digital hardware or by one or more computer programs, program routines and threads of execution running on a software programmable signal processor or processors.
- Each of the computer programs, routines and threads of execution may comprise a plurality of executable program instructions.
- the signal processing functions may be performed by a combination of dedicated digital hardware and computer programs, routines and threads of execution running on the software programmable signal processor or processors.
- Each of the above-mentioned methodologies of correlating the external microphone signal and the second hearing aid microphone signal may be carried out by a computer program, program routine or thread of execution executable on a suitable software programmable microprocessor such as a programmable Digital Signal Processor.
- the microprocessor and/or the dedicated digital hardware may be integrated on an ASIC or implemented on a FPGA device.
- the filtering of the received external microphone signal by the first spatial synthesis filter may be carried out by a computer program, program routine or thread of execution executable on a suitable software programmable microprocessor such as a programmable Digital Signal Processor.
- the software programmable microprocessor and/or the dedicated digital hardware may be integrated on an ASIC or implemented on a FPGA device.
- Each of the first and second wireless communication links may be based on RF signal transmission of the external microphone signal to the first and/or second hearing instruments, e.g. analog FM technology or various types of digital transmission technology for example complying with a Bluetooth standard, such as Bluetooth LE or other standardized RF communication protocols.
- each of the first and second wireless communication links may be based on optical signal transmission.
- the same type of wireless communication technology is preferably used for the first and second wireless communication links to minimize system complexity.
- FIG. 1 is a schematic illustration of a hearing aid system in accordance with a first embodiment of the present invention operating in an adverse sound or listening environment.
- the hearing aid system 101 comprises an external microphone arrangement mounted within a portable housing structure of a portable external microphone unit 105.
- the external microphone arrangement may comprise one or more separate omnidirectional or directional microphones.
- the portable housing structure 105 may comprise a rechargeable battery package supplying power to the one or more separate microphones and further supplying power to various electronic circuits such as digital control logic, user readable screens or displays and a wireless transceiver (not shown).
- the external microphone arrangement may comprise a spouse microphone, clip microphone, a conference microphone or form part of a smartphone or mobile phone.
- the hearing aid system 101 comprises a first hearing instrument or aid 107 mounted in, or at, a hearing impaired individual's right or left ear and a second hearing instrument or aid 109 mounted in, or at, the hearing impaired individual's other ear,
- the hearing impaired individual 102 is binaurally fitted with hearing aids in the present exemplary embodiment of the invention such that a hearing loss compensated output signal is provided both the left and right ear.
- hearing instruments such as so-called BTE types, ITE types, CIC types etc., may be utilized depending on factors such as the size of the hearing impaired individual's hearing loss, personal preferences and handling capabilities.
- Each of the first and second hearing instruments 107, 109 comprises a wireless receiver or transceiver (not shown) allowing each hearing instrument to receive a wireless signal or data, in particular the previously discussed external microphone signal transmitted from the portable external microphone unit 105.
- the external microphone signal may be modulated and transmitted as an analog signal or as a digitally encoded signal via the wireless communication link 104.
- the wireless communication link may be based on RF signal transmission, e.g. FM technology or digital transmission technology for example complying with a Bluetooth standard or other standardized RF communication protocols.
- the wireless communication link 10 may be based on optical signal transmission.
- the hearing impaired individual 102 wishes to receive sound from the target sound source 103 which is a particular speaker placed on some distance away from the hearing impaired individual 102 outside the latter's median plane.
- the sound environment surrounding the hearing impaired individual 102 is adverse with a low SNR at the respective microphones of the first and second hearing instruments 107, 109.
- the interfering noise sound v L,R (t) may in practice comprises many different types of common noise mechanisms or sources such as competing speakers, motorized vehicles, wind noise, babble noise, music etc.
- the interfering noise sound v L,R (t) may in addition to direct noise sound components from the various noise sources also comprise various boundary reflections from room boundaries such as walls, floors and ceiling of a room 110 where the hearing impaired individual 102 is placed.
- the noise sources will often produce noise sound components from multiple spatial directions at the hearing impaired individual's ears making the sound field in the room 110 very challenging for understanding speech of the target speaker 103 without assistance from the external microphone arrangement.
- a first linear transfer function between the target speaker 103 and the first hearing instrument 107 is schematically illustrated by dotted line h L (t) and a second linear transfer function between the target speaker 103 and the second hearing instrument 109 is likewise schematically illustrated by a second dotted line h R (t).
- the first and second transfer functions h L (t) and h R (t) may be represented by their respective impulse responses or by their respective frequency responses due to the Fourier transform equivalence.
- the first and second linear transfer functions describe the sound propagation from the target speaker or talker 103 to the right and left microphones, respectively, of the first/right and left/second hearing instruments.
- the acoustic or sound signal picked-up by the microphone 107 of the first hearing instrument produces a first hearing aid microphone signal denoted s L ( t ) and the acoustic or sound signal picked-up by the microphone 109 of the right ear hearing instrument produces a second hearing aid microphone signal denoted s R ( t )) in the following.
- the noise sound signal at the microphone 109 of the right hearing instrument is denoted v R (t) and the noise sound signal at the microphone 107 of the left hearing instrument is denoted v L (t) in the following.
- the target speech signal produced by the target speaker 103 is denoted x(t) in in the following.
- the present embodiment of the methodology of deriving and superimposing spatial auditory cues onto the external microphone signal picked-up by the external microphone arrangement of the portable external microphone unit 105 in each of the left and right ear hearing instruments preferably comprises steps of:
- the auditory spatial cue determination or estimation comprises a time delay estimator and a signal level estimator.
- the first synthesized microphone signal y L ( t ) is produced by convolving the impulse response g L ( t ) of the left spatial synthesis filter with the external microphone signal s E ( t ) received by the left hearing instrument via the wireless communication link 104.
- the above-mentioned computations of the functions r L ( t ) , A L , g L ( t ) and y L ( t ) are preferably performed by a first signal processor of the left hearing instrument.
- the first signal processor may comprise a microprocessor and/or dedicated digital computational hardware for example comprising a hard-wired Digital Signal Processor (DSP).
- DSP Digital Signal Processor
- the first signal processor may comprise a software programmable DSP or a combination of dedicated digital computational hardware and the software programmable DSP.
- the a software programmable DSP may be configured to perform the above-mentioned computations by suitable program routines or threads each comprising a set of executable program instructions stored in a non-volatile memory device of the hearing instrument.
- the second synthesized microphone signal y R ( t ) is produced in a corresponding manner by convolving the impulse response g R ( t ) of the right spatial synthesis filter with the external microphone signal s E ( t ) received by the right hearing instrument via the wireless communication link 104 and proceeding in corresponding manner to the signal processing in the left hearing instrument.
- each of the above-mentioned microphone signals and impulse responses in the left and right hearing instruments preferably are represented in the digital domain such that the computational operations to produce the functions r L ( t ), A L , g L ( t ) and y L ( t ) are executed numerically on digital signals by the previously discussed types of Digital Signal Processors.
- Each of the first synthesized microphone signal y L ( t ), the first hearing aid microphone signal s L (t) and the external microphone signal s E ( t ) may be a digital signal for example sampled at a sampling frequency between 16 kHz and 48 kHz.
- the first synthesized microphone signal is preferably further processed by the first hearing aid signal processor to adapt characteristics of a hearing loss compensated output signal to the individual hearing loss profile of the hearing impaired user's left ear.
- the skilled person will appreciate that this further processing may include numerous types of ordinary and well-known signal processing functions such as multi-band dynamic range compression, noise reduction etc.
- the first synthesized microphone signal is reproduced to the hearing impaired person's left ear as the hearing loss compensated output signal via the first output transducer.
- the first (and also second) output transducer may comprise a miniature speaker, receiver or possibly an implantable electrode array for cochlea implant hearing aids.
- the second synthesized microphone signal may be processed in a corresponding manner by the signal processor of the second hearing instrument to produce a second synthesized microphone signal and reproducing the same to the hearing impaired person's right ear.
- the external microphone signal picked-up by the remote microphone arrangement housed in the portable external microphone unit 105 is presented to the hearing impaired person's left and right ears with appropriate spatial auditory cues corresponding to the spatial cues that would have existed in the hearing aid microphone signals if the target speech signal produced by the target speaker 103 at his or hers actual position in the listening room was conveyed acoustically to the left and right ear microphones 109, 107 of the hearing instruments.
- This feature solves the previously discussed problems associated with the artificial and internalized perception of the target sound source inside the hearing aid user's head in connection with reproduction of remotely picked-up microphone signals in prior art hearing aid systems.
- the first hearing loss compensated output signal does not exclusively include the first synthesized microphone signal, but also comprises a component of the first hearing aid microphone signal recorded by the first hearing aid microphone or microphones such that a mixture of these different microphone signals are presented to the left ear of the hearing impaired individual.
- the step of processing the first synthesized microphone signal y L (t) comprises:
- the mixing feature may be exploited to adjust the relative level of the "raw" or unprocessed microphone signal and the external microphone signal such that the SNR of the left hearing loss compensated output signal can be adjusted.
- the inclusion of a certain component of the first hearing aid microphone signal s L (t) in the left hearing loss compensated output signal z L ( t ) is advantageous in many circumstances.
- the presence of a component or portion of the first hearing aid microphone signal s L (t) supplies the hearing impaired person with a beneficial amount of "environmental awareness" where other sound sources of potential interest than the target speaker becomes audible.
- the other sound sources of interest could for example comprise another person or a portable communication device sitting next to the hearing impaired person.
- the ratio between the first synthesized microphone signal and the first hearing aid microphone signal s L (t) is varied in dependence of a signal to noise ratio of first hearing aid microphone signal s L (t).
- the signal to noise ratio of the first hearing aid microphone signal s L (t) may for example be estimated based on certain target sound data derived from the external microphone signal s E (t).
- the latter microphone signal is assumed to mainly or entirely be dominated by the target sound source, e.g. the target speech discussed above, and may hence be used to detect the level of target speech present in the first hearing aid microphone signal s L (t).
- the mixing feature according to equation (10) above may be implemented such that b is close to 1, when the signal to noise ratio of first hearing aid microphone signal s L (t) is high and b approaches 0 when the signal to noise ratio of first hearing aid microphone signal s L (t) is low.
- the value of b may for example be larger than 0.9 when the signal to noise ratio of first hearing aid microphone signal s L (t) is larger than 10 dB.
- the value of b may for example be smaller than 0.1 when the signal to noise ratio of first hearing aid microphone signal s L (t) is smaller than 3 dB or 0 dB.
- the estimation or computation of the auditory spatial cues comprises a direct or on-line estimation of the impulse responses of the left and/or right spatial synthesis filter g L (t), g R (t) that describe or model the linear transfer functions between the target sound source and the left ear and right ear hearing aid microphones, respectively.
- the external microphone signal s E (t) can reasonably be assumed to be dominated by the target sound signal (because of the proximity between the external microphone arrangement and the target sound source).
- This assumption implies that the only way to minimize the error of equation (11) (and correspondingly the error of equation (12) below) is to completely remove the target sound signal or component from the first hearing aid microphone signal s L (t). This is accomplished by choosing the response of the filter g(t) to match the first linear transfer function h L (t) between the target sound source or speaker 103 and the first hearing instrument 107.
- This reasoning is based on the assumption that the target sound signal is uncorrelated with the interfering noise sound v L,R (t). Experience shows that this generally is a valid assumption in numerous real-life sound environments.
- FIG. 2 shows a simplified schematic block diagram of how the above-mentioned optimization equation (11) can be solved in real-time in the signal processor of the schematically illustrated left hearing instrument 200 using an adaptive filter 209. A corresponding solution may of course be applied in a corresponding right left hearing instrument (not shown).
- LMS Least Mean Square
- RLS Recursive Least Square
- the external microphone signal s E (t) is received by the previously discussed wireless receiver (not shown) decoded and possibly converted to a digital format if received in analog format.
- the digital external microphone signal s E (t) is applied to an input of the adaptive filter 209 and filtered by a current transfer function/impulse response of the adaptive filter 209 to produce a first synthesized microphone signal y L (t) at an output of the adaptive filter.
- the first hearing aid microphone signal s L (t) is substantially simultaneously applied to a first input of a subtractor 204 or subtraction function 204.
- the first, or left ear, synthesized microphone signal y L (t) is applied to a second input of a subtractor 204 such that the latter produces an error signal ⁇ on signal line 206 which represents a difference between y L (t) and s L (t).
- the error signal ⁇ is applied to an adaptive control input of the adaptive filter 209 via the signal line 206 in a conventional manner such that the filter coefficients of the adaptive filter are adjusted to minimize the error signal ⁇ in accordance with the particular adaptive algorithm implemented by the adaptive filter 209.
- the first, or left ear, spatial synthesis filter is formed by the adaptive filter 209 which makes a real-time adaptive computation of filter coefficients g L (t).
- the digital external microphone signal s E (t) is filtered by the adaptive transfer function of the adaptive filter 209 which in turn represents the left ear spatial synthesis filter, to produce the left ear synthesized microphone signal y L (t) comprising the first spatial auditory cues.
- the left hearing instrument 200 additionally comprises the previously discussed miniature receiver or loudspeaker 211 which converts the hearing loss compensated output signal produced by the signal processor 208 to audible sound for transmission to the hearing impaired person's ear drum.
- the signal processor 208 may comprise a suitable output amplifier, e.g. a class D amplifier, for driving the miniature receiver or loudspeaker 211.
- a right ear hearing instrument may be identical to the above-discussed features and functions of the left hearing instrument 200 to produce a binaural signal to the hearing aid user.
- the mixing coefficient b may either be a fixed value or may be user operated.
- the mixing coefficient b may alternatively be controlled by a separate algorithm which monitors the SNR by comparing the contribution of the target signal component measured by the external microphone present in the hearing aid microphone signals and comparing the level of the target signal component to the noise component. When the SNR s high, b would go to 1, and when the SNR is low, b would approach 0.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Circuit For Audible Band Transducer (AREA)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DK14200593.3T DK3041270T3 (da) | 2014-12-30 | 2014-12-30 | Fremgangsmåde til overlejring af rumlige auditive markeringer på eksternt opfangede mikrofonsignaler |
EP14200593.3A EP3041270B1 (de) | 2014-12-30 | 2014-12-30 | Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon |
US14/589,587 US9699574B2 (en) | 2014-12-30 | 2015-01-05 | Method of superimposing spatial auditory cues on externally picked-up microphone signals |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP14200593.3A EP3041270B1 (de) | 2014-12-30 | 2014-12-30 | Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3041270A1 true EP3041270A1 (de) | 2016-07-06 |
EP3041270B1 EP3041270B1 (de) | 2019-05-15 |
Family
ID=52282567
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14200593.3A Active EP3041270B1 (de) | 2014-12-30 | 2014-12-30 | Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP3041270B1 (de) |
DK (1) | DK3041270T3 (de) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3468228A1 (de) * | 2017-10-05 | 2019-04-10 | GN Hearing A/S | Binaurales hörsystem mit lokalisierung von schallquellen |
CN112911480A (zh) * | 2021-01-22 | 2021-06-04 | 成都市舒听医疗器械有限责任公司 | 一种助听器声音放大方法及助听器 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120063610A1 (en) * | 2009-05-18 | 2012-03-15 | Thomas Kaulberg | Signal enhancement using wireless streaming |
US20130094683A1 (en) * | 2011-10-17 | 2013-04-18 | Oticon A/S | Listening system adapted for real-time communication providing spatial information in an audio stream |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10146886B4 (de) * | 2001-09-24 | 2007-11-08 | Siemens Audiologische Technik Gmbh | Hörgerät mit automatischer Umschaltung auf Hörspulenbetrieb |
DE102006046703A1 (de) * | 2006-10-02 | 2008-04-17 | Siemens Audiologische Technik Gmbh | Hörvorrichtung mit gesteuerten Eingangskanälen und entsprechendes Verfahren |
-
2014
- 2014-12-30 DK DK14200593.3T patent/DK3041270T3/da active
- 2014-12-30 EP EP14200593.3A patent/EP3041270B1/de active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120063610A1 (en) * | 2009-05-18 | 2012-03-15 | Thomas Kaulberg | Signal enhancement using wireless streaming |
US20130094683A1 (en) * | 2011-10-17 | 2013-04-18 | Oticon A/S | Listening system adapted for real-time communication providing spatial information in an audio stream |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3468228A1 (de) * | 2017-10-05 | 2019-04-10 | GN Hearing A/S | Binaurales hörsystem mit lokalisierung von schallquellen |
US11438713B2 (en) | 2017-10-05 | 2022-09-06 | Gn Hearing A/S | Binaural hearing system with localization of sound sources |
CN112911480A (zh) * | 2021-01-22 | 2021-06-04 | 成都市舒听医疗器械有限责任公司 | 一种助听器声音放大方法及助听器 |
Also Published As
Publication number | Publication date |
---|---|
EP3041270B1 (de) | 2019-05-15 |
DK3041270T3 (da) | 2019-08-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10431239B2 (en) | Hearing system | |
US9699574B2 (en) | Method of superimposing spatial auditory cues on externally picked-up microphone signals | |
US10225669B2 (en) | Hearing system comprising a binaural speech intelligibility predictor | |
EP2899996B1 (de) | Signalverbesserung mittels drahtlosem Streaming | |
US10349191B2 (en) | Binaural gearing system and method | |
US10070231B2 (en) | Hearing device with input transducer and wireless receiver | |
US20100020995A1 (en) | System for reducing acoustic feedback in hearing aids using inter-aural signal transmission, method and use | |
CN105744455B (zh) | 用于在外部拾取的麦克风信号上叠加空间听觉提示的方法 | |
US11330375B2 (en) | Method of adaptive mixing of uncorrelated or correlated noisy signals, and a hearing device | |
EP2928213B1 (de) | Hörgerät mit verbesserter Lokalisierung einer monauralen Signalquelle | |
CN113940097B (zh) | 包含时间去相关波束形成器的双边助听器系统 | |
EP2916320A1 (de) | Multi-Mikrofonverfahren zur Schätzung von Ziel- und Rauschspektralvarianzen | |
EP3041270B1 (de) | Verfahren zum Überlagern von räumlichen auditorischen Merkmalen auf Signalen eines externen Mikrofon | |
US20230080855A1 (en) | Method for operating a hearing device, and hearing device | |
US11115761B2 (en) | Methods and systems for hearing device signal enhancement using a remote microphone | |
EP4325892A1 (de) | Verfahren zur audiosignalverarbeitung, hörsystem und hörgerät | |
US20230143325A1 (en) | Hearing device or system comprising a noise control system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20170105 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20170926 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: GN HEARING A/S |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20181211 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602014046737 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20190801 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190815 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190915 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190815 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190816 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1134807 Country of ref document: AT Kind code of ref document: T Effective date: 20190515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602014046737 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
26N | No opposition filed |
Effective date: 20200218 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20191231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191230 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191230 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190915 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20141230 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20211221 Year of fee payment: 8 Ref country code: DK Payment date: 20211217 Year of fee payment: 8 Ref country code: FR Payment date: 20211215 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20211217 Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190515 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: EBP Effective date: 20221231 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20221230 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221230 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20221231 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20231218 Year of fee payment: 10 |