EP2822301A1 - Bestimmung von Individuellen HRTFs - Google Patents

Bestimmung von Individuellen HRTFs Download PDF

Info

Publication number
EP2822301A1
EP2822301A1 EP13175052.3A EP13175052A EP2822301A1 EP 2822301 A1 EP2822301 A1 EP 2822301A1 EP 13175052 A EP13175052 A EP 13175052A EP 2822301 A1 EP2822301 A1 EP 2822301A1
Authority
EP
European Patent Office
Prior art keywords
hrtfs
individual
hrtf
approximate
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP13175052.3A
Other languages
English (en)
French (fr)
Other versions
EP2822301B1 (de
Inventor
Jesper UDESEN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GN Hearing AS
Original Assignee
GN Resound AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GN Resound AS filed Critical GN Resound AS
Priority to DK13175052.3T priority Critical patent/DK2822301T3/da
Priority to EP13175052.3A priority patent/EP2822301B1/de
Priority to US13/949,134 priority patent/US9426589B2/en
Priority to JP2014137176A priority patent/JP5894634B2/ja
Priority to CN201410317428.9A priority patent/CN104284286B/zh
Publication of EP2822301A1 publication Critical patent/EP2822301A1/de
Application granted granted Critical
Publication of EP2822301B1 publication Critical patent/EP2822301B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/70—Adaptation of deaf aid to hearing loss, e.g. initial electronic fitting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation

Definitions

  • a new method of determining individual HRTFs, a new fitting system configured to determine individual HRTFs according to the new method, and a hearing instrument, or a device supplying audio to the hearing instrument, with the individual HRTFs determined according to the new method, are provided.
  • Hearing aid users have been reported to have poorer ability to localize sound sources when wearing their hearing aids than without their hearing aids. This represents a serious problem for the hearing impaired population.
  • hearing aids typically reproduce sound in such a way that the user perceives sound sources to be localized inside the head. The sound is said to be internalized rather than being externalized.
  • SNR signal to noise ratio
  • a significant contributor to this fact is that the hearing aid reproduces an internalized sound field. This adds to the cognitive loading of the hearing aid user and may result in listening fatigue and ultimately that the user removes the hearing aid(s).
  • a human with normal hearing will also experience benefits of improved externalization and localization of sound sources when using a hearing instrument, such as a headphone, headset, etc, e.g. playing computer games with moving virtual sound sources or otherwise enjoying replayed sound with externalized sound sources.
  • a hearing instrument such as a headphone, headset, etc, e.g. playing computer games with moving virtual sound sources or otherwise enjoying replayed sound with externalized sound sources.
  • Human beings detect and localize sound sources in three-dimensional space by means of the human binaural sound localization capability.
  • the input to the hearing consists of two signals, namely the sound pressures at each of the eardrums, in the following termed the binaural sound signals.
  • the human auditory system extracts information about distance and direction to a sound source, but it is known that the human auditory system uses a number of cues in this determination. Among the cues are spectral cues, reverberation cues, interaural time differences (ITD), interaural phase differences (IPD) and interaural level differences (ILD).
  • the transmission of a sound wave from a sound source to the ears of the listener, wherein the sound source is positioned at a given direction and distance in relation to the left and right ears of the listener is described in terms of two transfer functions, one for the left ear and one for the right ear, that include any linear transformation, such as coloration, interaural time differences and interaural spectral differences. These transfer functions change with direction and distance of the sound source in relation to the ears of the listener. It is possible to measure the transfer functions for any direction and distance and simulate the transfer functions, e.g. electronically, e.g. with digital filters.
  • a pair of filters are inserted in the signal path between a playback unit, such as a MP3-player, and headphones used by the listener, the pair of filters having transfer functions, one for the left ear and one for the right ear, of the transmission of a sound wave from a sound source positioned at a certain direction and distance in relation to the listener, to the positions of the headphones at the respective ears of the listener, the listener will achieve the perception that the sound generated by the headphones originates from a sound source, in the following denoted a "virtual sound source", positioned at the distance and in the direction in question, because of the true reproduction of the sound pressures at the eardrums in the ears.
  • the set of the two transfer functions is called a Head-Related Transfer Function (HRTF).
  • HRTF Head-Related Transfer Function
  • Each transfer function of the HRTF is defined as the ratio between a sound pressure p generated by a plane wave at a specific point in or close to the appertaining ear canal (p L ) in the left ear canal and p R in the right ear canal) in relation to a reference (p 1 ).
  • the reference traditionally chosen is the sound pressure p I that would have been generated by a plane wave at a position right in the middle of the head, but with the listener absent.
  • L designates the left ear and R designates the right ear
  • P is the pressure level in the frequency domain.
  • the time domain representation or description of the HRTF i.e. the inverse Fourier transforms of the HRTF, is designated the Head Related Impulse Response (HRIR).
  • HRIR Head Related Impulse Response
  • the time domain representation of the HRTF is a set of two impulse responses, one for the left ear and one for the right ear, each of which is the inverse Fourier transform of the corresponding transfer function of the set of two transfer functions of the HRTF in the frequency domain.
  • the HRTF contains all information relating to the sound transmission to the ears of the listener, including the geometries of a human being which are of influence to the sound transmission to the ears of the listener, e.g. due to diffraction around the head, reflections from shoulders, reflections in the ear canal, transmission characteristics through the ear canals, if the HRTF is determined for points inside the respective ear canals, etc. Since the anatomy of humans shows a substantial variability from one individual to the other, the HRTFs vary from individual to individual.
  • the complex shape of the ear is a major contributor to the individual spatial-spectral cues (ITD, ILD and spectral cues) of a listener.
  • one of the transfer functions of the HRTF i.e. the left ear part of the HRTF or the right ear part of the HRTF, will also be termed the HRTF for convenience.
  • the pair of transfer functions of a pair of filters simulating an HRTF is also denoted a Head-Related Transfer Function even though the pair of filters can only approximate an HRTF.
  • Reproduction of sound to the ears of a listener in such a way that spatial information about positions of sound sources with relation to the listener is maintained has several positive effects, including externalization of sound sources, maintenance of sense of direction, synergy between the visual and auditory systems, and better understanding of speech in noise.
  • measurement of individual HRTFs is performed with the individual standing in an anechoic chamber.
  • Such measurements are expensive, time consuming, and cumbersome, and probably unacceptable to the user.
  • HRTFs obtained by measurements with an artificial head, e.g. a KEMAR manikin.
  • An artificial head is a model of a human head where geometries of a human being which influence the propagation of sound to the eardrums of a human, including diffraction around the body, shoulder, head, and ears, are modelled as closely as possible.
  • two microphones are positioned in the ear canals of the artificial head to sense sound pressures, similar to the procedure for determination of HRTFs of a human.
  • the approximate HRTFs may be HRTFs determined in any other way than measurement of the HRTFs of the human in question with microphones positioned at the ears of the human in question, e.g. at the entrance to the ear canal of the left ear and right ear.
  • the approximate HRTFs may be previously determined for an artificial head, such as a KEMAR manikin, and stored for subsequent use.
  • the approximate HRTFs may for example be stored locally in a memory at the dispenser's office, or may be stored remotely on a server, e.g. in a database, for access through a network, such as a Wide-Area-Network, such as the Internet.
  • the approximate HRTFs may also be determined as an average of previously determined HRTFs for a group of humans.
  • the group of humans may be selected to fit certain features of the human for which the individual HRTFs are to be determined in order to obtain approximate HRTFs that more closely match the respective corresponding individual HRTFs.
  • the group of humans may be selected according to age, race, gender, family, ear size, etc, either alone or in any combination.
  • the approximate HRTFs may also be HRTFs previously determined for the human in question, e.g. during a previous fitting session at an earlier age.
  • HRTFs for the same combination of direction and distance, but obtained in different ways and/or for different humans and/or artificial heads, are termed corresponding HRTFs.
  • the deviation(s) of the one or more individual measured HRTF(s) with relation to the corresponding approximate HRTF(s) of the set of approximate HRTFs is/are determined by comparison in the time or frequency domain.
  • phase information may be disregarded.
  • the ears of a human are not sensitive to the phase of sound signals. What is important is the relative phase or time difference of sound signals as received at the ears of the human and as long as the relative time or phase differences are not disturbed; the HRTFs may be modified disregarding timing or phase information.
  • a single individual HRTF is measured, preferably a far field measurement in the forward looking direction is performed, i.e. 0° azimuth, 0° elevation.
  • the HRTFs do not change with distance.
  • the listener resides in the far field of a sound source, when the distance to the sound source is larger than 1.5 m.
  • the far field HRTF of one direction typically the forward looking direction is already measured.
  • the individual HRTFs may then be obtained by modification of the corresponding approximate HRTFs in accordance with a deviation(s) of the measured individual HRTF(s) with relation to the corresponding approximate HRTF(s) as determined in the frequency domain or in the time domain.
  • ⁇ is the azimuth
  • ⁇ is the elevation
  • d is the distance to the sound source position for which the individual HRTF is obtained.
  • ⁇ is the azimuth
  • ⁇ is the elevation
  • d is the distance to the sound source position for which the individual impulse response is obtained.
  • HRTFs of a plurality of combinations of directions and distances may be determined during a fitting session of a hearing instrument, typically including the forward looking direction.
  • Remaining individual HRTFs may then be obtained by modification of the corresponding approximate HRTFs in accordance with deviation(s) in the frequency domain or in the time domain of the measured individual HRTF(s) with relation to the corresponding approximate HRTF(s).
  • H d HRTF d individual / HRTF d app ,
  • a corresponding synthesizing filter H s may be determined by interpolation or extrapolation of the synthesizing filters H d , and each of the remaining individual HRTF r s of the human may be determined by multiplication of the corresponding approximate HRTF r with the synthesizing filter H s :
  • HRTF r individual ⁇ ⁇ ⁇ ⁇ d H s ⁇ HRTF r app ⁇ ⁇ ⁇ ⁇ d .
  • HRTF r individual ⁇ ⁇ ⁇ H s ⁇ HRTF r app ⁇ ⁇ ⁇ .
  • ⁇ is the azimuth
  • ⁇ is the elevation
  • d is the distance to the sound source position for which the individual HRTF is obtained.
  • a corresponding synthesizing impulse response h s may be determined by interpolation or extrapolation of the synthesizing impulse responses h d
  • ⁇ is the azimuth
  • ⁇ is the elevation
  • d is the distance to the sound source position for which the individual impulse response is obtained.
  • a large number of individual HRTFs may be provided without individual measurement of each of the individual HRTFs; rather measurement of a single or a few individual HRTFs is sufficient so that the set of individual HRTFs can be provided without discomfort to the intended user of the hearing instrument.
  • a hearing instrument comprising
  • the hearing instrument provides the user with improved sense of direction.
  • the hearing instrument may be a headset, a headphone, an earphone, an ear defender, an earmuff, etc, e.g. of the following types: Ear-Hook, In-Ear, On-Ear, Over-the-Ear, Behind-the-Neck, Helmet, Headguard, etc.
  • the hearing instrument may be a hearing aid, e.g. a binaural hearing aid, such as a BTE, a RIE, an ITE, an ITC, a CIC, etc, (binaural) hearing aid.
  • a binaural hearing aid such as a BTE, a RIE, an ITE, an ITC, a CIC, etc, (binaural) hearing aid.
  • the audio input signal may originate from a sound source, such as a monaural signal received from a spouse microphone, a media player, a hearing loop system, a teleconference system, a radio, a TV, a telephone, a device with an alarm, etc.,
  • a sound source such as a monaural signal received from a spouse microphone, a media player, a hearing loop system, a teleconference system, a radio, a TV, a telephone, a device with an alarm, etc.
  • the audio input signal is filtered with the binaural filter in such a way that the user perceives the received audio signal to be emitted by the sound source positioned in a position and/or arriving from a direction in space corresponding to the HRTF of the binaural filter.
  • the hearing instrument may be interconnected with a device, such as a hand-held device, such as a smart phone, e.g. an Iphone, an Android phone, a windows phone, etc.
  • a hand-held device such as a smart phone, e.g. an Iphone, an Android phone, a windows phone, etc.
  • the hearing instrument may comprise a data interface for transmission of data to the device.
  • the data interface may be a wired interface, e.g. a USB interface, or a wireless interface, such as a Bluetooth interface, e.g. a Bluetooth Low Energy interface.
  • a wired interface e.g. a USB interface
  • a wireless interface such as a Bluetooth interface, e.g. a Bluetooth Low Energy interface.
  • the hearing instrument may comprise an audio interface for reception of an audio signal from the device and for provision of the audio input signal.
  • the audio interface may be a wired interface or a wireless interface.
  • the data interface and the audio interface may be combined into a single interface, e.g. a USB interface, a Bluetooth interface, etc.
  • the hearing instrument may for example have a Bluetooth Low Energy data interface for exchange of control data between the hearing instrument and the device, and a wired audio interface for exchange of audio signals between the hearing instrument and the device.
  • the device may comprise a sound generator connected for outputting audio signals to the hearing instrument via pairs of filters with the determined individual HRTFs for generation of a binaural acoustic sound signal emitted towards the eardrums of the user.
  • a sound generator connected for outputting audio signals to the hearing instrument via pairs of filters with the determined individual HRTFs for generation of a binaural acoustic sound signal emitted towards the eardrums of the user.
  • the hearing instrument may comprise an ambient microphone for receiving ambient sound for transmission towards the ears of the user.
  • an ambient microphone for example in the event that the hearing instrument provides a sound proof, or substantially, sound proof, transmission path for sound emitted by the loudspeaker(s) of the hearing instrument towards the ear(s) of the user, the user may be acoustically disconnected in an undesirable way from the surroundings. This may for example be dangerous when moving in traffic.
  • the hearing instrument may have a user interface, e.g. a push button, so that the user can switch the microphone on and off as desired thereby connecting or disconnecting the ambient microphone and one loudspeaker of the hearing instrument.
  • a user interface e.g. a push button
  • the hearing instrument may have a mixer with an input connected to an output of the ambient microphone and another input connected to an output of the device supplying an audio signal, and an output providing an audio signal that is a weighted combination of the two input audio signals.
  • the user input may further include means for user adjustment of the weights of the combination of the two input audio signals, such as a dial, or a push button for incremental adjustment.
  • the hearing instrument may have a threshold detector for determining the loudness of the ambient signal received by the ambient microphone, and the mixer may be configured for including the output of the ambient microphone signal in its output signal only when a certain threshold is exceeded by the loudness of the ambient signal.
  • a fitting instrument for fitting a hearing aid to a user and operating in accordance with the new method for provision of individual HRTFs of the user to the hearing aid, is also provided.
  • Fitting instruments are well known in the art and have proven adequate for adjusting signal processing parameters of a hearing aid so that the hearing aid accurately compensates the actual hearing loss of the hearing aid user.
  • the fitting process typically involves measuring the auditory characteristics of the hearing aid user's hearing, estimating the acoustic characteristics needed to compensate for the particular auditory deficiency measured, adjusting the auditory characteristics of the acoustic hearing aid so that the appropriate acoustic characteristics may be delivered, and verifying that these particular auditory characteristics do compensate for the hearing deficiency found by operating the acoustic hearing aid in conjunction with the user.
  • Standard techniques are known for these fittings which are typically performed by an audiologist, hearing aid dispenser, otologist, otolaryngologist, or other doctor or medical specialist.
  • the threshold of the individual's hearing is typically measured using an audiometer, i.e. a calibrated sound stimulus producing device and calibrated headphones.
  • the measurement of the threshold of hearing takes place in a room with very little audible noise.
  • the audiometer generates pure tones at various frequencies between 125 Hz and 8,000 Hz. These tones are transmitted to the individual being tested, e.g. through headphones of the audiometer. Normally, the tones are presented in step of an octave or half an octave. The intensity or volume of the pure tones is varied and reduced until the individual can just barely detect the presence of the tone. This intensity threshold is often defined and found as the intensity of which the individual can detect 50 percent of the tones presented. For each pure tone, this intensity threshold is known as the individual's air conduction threshold of hearing. Although the threshold of hearing is only one element among several that characterizes an individual's hearing loss, it is the predominant measure traditionally used to acoustically fit a hearing aid.
  • this threshold is used to estimate the amount of amplification, compression, and/or other adjustment that will be employed to compensate for the individual's loss of hearing.
  • the implementation of the amplification, compression, and/or other adjustments and the hearing compensation achieved thereby depends upon the hearing aid being employed.
  • the new fitting instrument has a processor that is further configured for determining individual HRTFs of a user of the hearing aid to be fitted, by obtaining approximate HRTFs , e.g. from a server accessed through the Internet.
  • the processor is further configured for determination of individual HRTFs or HRIRs by determination of deviation(s) of the measured one or more individual HRTF(s) or HRIR(s) with relation to the corresponding approximated HRTF(s) or HRIR(s), respectively, and subsequent determination of other HRTFs or HRIRs based on the corresponding approximate HTRFs or HRIRs and the determined deviation(s).
  • Signal processing in the new hearing aid and in the new fitting instrument may be performed by dedicated hardware or may be performed in a signal processor, or performed in a combination of dedicated hardware and one or more signal processors.
  • processor signal processor
  • controller system
  • CPU central processing unit
  • a "processor”, “signal processor”, “controller”, “system”, etc. may be, but is not limited to being, a process running on a processor, a processor, an object, an executable file, a thread of execution, and/or a program.
  • processor designate both an application running on a processor and a hardware processor.
  • processors may reside within a process and/or thread of execution, and one or more "processors”, “signal processors”, “controllers”, “systems”, etc., or any combination hereof, may be localized on one hardware processor, possibly in combination with other hardware circuitry, and/or distributed between two or more hardware processors, possibly in combination with other hardware circuitry.
  • a processor may be any component or any combination of components that is capable of performing signal processing.
  • the signal processor may be an ASIC processor, a FPGA processor, a general purpose processor, a microprocessor, a circuit component, or an integrated circuit.
  • the at least one measured HRTF comprises only a single measured HRTF.
  • the act of obtaining the set of approximate HRTFs includes determining the approximate HRTFs for an artificial head.
  • the act of obtaining the set of approximate HRTFs includes retrieving the approximate HRTFs from a database.
  • the method further includes: classifying the specific human into a predetermined group of humans; and retrieving the approximate HRTFs from a database with HRTFs relating to the predetermined group of humans, such as average HRTFs of the predetermined group of humans, or previously measured HRTFs of one or more humans representing the predetermined group of humans.
  • the act of modifying includes: calculating ratio(s) between the at least one measured HRTF and the corresponding approximate HRTF(s), and forming the set of individual HRTFs by modification of the set of approximate HRTFs in accordance with the calculated ratio(s).
  • the at least one measured HRTF comprises a plurality of measured HRTFs; the method further comprises determining additional deviation(s) of other one(s) of the measured HRTFs with relation to corresponding one(s) of the set of approximate HRTFs; and the act of forming the set of individual HRTFs comprises modifying the set of approximate HRTFs based at least in part on the determined deviation and the determined additional deviation(s).
  • a fitting instrument for fitting a hearing aid to a user includes a processor configured for retrieving a set of approximate HRTFs from a memory of the fitting instrument or a remote server; obtaining at least one measured HRTF of the user; determining a deviation of one of the at least one measured HRTF with relation to a corresponding one of the set of approximate HRTFs; and forming a set of individual HRTFs by modification of the set of approximate HRTFs based at least in part on the determined deviation.
  • a hearing instrument includes: an input for provision of an audio input signal representing sound output by a sound source; and a binaural filter for filtering the audio input signal, and configured to output a right ear signal for a right ear of a user of the hearing instrument and a left ear signal for a left ear of the user; wherein the binaural filter comprises an individual HRTF, which is one of the individual HRTFs determined in a accordance with one or more of the methods described herein.
  • the hearing instrument is a binaural hearing aid.
  • a device includes: a sound generator; and a binaural filter for filtering an audio output signal of the sound generator into a right ear signal for a right ear of a user of the device and a left ear signal for a left ear of the user; wherein the binaural filter comprises an individual HRTF, which is one of the individual HRTFs determined in a accordance with one or more of the methods described herein.
  • Fig. 1 schematically illustrates a new fitting instrument 100 and its interconnections with the Internet 200 and a new BTE hearing aid 10 shown in its operating position with the BTE housing behind the ear, i.e. behind the pinna, of a user.
  • the fitting instrument 100 has a processor 110 that is configured for determining individual HRTFs of a user of the hearing aid 10 to be fitted, by obtaining approximate HRTFs , e.g. from a server (not shown) accessed through the Internet 200.
  • the processor 110 is further configured for determination of individual HRTFs or HRIRs by determination of deviation(s) of the measured one or more individual HRTF(s) or HRIR(s) with relation to the corresponding approximated HRTF(s) or HRIR(s), respectively, and subsequent determination of other HRTFs or HRIRs based on the corresponding approximate HTRFs or HRIRs and the determined deviation(s).
  • the fitting instrument 100 is further configured for transmission of some or all of the determined individual HRTFs and/or HRIRs to the hearing aid through a wireless interface 80.
  • the fitting instrument 100 may further be configured for storing some or all of the determined individual HRTFs and/or HRIRs on a remote server accessed through the Internet for subsequent retrieval, e.g. by the hand-held device, such as a smartphone.
  • the BTE hearing aid 10 has at least one BTE sound input transducer with a front microphone 82A and a rear microphone 84A for conversion of a sound signal into a microphone audio sound signal, optional pre-filters (not shown) for filtering the respective microphone audio sound signals, A/D converters (not shown) for conversion of the respective microphone audio sound signals into respective digital microphone audio sound signals 86, 88 that are input to a processor 90 configured to generate a hearing loss compensated output signal 92 based on the input digital audio sound signals 86, 88.
  • the illustrated BTE hearing aid further has a memory for storage of right ear parts of individual HRIRs of the user determined by the fitting instrument and transmitted to the hearing aid.
  • the processor is further configured for selection of a right ear part of a HRIR for convolution with an audio sound signal input to the processor so that the user perceives the audio sound signal to arrive from a virtual sound source position at a distance and in a direction corresponding to the selected HRIR, provided that similar processing takes place at the left ear.
  • Fig. 2 shows a virtual sound source 20 positioned in a head reference coordinate system 22 that is defined with its centre 24 located at the centre of the user's head 26, which is defined as the midpoint 24 of a line 28 drawn between the respective centres of the eardrums (not shown) of the left and right ears 30, 32 of the user.
  • the x-axis 34 of the head reference coordinate system 22 is pointing ahead through a centre of the nose 36 of the user, its y-axis 38 is pointing towards the left ear 33 through the centre of the left eardrum (not shown), and its z-axis 40 is pointing upwards.
  • a line 42 is drawn through the centre 24 of the coordinate system 22 and the virtual sound source 20 and projected onto the XY-plane as line 44.
  • Azimuth ⁇ is the angle between line 44 and the X-axis 34.
  • the X-axis 34 also indicates the forward looking direction of the user.
  • Azimuth ⁇ is positive for negative values of the ⁇ -coordinate of the virtual sound source 20, and azimuth ⁇ is negative for positive values of the ⁇ -coordinate of the virtual sound source 20.
  • Elevation ⁇ is the angle between line 42 and the XY-plane. Elevation ⁇ is positive for positive values of the z-coordinate of the virtual sound source 20, and elevation ⁇ is negative for negative values of the z-coordinate of the virtual sound source 20.
  • Distance d is the distance between the virtual sound source 20 and the centre 24 of the user's head 26.
  • the illustrated new fitting instrument 100 is configured for measurement of individual HRTFs by measurement of sound pressures at the closed entrances to the left and right ear canals, respectively, of the user.
  • WO 95/23493 A1 discloses determination of HRTFs and HRIRs that constitute good approximations to individual HRTFs of a number of humans.
  • the HRTFs and HRIRs are determined at the entrances to the ear closed canals; see Figs. 5 and 6 of WO 95/23493 A1 .
  • Examples of individual HRTFs and HRIRs for various values of azimuth ⁇ and elevation ⁇ are shown in Fig. 1 of WO 95/23493 A1 .
  • the illustrated fitting instrument 100 has a processor that is configured for determining individual HRTFs of a user of the hearing aid 10 to be fitted, by accessing a remote server (not shown) through the Internet 200 to retrieve approximate HRTFs stored on a memory of the server and e.g. obtained as disclosed in WO 95/23493 A1 , however with 2° intervals.
  • the processor is configured for determination of the corresponding impulse response h d individual .
  • the determined h d individual is compared to the corresponding approximate impulse response h d app .
  • a large number of individual HRTFs is provided without individual measurement of each of the individual HRTFs; rather measurement of a single or a few individual HRTFs is sufficient so that the set of individual HRTFs can be provided without discomfort to the intended user of the hearing aid.
  • Fig. 3 shows a hearing system 50 with a binaural hearing aid 52A, 52B and a hand-held device 54.
  • the illustrated hearing system 50 uses speech syntheses to issue messages and instructions to the user and speech recognition is used to receive spoken commands from the user.
  • the illustrated hearing system 50 comprises a binaural hearing aid 52A, 52B comprising electronic components including two receivers 56A, 56B for emission of sound towards the ears of the user (not shown), when the binaural hearing aid 52A, 52B is worn by the user in its intended operational position on the user's head.
  • the binaural hearing aid 52A, 52B shown in Fig. 3 may be substituted with another hearing instrument of any known type including an Ear-Hook, In-Ear, On-Ear, Over-the-Ear, Behind-the-Neck, Helmet, Headguard, etc, headset, headphone, earphone, ear defenders, earmuffs, etc.
  • the illustrated binaural hearing aid 52A, 52B may be of any type of hearing aid, such as a BTE, a RIE, an ITE, an ITC, a CIC, etc, binaural hearing aid.
  • the illustrated binaural hearing aid may also be substituted by a single monaural hearing aid worn at one of the ears of the user, in which case sound at the other ear will be natural sound inherently containing the characteristics of the user's individual HRTFs.
  • the illustrated binaural hearing aid 52A, 52B has a user interface (not shown), e.g. with push buttons and dials as is well-known from conventional hearing aids, for user control and adjustment of the binaural hearing aid 52A, 52B and possibly the hand-held device 54 interconnected with the binaural hearing aid 52A, 52B, e.g. for selection of media to be played back.
  • a user interface e.g. with push buttons and dials as is well-known from conventional hearing aids, for user control and adjustment of the binaural hearing aid 52A, 52B and possibly the hand-held device 54 interconnected with the binaural hearing aid 52A, 52B, e.g. for selection of media to be played back.
  • the microphones of binaural hearing aid 52A, 52B may be used for reception of spoken commands by the user transmitted (not shown) to the hand-held device 54 for speech recognition in a processor 58 of the hand-held device 54, i.e. decoding of the spoken commands, and for controlling the hearing system 50 to perform actions defined by respective spoken commands.
  • the hand-held device 54 filters the output of a sound generator 60 of the hand-held device 54 with a binaural filter 63, i.e. a pair of filters 62A, 62B, with a selected HRTF into two output audio signals, one for the left ear and one for the right ear, corresponding to the filtering of the HRTF of a selected direction.
  • This filtering process causes sound reproduced by the binaural hearing aid 50 to be perceived by the user as coming from a virtual sound source localized outside the head from a direction corresponding to the HRTF in question.
  • the sound generator 60 may output audio signals representing any type of sound suitable for this purpose, such as speech, e.g. from an audio book, radio, etc, music, tone sequences, etc.
  • the user may for example decide to listen to a radio station while walking, and the sound generator 60 generates audio signals reproducing the signals originating from the desired radio station filtered by binaural filter 63, i.e. filter pair 62A, 62B, with the HRTFs in question, so that the user perceives to hear the desired music from the direction corresponding to the selected HRTFs.
  • binaural filter 63 i.e. filter pair 62A, 62B
  • the illustrated hand-held device 54 may be a smartphone with a GPS-unit 66 and a mobile telephone interface 68 and a WiFi interface 80.
  • Fig. 4 is a flowchart of the new method comprising the steps of:

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)
EP13175052.3A 2013-07-04 2013-07-04 Bestimmung von Individuellen HRTFs Active EP2822301B1 (de)

Priority Applications (5)

Application Number Priority Date Filing Date Title
DK13175052.3T DK2822301T3 (da) 2013-07-04 2013-07-04 Bestemmelse af individuelle HRTF
EP13175052.3A EP2822301B1 (de) 2013-07-04 2013-07-04 Bestimmung von Individuellen HRTFs
US13/949,134 US9426589B2 (en) 2013-07-04 2013-07-23 Determination of individual HRTFs
JP2014137176A JP5894634B2 (ja) 2013-07-04 2014-07-02 個人ごとのｈｒｔｆの決定
CN201410317428.9A CN104284286B (zh) 2013-07-04 2014-07-04 个体hrtf的确定

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP13175052.3A EP2822301B1 (de) 2013-07-04 2013-07-04 Bestimmung von Individuellen HRTFs

Publications (2)

Publication Number Publication Date
EP2822301A1 true EP2822301A1 (de) 2015-01-07
EP2822301B1 EP2822301B1 (de) 2019-06-19

Family

ID=48703330

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13175052.3A Active EP2822301B1 (de) 2013-07-04 2013-07-04 Bestimmung von Individuellen HRTFs

Country Status (2)

Country Link
EP (1) EP2822301B1 (de)
DK (1) DK2822301T3 (de)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016134982A1 (en) * 2015-02-26 2016-09-01 Universiteit Antwerpen Computer program and method of determining a personalized head-related transfer function and interaural time difference function
EP3509327A1 (de) * 2018-01-07 2019-07-10 Creative Technology Ltd. Verfahren zur erzeugung von personalisiertem raumklang mit kopfverfolgung
LU100981A1 (en) * 2018-11-07 2019-07-15 Technische Hochschule Koeln Wavefield processing method
GB2581785A (en) * 2019-02-22 2020-09-02 Sony Interactive Entertainment Inc Transfer function dataset generation system and method
EP4138418A1 (de) * 2021-08-20 2023-02-22 Oticon A/s Hörsystem mit einer datenbank mit akustischen übertragungsfunktionen

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1995023493A1 (en) 1994-02-25 1995-08-31 Moeller Henrik Binaural synthesis, head-related transfer functions, and uses thereof
US6181800B1 (en) * 1997-03-10 2001-01-30 Advanced Micro Devices, Inc. System and method for interactive approximation of a head transfer function
WO2007031905A1 (en) * 2005-09-13 2007-03-22 Koninklijke Philips Electronics N.V. Method of and device for generating and processing parameters representing hrtfs
WO2011045751A1 (en) * 2009-10-12 2011-04-21 Nokia Corporation Multi-way analysis for audio processing

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9031242B2 (en) * 2007-11-06 2015-05-12 Starkey Laboratories, Inc. Simulated surround sound hearing aid fitting system
US8885834B2 (en) * 2008-03-07 2014-11-11 Sennheiser Electronic Gmbh & Co. Kg Methods and devices for reproducing surround audio signals

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1995023493A1 (en) 1994-02-25 1995-08-31 Moeller Henrik Binaural synthesis, head-related transfer functions, and uses thereof
US6181800B1 (en) * 1997-03-10 2001-01-30 Advanced Micro Devices, Inc. System and method for interactive approximation of a head transfer function
WO2007031905A1 (en) * 2005-09-13 2007-03-22 Koninklijke Philips Electronics N.V. Method of and device for generating and processing parameters representing hrtfs
WO2011045751A1 (en) * 2009-10-12 2011-04-21 Nokia Corporation Multi-way analysis for audio processing

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ALGAZI V R ET AL: "The CIPIC HRTF database", APPLICATIONIS OF SIGNAL PROCESSING TO AUDIO AND ACOUSTICS, 2001 IEEE W ORKSHOP ON THE OCT. 21-24, 2001, PISCATAWAY, NJ, USA,IEEE, 21 October 2001 (2001-10-21), pages 99 - 102, XP010566884, ISBN: 978-0-7803-7126-2 *
XIE BO-SUN: "Recovery of individual head-related transfer functions from a small set of measurements", THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, AMERICAN INSTITUTE OF PHYSICS FOR THE ACOUSTICAL SOCIETY OF AMERICA, NEW YORK, NY, US, vol. 132, no. 1, 1 July 2012 (2012-07-01), pages 282 - 294, XP012163090, ISSN: 0001-4966, [retrieved on 20120710], DOI: 10.1121/1.4728168 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107409266A (zh) * 2015-02-26 2017-11-28 安特卫普大学 确定个体化头部相关传输函数和耳间时间差函数的计算机程序和方法
US10257630B2 (en) 2015-02-26 2019-04-09 Universiteit Antwerpen Computer program and method of determining a personalized head-related transfer function and interaural time difference function
WO2016134982A1 (en) * 2015-02-26 2016-09-01 Universiteit Antwerpen Computer program and method of determining a personalized head-related transfer function and interaural time difference function
CN107409266B (zh) * 2015-02-26 2020-09-04 安特卫普大学 确定个体化头部相关传输函数和耳间时间差函数的方法
TWI797230B (zh) * 2018-01-07 2023-04-01 新加坡商創新科技有限公司 用於以頭部追蹤產生客製化空間音訊的方法
EP3509327A1 (de) * 2018-01-07 2019-07-10 Creative Technology Ltd. Verfahren zur erzeugung von personalisiertem raumklang mit kopfverfolgung
CN110021306A (zh) * 2018-01-07 2019-07-16 创新科技有限公司 用于利用头部跟踪生成自定义空间音频的方法
US12022277B2 (en) 2018-01-07 2024-06-25 Creative Technology Ltd Method for generating customized spatial audio with head tracking
CN110021306B (zh) * 2018-01-07 2023-12-12 创新科技有限公司 用于利用头部跟踪生成自定义空间音频的方法
US11785412B2 (en) 2018-01-07 2023-10-10 Creative Technology Ltd. Method for generating customized spatial audio with head tracking
LU100981A1 (en) * 2018-11-07 2019-07-15 Technische Hochschule Koeln Wavefield processing method
GB2581785B (en) * 2019-02-22 2023-08-02 Sony Interactive Entertainment Inc Transfer function dataset generation system and method
US10999694B2 (en) 2019-02-22 2021-05-04 Sony Interactive Entertainment Inc. Transfer function dataset generation system and method
GB2581785A (en) * 2019-02-22 2020-09-02 Sony Interactive Entertainment Inc Transfer function dataset generation system and method
EP4138418A1 (de) * 2021-08-20 2023-02-22 Oticon A/s Hörsystem mit einer datenbank mit akustischen übertragungsfunktionen
US12063477B2 (en) 2021-08-20 2024-08-13 Oticon A/S Hearing system comprising a database of acoustic transfer functions

Also Published As

Publication number Publication date
DK2822301T3 (da) 2019-07-01
EP2822301B1 (de) 2019-06-19

Similar Documents

Publication Publication Date Title
US9426589B2 (en) Determination of individual HRTFs
US9930456B2 (en) Method and apparatus for localization of streaming sources in hearing assistance system
US9031242B2 (en) Simulated surround sound hearing aid fitting system
CN105530580B (zh) 听力系统
CN104412618B (zh) 用于助听器的方法
US9307331B2 (en) Hearing device with selectable perceived spatial positioning of sound sources
JP3805786B2 (ja) バイノーラル信号合成と頭部伝達関数とその利用
US10425747B2 (en) Hearing aid with spatial signal enhancement
Ranjan et al. Natural listening over headphones in augmented reality using adaptive filtering techniques
EP3468228B1 (de) Binaurales hörsystem mit lokalisierung von schallquellen
JP2002209300A (ja) 音像定位装置、並びに音像定位装置を用いた会議装置、携帯電話機、音声再生装置、音声記録装置、情報端末装置、ゲーム機、通信および放送システム
US11405723B2 (en) Method and apparatus for processing an audio signal based on equalization filter
EP2822301B1 (de) Bestimmung von Individuellen HRTFs
DK201370793A1 (en) A hearing aid system with selectable perceived spatial positioning of sound sources
Kuk et al. Evaluation of a pinna compensation algorithm for sound localization and speech perception in noise
EP2806661A1 (de) Hörgerät mit räumlicher Signalverstärkung
DK2887695T3 (en) A hearing aid system with selectable perceived spatial location of audio sources
US20070127750A1 (en) Hearing device with virtual sound source
Engel et al. Long-term user adaptation to an audio augmented reality system
US11218832B2 (en) System for modelling acoustic transfer functions and reproducing three-dimensional sound
Prawda et al. Augmented Reality: Hear-through
Taddei Best Hearing Aids in Background Noise of 2023: OTC & Rx Options

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130704

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

R17P Request for examination filed (corrected)

Effective date: 20150707

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

D17P Request for examination filed (deleted)
R17P Request for examination filed (corrected)

Effective date: 20150707

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20161214

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: GN HEARING A/S

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602013056717

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04S0001000000

Ipc: H04R0025000000

RIC1 Information provided on ipc code assigned before grant

Ipc: H04S 7/00 20060101ALN20181128BHEP

Ipc: H04R 25/00 20060101AFI20181128BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04S 7/00 20060101ALN20181130BHEP

Ipc: H04R 25/00 20060101AFI20181130BHEP

INTG Intention to grant announced

Effective date: 20190108

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20190624

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013056717

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1147045

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190715

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190919

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190920

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1147045

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191021

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191019

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200224

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190731

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190704

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013056717

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG2D Information on lapse in contracting state deleted

Ref country code: IS

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190704

26N No opposition filed

Effective date: 20200603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130704

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210716

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20210722

Year of fee payment: 9

Ref country code: DK

Payment date: 20210716

Year of fee payment: 9

Ref country code: CH

Payment date: 20210709

Year of fee payment: 9

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190619

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20220731

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230719

Year of fee payment: 11