EP3618458B1 - Hearing device with position data, audio system and related methods - Google Patents
Hearing device with position data, audio system and related methods Download PDFInfo
- Publication number
- EP3618458B1 EP3618458B1 EP19197521.8A EP19197521A EP3618458B1 EP 3618458 B1 EP3618458 B1 EP 3618458B1 EP 19197521 A EP19197521 A EP 19197521A EP 3618458 B1 EP3618458 B1 EP 3618458B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- hearing device
- audio
- input signal
- position data
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 44
- 238000012545 processing Methods 0.000 claims description 97
- 230000033001 locomotion Effects 0.000 claims description 22
- 230000008859 change Effects 0.000 claims description 20
- 238000001514 detection method Methods 0.000 claims description 10
- 230000000977 initiatory effect Effects 0.000 claims description 7
- 206010011878 Deafness Diseases 0.000 claims description 5
- 230000010370 hearing loss Effects 0.000 claims description 5
- 231100000888 hearing loss Toxicity 0.000 claims description 5
- 208000016354 hearing loss disease Diseases 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000005236 sound signal Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 210000005069 ears Anatomy 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 238000006722 reduction reaction Methods 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils
Definitions
- the present disclosure relates to a method of operating a hearing device, and a hearing device.
- the present disclosure relates to methods and apparatus for improving the correspondence of the visual and the auditory cues of a hearing aid user in a crowded environment.
- EP2584794 A1 discloses a binaural listening system comprising first and second listening devices adapted for being located at or in left and right ears, respectively, of a user, the binaural listening system being adapted for receiving a wirelessly transmitted signal comprising a target signal and an acoustically propagated signal comprising the target signal as modified by respective first and second acoustic propagation paths from an audio source to the first and second listening devices.
- a method and an audio processing system Disclosed are also a method and an audio processing system. The object is to provide a scheme for providing spatial information to an audio signal streamed to a pair of listening devices of a binaural listening system.
- the first and second listening devices each comprise an alignment unit for aligning the first and second streamed target audio signals with the first and second propagated electric signals in the first and second listening devices, respectively, to provide first and second aligned streamed target audio signals in the first and second listening devices, respectively.
- the use is e.g. in applications comprising simultaneous acoustic propagation and wireless transmission of an audio signal to an audio receiving device.
- US6845338 B1 discloses a telemetric contextually based spatial audio system integrated into a mobile terminal including a headset and antenna.
- the headset includes an electronic compass which determines head orientation data for the user.
- the terminal's geographical position can either be determined locally (e.g. onboard GPS) or by network triangulation.
- the orientation and position data are sent to a server which overlays the position and head orientation data onto a locally stored "floor plan", describing an object(s) of interest.
- the floor plan relates the user's viewpoint to a database of contextual audio clips.
- a regionally significant portion of the database and respective triggers are sent to the terminal.
- the terminal using the database information recalculates which audio clip(s) to play.
- the terminal then convolves the respective audio clip(s) via an impulse response function so as to be perceived by the user as emanating from the exact spatial position of the object specified by the system.
- the hearing device comprises a first antenna, a first transceiver coupled to the first antenna, an acoustic output transducer, a microphone, and a primary processing unit, the primary processing unit being coupled to the first transceiver, the acoustic output transducer, and the microphone.
- the method comprises pairing the hearing device with an audio system; initiating detection of hearing device motion; sending position data indicative of hearing device position to the audio system, if a position change criterion is fulfilled; wirelessly receiving a first audio stream based on the position data of the hearing device; converting the first audio stream to a first wireless input signal; mixing the first wireless input signal with an audio input signal from the microphone to form an input signal; processing the input signal to form an output signal; and converting the output signal to an audio output signal.
- the hearing device comprises a first antenna; a first transceiver coupled to the first antenna and configured to wirelessly receive one or more audio streams from an audio system, the one or more audio streams including a first audio stream, wherein the hearing device is configured for pairing the hearing device with the audio system; a microphone; a primary processing unit adapted for processing an input signal according to a hearing prescription in order to alleviate a hearing loss; an acoustic output transducer coupled to an output of the primary processing unit for conversion of an output signal from the primary processing unit into an audio output signal; a secondary processing unit with a first input and a second input, wherein the first input is coupled to an output of the first transceiver for receiving a first wireless input signal representative of the first audio stream from the first transceiver, and the second input is coupled to an output of the microphone for receiving an audio input signal from the microphone, wherein the secondary processing unit is configured to mix the first wireless input signal and the audio input signal to form an input signal on an output of the secondary processing unit, where
- the audio stream received in the hearing device is based on the position (direction and/or distance) of the hearing device, thereby improving the user experience of the hearing device.
- the position (distance and/or orientation) of the hearing device may provide prioritization of audio signals.
- the disclosed hearing device and method improve the correspondence of the visual and the auditory cues of the listener, otherwise lost in crowd amplification.
- the primary processing unit and/or the secondary processing unit of the hearing device may be configured to compensate for hearing loss or hearing disability of the user of the hearing device.
- the primary processing unit and/or the secondary processing unit of the hearing device may be configured to alter a received audio stream, e.g. the first audio stream, a wireless input signal, e.g. the first wireless input signal, and/or the input signal to compensate for hearing loss or hearing disability of the user of the hearing device.
- the hearing device comprises a secondary processing unit and a primary processing unit.
- the secondary processing unit and the primary processing unit may be embedded as a single processing unit performing the tasks of both the secondary processing unit and the primary processing unit.
- the present invention relates to a hearing device, audio system and methods enabling adaptation of signal processing in the hearing device based on position data of the hearing device
- one or more transceivers or transmitter units comprising transceivers are configured for broadcasting or transmitting audio track(s) and/or audio stream(s), e.g. to one or more hearing devices.
- the audio system may comprise a first transceiver and optionally a second transceiver.
- a transceiver may transmit one or more audio streams.
- a transceiver may be identified by a transceiver address.
- a transmitter unit may be configured to wirelessly transmit audio track(s) and/or audio stream(s).
- An audio track is an audible signal.
- Exemplary audible signals are speech, music, an airport call, a movie sound track or a combination thereof.
- An audio stream is an electromagnetic signal representative of an audio track.
- An audio stream is represented by a number of audio blocks each comprising one or more audio packets.
- An audio packet comprises a finite part of an audio stream.
- An audio packet may comprise an audio stream identifier, which may comprise one or more elements such as audio track identification, transceiver address or identification, audio group identification, audio source identification, etc.
- Audio track identification may enable selection of audio streams representative of the same audio track
- audio group identification may identify a certain group of audio tracks or audio streams e.g. an audio group may be audio tracks in the same or similar language, e.g. English, Danish or Chinese.
- An audio packet may comprise transceiver address of the transceiver transmitting the audio packet, thus enabling the hearing device to sort out or separate audio packets sent from other transceivers.
- Transmitting and receiving the wireless audio stream(s) may be achieved by using wireless technology, thus transmitting/receiving audio blocks at one or more frequencies, e.g. in the range from 2.4 GHz to 2.5 GHz, in the range from 800 MHz to 1 GHz, in the range from 3.6 GHz to 3.7 GHz, and/or in the range from 4.9 GHz to 5.9 GHz.
- frequencies e.g. in the range from 2.4 GHz to 2.5 GHz, in the range from 800 MHz to 1 GHz, in the range from 3.6 GHz to 3.7 GHz, and/or in the range from 4.9 GHz to 5.9 GHz.
- the primary processing unit may be configured to control the first transceiver.
- the first transceiver and/or the second transceiver may be configured to receive audio streams at frequencies in the range from 2.4 GHz to 2.5 GHz, in the range from 800 MHz to 1 GHz, in the range from 3.6 GHz to 3.7 GHz, and/or in the range from 4.9 GHz to 5.9 GHz.
- the first transceiver and/or the second transceiver may be configured to receive audio streams at frequencies in the range from 169 MHz to 218 MHz and/or in the range from 480 MHz to 520 MHz.
- Pairing of the hearing device with the audio system may comprise transmitting hearing device information from the hearing device to the audio system.
- the hearing device information may comprise one or more of hearing device idenfication, hearing device manufacturer, hearing device model, and/or hearing device configuration data. Pairing of the hearing device with the audio system may additionally or alternatively comprise transmitting audio system information from the audio system to the hearing device and/or receiving audio system information from the audio system.
- the audio system information may comprise one or more of audio system identification, audio source identification(s), transceiver identification(s) and/or audio stream configuration data.
- the audio system may be configured to adapt the format of the first audio stream to be sent to the hearing device based on hearing device information received from the hearing device, e.g. model and/or manufacturer, data encoding format etc. Furthermore, pairing of the hearing device with the audio system may comprise exchanging an encryption key, such as to enable encryption of an audio stream, such as the first audio stream, before sending from the audio system, and decryption of the received wireless input signal, e.g. the first wireless input signal, in the hearing device.
- an encryption key such as to enable encryption of an audio stream, such as the first audio stream
- decryption of the received wireless input signal e.g. the first wireless input signal
- Pairing of the hearing device with the audio system may initiate sending of an audio stream, e.g. the first audio stream, from the audio system.
- Pairing of the hearing device with the audio system may cause the hearing device to inititate receiving a wireless input signal, e.g. the first wireless input signal, which is sent by the audio system.
- a wireless input signal e.g. the first wireless input signal
- the position controller may be coupled to the primary processing unit for receiving and/or sending position control signal(s) from/to the primary processing unit.
- the hearing device may comprise a second transceiver and/or a second antenna.
- the second transceiver and/or the second antenna may be coupled to the position controller.
- the second transceiver and/or the second antenna may be coupled to the primary processing unit.
- Pairing of the hearing device with the audio system may be performed via the first transceiver and/or the second transceiver of the hearing device.
- the hearing device sends position data indicative of hearing device position to the audio system or device(s) in the audio system.
- the hearing device may send position data indicative of hearing device position to the audio system via another hearing device or a peripheral or external device, such as a smartphone or a tablet computer.
- the first transceiver and/or the second transceiver may be configured to transmit and/or receive wireless signals from/to one or more audio systems or other external devices, such as other hearing devices, mobile phones, or tablet computers.
- the hearing device may send position data indicative of hearing device position to the audio system with the first transceiver and/or the second transceiver.
- the position controller of the hearing device may comprise a motion detector and optionally a position estimator connected to the motion detector.
- the position estimator may be configured to estimate position data of the hearing device, e.g. based on motion detector output from a motion detector.
- the motion detector may comprise a plurality of sensors e.g. one or more gyroscopes and/or one or more accelerometers.
- Position data of the hearing device and/or position data from audio source(s) may be sent continuously or regularly with a fixed time interval, e.g. every 100 ms, or every 500 ms, or every 1000 ms.
- position data may be sent to the audio system if a position change criterion is fulfilled.
- the hearing device may detect that the position is changing, and when the hearing device detects that the position is changing, the new position is sent to the first external device.
- the change criterion may e.g. be a change in the azimuth angle larger than a change threshold.
- the change threshold may be more than 0.5 degrees, or more than 1 degree, or more than 2 degrees, or more than 5 degrees.
- the position data may be sent to the audio system if the position change criterion is fulfilled and at least by a fixed time interval, e.g. every 100 ms, or every 500 ms, or every 1000 ms. Thereby, the position data are updated regularly, but more frequently by a detection of change.
- the audio system may receive position data correspondingly to the sending of position data from the hearing device.
- Initiating detection of hearing device motion may comprise setting up a reference coordinate system.
- Initiating detection of hearing device motion may comprise a calibration procedure where information of an initial alignment of the hearing device and the audio source(s) are determined.
- the calibration procedure may comprise setting up a reference coordinate system.
- the setting up of a reference coordinate system may comprise registering absolute position data of the hearing device and/or the audio source(s), e.g. while the user of the hearing device is looking in a predetermined (reference) direction and/or at a given (reference) distance, such as looking at a specific audio source or other reference at a predetermined distance.
- the reference coordinate system may have an origin at a specific point in relation to an audio source and/or the hearing device.
- the position data may comprise angular position data of the hearing device.
- the position data may comprise information of the azimuth angle of the hearing device.
- the azimuth angle may be indicative of the looking direction of the user wearing the hearing device.
- the position data may comprise information of the altitude angle.
- the position data may comprise distance position data of the hearing device.
- the position data may comprise information of the distance of the hearing device relative to an audio source and/or a reference point.
- the position data may be sent via the first transceiver and/or the first antenna of the hearing device.
- the position data may be sent via a second transceiver of the hearing device.
- the hearing device may comprise a second antenna.
- the second transceiver may be coupled to the second antenna.
- the audio system comprises one or more audio sources including the first audio source and optionally a second audio source.
- the audio system may comprise a third audio source and/or a fourth audio source.
- the first audio source may comprise a first microphone.
- the second audio source may comprise a second microphone.
- the audio system may comprise a central unit with the processor and the first transceiver. The audio source(s) may be wired and/or wirelessly coupled to the central unit for feeding input signals to the processor.
- the method of operating an audio system may comprise receiving a plurality of input signals from respective plurality of audio sources, such as receiving a first input signal from the first audio source, and/or receiving a second input signal from the second audio source.
- the method of operating an audio system may further comprise processing input signals from the plurality of audio sources, e.g. the first audio source and the second audio source, i.e. the first input signal and the second input signal. Processing of the input signals may be based on position data of the hearing device to form the first wireless output signal.
- Processing the input signal(s) based on position data of the hearing device may comprise determining one or more gains, including a first gain for the first input signal and/or a second gain for the second input signal, based on position data of the hearing device, and applying the gain(s) to respective input signal(s) from audio source(s).
- the method of operating the audio system may comprise obtaining position data of one or more audio source(s), e.g. the first audio source and/or the second audio source.
- Position data of audio source(s) may be obtained during an audio system setup procedure. This is in particular useful, when the audio sources are stationary.
- Processing of input signal(s) may be based on position data of the audio source(s). Processing of the first input signal may be based on position data of the first audio source and/or processing of the second input signal may be based on position data of the second audio source. For example, a movement detected by the hearing device may be caused by the user looking at a moving audio source, which preferably should not result in a change of the audio received in the hearing device. Processing of an input signal, such as the first input signal and/or the second input signal, to form the first wireless output signal may be based both on position data of the respective audio source(s), such as the first audio source and/or the second audio source, and the position data of the hearing device.
- the hearing device may be a binaural hearing device system comprising a first hearing device and a second hearing device.
- Processing the input signal(s) may comprise processing the input signal(s) based on position data of the hearing device to form a first primary wireless output signal and a first secondary wireless output signal.
- the sending of a first audio stream to the hearing device may comprise sending a first primary audio stream to the first hearing device and sending a first secondary audio stream to the second hearing device.
- the first primary wireless output signal and the first secondary wireless output may at least in part be based on a head-related-transfer-function (HRTF).
- HRTF head-related-transfer-function
- Processing the first input signal may comprise applying a first gain G 1 to the first input signal.
- the first gain G 1 may be applied to the first input signal to form the first wireless output signal.
- processing of the input signals from the plurality of audio sources may comprise applying a vector of gains.
- the vector of gains may comprise a gain for each of the input signals.
- P HD position data for the hearing device
- P AS is position data for an audio source.
- P AS may be a vector with position data of each audio source.
- Processing the first input signal may comprise setting the first gain to a first value if a first field of listening (FIL) criterion is fulfilled, i.e. if the position of the hearing aid indicates that the hearing aid user wishes to listen to the first audio source.
- Processing the first input signal may comprise setting the first gain to a second value if the first FIL criterion is not fulfilled. The first value may be larger than the second value.
- FIL field of listening
- the first FIL criterion may be based on the first view angle being the angle between the users direction of view and the direction from the hearing device to the first audio source.
- the first FIL criterion may be given by: ⁇ 1 ⁇ T 1,1 , where ⁇ 1 is the first view angle and T 1,1 is a first threshold angle for the first audio source.
- the first threshold angle T 1,1 may be in the range from 15 to 60 degrees.
- Processing the second input signal may comprise setting the second gain to a first value if a second field of listening (FIL) criterion is fulfilled, i.e. if the position of the hearing aid indicates that the hearing aid user wishes to listen to the second audio source.
- Processing the second input signal may comprise setting the second gain to a second value if the second FIL criterion is not fulfilled.
- the first value may be larger than the second value.
- the second FIL criterion may be based on the second view angle being the angle between the users direction of view and the direction from the hearing device to the second audio source.
- the second FIL criterion may be given by: ⁇ 2 ⁇ T 2,1 , where ⁇ 2 is the second view angle and T 2,1 is a first threshold angle for the second audio source.
- the first threshold angle T 2,1 may be in the range from 15 to 60 degrees.
- Fig. 1 schematically illustrates a hearing device.
- the hearing device 2 comprises a first antenna 4, a first transceiver 6, a primary processing unit 8, an acoustic output transducer 10, a position controller 12, a microphone 14, and a secondary processing unit 16.
- the hearing device 2 is configured to pair with an audio system including one or more audio sources. For example, pairing of the hearing device 2 with the one or more audio sources is achieved by pairing the hearing device 2 with the audio system.
- the first transceiver 6 is coupled to the first antenna 4 and configured to wirelessly receive one or more audio streams including a first audio stream 54 from the audio system.
- the one or more audio streams are converted to respective wireless input signals, e.g. the first audio stream 54 is converted to a first wireless input signal 56.
- the secondary processing unit 16 has a first input 18 and a second input 20.
- the first input 18 is coupled to an output of the first transceiver 6.
- the second input 20 is coupled to an output of the microphone 14.
- the secondary processing unit 16 receives the first wireless input signal 56 from the first transceiver 6 on the first input 18, and receives the audio input signal 58 from the microphone 14 on the second input 20.
- the secondary processing unit 16 is configured to mix the first wireless input signal 56 and the audio input signal 58 to form the input signal 60 on an output of the secondary processing unit 16.
- the primary processing unit 8 processes the hearing device input signal 60 to form an output signal 62.
- the primary processing unit 8 has an input coupled to the output of the secondary processing unit 16.
- the primary processing unit 8 may perform signal processing such as noise reduction, filtering, amplification, etc. of the input signal 60.
- the primary processing unit 8 may perform signal processing of the input signal 60 to account for hearing disability of the user of the hearing device 2.
- the secondary processing unit 16 and the primary processing unit 8 may be embedded as a single processing unit as indicated with the dotted box in Fig. 1
- the acoustic output transducer 10 is coupled to an output of the primary processing unit 8 and receives an output signal 62 of the primary processing unit and converts the output signal 62 into an audio output signal 64.
- the audio output signal 64 is perceptible by the user of the hearing device 2.
- the position controller 12 is configured for detecting and sending position data of the hearing device 2 to an audio system.
- the position controller 12 comprises a motion detector 22 and optionally a position estimator 24 configured for estimating position data of the hearing device based on an output of the motion detector 22.
- motion detector output may be coupled directly to the transceiver 6.
- the position data of the hearing device 2 may be sent to the audio system via a first external device.
- the position controller 12 transmits position data output signal 66 indicative of the position data to the first transceiver 6.
- the first transceiver 6 sends a wireless position data signal 68 based on the position data signal 66 via the first antenna 4 to the audio system either directly or via a first external device.
- the wireless position data signal 68 may be sent to the audio system via a second transceiver and/or a second antenna in the hearing device.
- Fig. 2 schematically illustrates an exemplary audio system 30.
- the audio system 30 comprises a first audio source 32, a processing unit 38, and a first transceiver 36 coupled to a first antenna 34.
- the processing unit 38 is coupled to the first transceiver 36.
- the first audio source 32 comprises a microphone and is connected to the processing unit 38.
- the audio system 30 receives a first input signal 50 from the first audio source 32.
- the first input signal 50 is processed to form a first wireless output signal 52.
- a first audio stream 54 is sent to the hearing device 2 based on the first wireless output signal 52.
- the first audio stream 54 send by the audio system corresponds to the audio stream 54 received by the hearing device as illustrated in Fig. 1 .
- the audio system 30 receives wireless position data signal 68 from the hearing device 2, the wireless position data signal 68 is indicative of the position of the hearing device 2.
- the wireless position data signal 68 is received by the first transceiver 36 via the first antenna 34.
- the first transceiver transmits a position data input signal 70 based on the wireless position data signal 68 to the processing unit 38 of the audio system 30.
- the processing of the input signal 50 is based on the position data of the hearing device 2.
- the first wireless output signal 52 is adapted to the position of the hearing device 2.
- the wireless position data signal 68 may be received by a second transceiver and/or a second antenna in the audio system.
- Fig. 3 schematically illustrates an exemplary audio system 30' and an exemplary hearing device system comprising a first hearing device 26 and a second hearing device 28.
- the first hearing device 26 and/or the second hearing device 28 may be a hearing device as disclosed herein, e.g. a hearing device 2.
- the audio system 30' comprises a first audio source 32 comprising a first microphone, and a second audio source 32' comprising a second microphone.
- the first audio source 32 and the second audio source 32' are wirelessly coupled to a central unit 33 with processor 38 via interface 35 for feeding first and second input signals to the processor.
- Processor 38 receives position data of one of or both hearing devices 26, 28 via second antenna 42 and second transceiver 44 in the audio system and processes the first and second input signals from audio sources 32, 32' based on the received hearing device position data to form a first primary wireless output signal and a first secondary wireless output signal.
- Hearing devices 26, 28 each comprises second antenna 42 and second transceiver 44 coupled to position controller 12 for sending position data to the audio system.
- the position controller 12 may be omitted in one of the hearing devices 26, 28.
- the first primary wireless output signal and the first secondary wireless output signal are sent to the first hearing device 26 and second hearing device 28, respectively, as a first primary audio stream to the first hearing device and a first secondary audio stream to the second hearing device via the first transceiver 36.
- the first primary wireless output signal and the first secondary wireless output signal may be multiplexed to a single first audio stream comprising first and second channels.
- Fig. 4 shows a flow diagram of a method 100 of operating a hearing device or a hearing device system.
- the method 100 comprises pairing 102 the hearing device with an audio system, initiating 104 detection of hearing device motion, sending 106 position data, receiving 108 audio stream(s), converting 110 audio stream(s) to wireless input signal(s), mixing 112 the wireless input signal(s) with audio input signal(s) to form input signal(s), processing 114 the input signal(s) to form output signal(s), and converting 116 the output signal(s) to audio output signal(s).
- Pairing 102 the hearing device with the audio system may comprise pairing the hearing device with one or more audio sources of the audio system. Pairing of the hearing device with the audio system 102 may further comprise exchanging hearing device identification data, audio system identification data and/or audio source identification data, between the hearing device and the audio system.
- Initiating 104 detection of hearing device motion may comprise activating motion detection in the hearing device.
- Sending 106 position data may comprise sending position data indicative of hearing device position to the audio system.
- Receiving 108 audio stream(s) may comprise receiving a first audio stream based on the position data of the hearing device and/or receiving a second audio stream based on the position data of the hearing device.
- Converting 110 audio stream(s) may comprise converting the first audio stream to a first wireless input signal and/or converting the second audio stream to a second wireless input signal.
- Mixing 112 of wireless input signal(s) with audio input signal(s) to form input signal(s) may comprise mixing the first wireless input signal with an audio input signal from a microphone of the hearing device to form an input signal.
- mixing 112 of wireless input signal(s) with audio input signal(s) to form input signal(s) may comprise mixing a plurality of wireless input signals, e.g. a first and a second wireless input signal, with one or more audio input signals, to form one or more input signals.
- Processing 114 the input signal(s) to form output signal(s) may comprise applying signal processing to the input signal(s) e.g. filtering, amplification, noise reduction and/or adjustments to compensate for hearing loss or hearing disability of the user of the hearing device.
- Processing 114 of the input signal(s) to form output signal(s) may comprise processing of a first input signal to form a first output signal.
- Converting 116 the output signal(s) to an audio output signal(s) may comprise converting a first output signal to a first audio output signal.
- the audio output signal(s) may be perceptible by the user of the hearing device.
- the method 100 may be looped, such that the hearing device is continuously sending position data 106, receiving audio stream(s) 108, converting audio stream(s) to wireless input signal(s) 110, mixing the wireless input signal(s) with audio input signal(s) to form input signal(s) 112, processing the input signal(s) to form output signal(s) 114, and converting the output signal(s) to audio output signal(s) 116. Further, different parts of the method may be run in parallel and/or sequentially.
- Fig. 5 shows a flow diagram of a method 100' of operating a hearing device.
- the method 100' comprises the same steps as the method 100 as shown in Fig. 4 .
- the method 100' illustrates the loop of the method, where the hearing device is continuously receiving audio stream(s) 108, converting audio stream(s) to wireless input signal(s) 110, mixing the wireless input signal(s) with audio input signal(s) to form input signal(s) 112, processing the input signal(s) to form output signal(s) 114, and converting the output signal(s) to audio output signal(s) 116.
- the method 100' further comprises a position change criterion 118. If the position change criterion is fulfilled, the method 100' returns to sending position data 106. If the position change criterion is not fulfilled the method 100' returns to the receiving of audio stream(s) 108.
- the position change criterion 118 determines if a position change large enough to justify resending of position data in 106 has occurred. Incorporation of a position change criterion 118 may improve battery life of the hearing device by reducing the amount of wireless data transmission from the transceiver of the hearing device to the audio system.
- Fig. 6 shows a flow diagram of an exemplary method 200 of operating an audio system which is presently not explicitly claimed.
- the method 200 comprise pairing with a hearing device 202, receiving position data of the hearing device 204, receiving input signal(s) from audio source(s) 206, processing input signal(s) based on position data to form wireless output signal(s) 208, sending audio stream(s) to the hearing device based on the wireless output signal(s) 210.
- Pairing 202 with a hearing device may comprise pairing the hearing device with one or more audio sources of the audio system. Pairing 202 with the hearing device may further comprise exchanging hearing device identification data, audio system identification data and/or audio source identification data, between the hearing device and the audio system.
- Receiving 204 position data of the hearing device may comprise continuously receiving updated position data.
- receiving 204 position data of the hearing device may comprise receiving position data if the position data has changed. The position data received is indicative of the hearing device position.
- Receiving 206 input signal(s) from audio source(s) may include receiving a first input signal from a first audio source and/or receiving a second input signal from a second audio source.
- Processing 208 the input signal(s) based on position data to form wireless output signal(s) may include processing of a first input signal based on position data to form a first wireless output signal and/or processing of a second input signal based on position data to form a second wireless output signal. Alternatively, a plurality of input signals may be processed based on position data to form a first wireless output signal.
- Sending 210 audio stream(s) to the hearing device based on wireless output signal(s) may include sending a first audio stream to the hearing device based on a first wireless output signal and/or sending a second audio stream to the hearing device based on a second wireless output signal.
- the method 200 comprises obtaining 212 position data of audio source(s), e.g. between pairing 202 with a hearing device and receiving 204 position data of the hearing device 204.
- the audio source(s) may be movable, and thus it may be beneficial to obtain or detect position data of the audio source(s) 212, and include the position data of the audio sources in the processing of the input signal(s) 208.
- the processing 208 of the input signal(s) to form wireless output signal(s) may be based on position data from the hearing device and/or position data from the audio source(s).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Circuit For Audible Band Transducer (AREA)
- Stereophonic System (AREA)
Description
- The present disclosure relates to a method of operating a hearing device, and a hearing device. In particular, the present disclosure relates to methods and apparatus for improving the correspondence of the visual and the auditory cues of a hearing aid user in a crowded environment.
- It is known to stream audio to hearing devices via a telecoil solution having a limited bandwidth with limited possibilities of separating different audio streams. Further, wireless communication to and from hearing devices has been increasing in continuation of the developments within wireless communication technology. In an environment where audio is distributed to the crowd via a number of external microphones or other audio sources the present technology provides limited possibilities of separating audio signals from different positions.
-
EP2584794 A1 discloses a binaural listening system comprising first and second listening devices adapted for being located at or in left and right ears, respectively, of a user, the binaural listening system being adapted for receiving a wirelessly transmitted signal comprising a target signal and an acoustically propagated signal comprising the target signal as modified by respective first and second acoustic propagation paths from an audio source to the first and second listening devices. Disclosed are also a method and an audio processing system. The object is to provide a scheme for providing spatial information to an audio signal streamed to a pair of listening devices of a binaural listening system. The problem is solved in that the first and second listening devices each comprise an alignment unit for aligning the first and second streamed target audio signals with the first and second propagated electric signals in the first and second listening devices, respectively, to provide first and second aligned streamed target audio signals in the first and second listening devices, respectively. The use is e.g. in applications comprising simultaneous acoustic propagation and wireless transmission of an audio signal to an audio receiving device. -
US6845338 B1 discloses a telemetric contextually based spatial audio system integrated into a mobile terminal including a headset and antenna. The headset includes an electronic compass which determines head orientation data for the user. The terminal's geographical position can either be determined locally (e.g. onboard GPS) or by network triangulation. The orientation and position data are sent to a server which overlays the position and head orientation data onto a locally stored "floor plan", describing an object(s) of interest. The floor plan relates the user's viewpoint to a database of contextual audio clips. A regionally significant portion of the database and respective triggers are sent to the terminal. The terminal using the database information recalculates which audio clip(s) to play. The terminal then convolves the respective audio clip(s) via an impulse response function so as to be perceived by the user as emanating from the exact spatial position of the object specified by the system. - There is a need for a hearing device that improves separation of audio signals from different audio sources.
- Despite the known solutions there is still a need for improved user experience in a crowded area with multiple microphones/audio sources.
- Accordingly, a method of operating a hearing device is provided. The hearing device comprises a first antenna, a first transceiver coupled to the first antenna, an acoustic output transducer, a microphone, and a primary processing unit, the primary processing unit being coupled to the first transceiver, the acoustic output transducer, and the microphone. The method comprises pairing the hearing device with an audio system; initiating detection of hearing device motion; sending position data indicative of hearing device position to the audio system, if a position change criterion is fulfilled; wirelessly receiving a first audio stream based on the position data of the hearing device; converting the first audio stream to a first wireless input signal; mixing the first wireless input signal with an audio input signal from the microphone to form an input signal; processing the input signal to form an output signal; and converting the output signal to an audio output signal.
- Furthermore a hearing device is provided. The hearing device comprises a first antenna; a first transceiver coupled to the first antenna and configured to wirelessly receive one or more audio streams from an audio system, the one or more audio streams including a first audio stream, wherein the hearing device is configured for pairing the hearing device with the audio system; a microphone; a primary processing unit adapted for processing an input signal according to a hearing prescription in order to alleviate a hearing loss; an acoustic output transducer coupled to an output of the primary processing unit for conversion of an output signal from the primary processing unit into an audio output signal; a secondary processing unit with a first input and a second input, wherein the first input is coupled to an output of the first transceiver for receiving a first wireless input signal representative of the first audio stream from the first transceiver, and the second input is coupled to an output of the microphone for receiving an audio input signal from the microphone, wherein the secondary processing unit is configured to mix the first wireless input signal and the audio input signal to form an input signal on an output of the secondary processing unit, wherein the output of the secondary processing unit is coupled to the input of the primary processing unit; and a position controller configured for detecting and sending position data of the hearing device to the audio system, if a position change criterion is fulfilled.
- It is an advantage of the present invention that the audio stream received in the hearing device is based on the position (direction and/or distance) of the hearing device, thereby improving the user experience of the hearing device. Thus, the position (distance and/or orientation) of the hearing device may provide prioritization of audio signals.
- Further, the disclosed hearing device and method improve the correspondence of the visual and the auditory cues of the listener, otherwise lost in crowd amplification.
- The primary processing unit and/or the secondary processing unit of the hearing device may be configured to compensate for hearing loss or hearing disability of the user of the hearing device. The primary processing unit and/or the secondary processing unit of the hearing device may be configured to alter a received audio stream, e.g. the first audio stream, a wireless input signal, e.g. the first wireless input signal, and/or the input signal to compensate for hearing loss or hearing disability of the user of the hearing device.
- The hearing device comprises a secondary processing unit and a primary processing unit. The secondary processing unit and the primary processing unit may be embedded as a single processing unit performing the tasks of both the secondary processing unit and the primary processing unit.
- The above and other features and advantages of the present invention will become readily apparent to those skilled in the art by the following detailed description of embodiments thereof with reference to the attached drawings, in which:
- Fig. 1
- schematically illustrates ahearing device,
- Fig. 2
- schematically illustrates an exemplary audio system,
- Fig. 3
- schematically illustrates an exemplary audio system and an exemplary hearing device in the form of a binaural hearing device,
- Fig. 4
- is a flow diagram for amethod for operating a hearing device,
- Fig. 5
- is a flow diagram for amethod for operating a hearing device, and
- Fig. 6
- is a flow diagram for an exemplary method for operating an audio system.
- The figures are schematic and simplified for clarity, and they merely show details which are essential to the understanding of the invention, while other details have been left out. Throughout, the same reference numerals are used for identical or corresponding parts.
- The present invention relates to a hearing device, audio system and methods enabling adaptation of signal processing in the hearing device based on position data of the hearing device
- In an audio system, one or more transceivers or transmitter units comprising transceivers are configured for broadcasting or transmitting audio track(s) and/or audio stream(s), e.g. to one or more hearing devices. The audio system may comprise a first transceiver and optionally a second transceiver. A transceiver may transmit one or more audio streams. A transceiver may be identified by a transceiver address. A transmitter unit may be configured to wirelessly transmit audio track(s) and/or audio stream(s).
- An audio track is an audible signal. Exemplary audible signals are speech, music, an airport call, a movie sound track or a combination thereof.
- An audio stream is an electromagnetic signal representative of an audio track. An audio stream is represented by a number of audio blocks each comprising one or more audio packets. An audio packet comprises a finite part of an audio stream.
- An audio packet may comprise an audio stream identifier, which may comprise one or more elements such as audio track identification, transceiver address or identification, audio group identification, audio source identification, etc. Audio track identification may enable selection of audio streams representative of the same audio track, and audio group identification may identify a certain group of audio tracks or audio streams e.g. an audio group may be audio tracks in the same or similar language, e.g. English, Danish or Chinese.
- An audio packet may comprise transceiver address of the transceiver transmitting the audio packet, thus enabling the hearing device to sort out or separate audio packets sent from other transceivers.
- Transmitting and receiving the wireless audio stream(s) may be achieved by using wireless technology, thus transmitting/receiving audio blocks at one or more frequencies, e.g. in the range from 2.4 GHz to 2.5 GHz, in the range from 800 MHz to 1 GHz, in the range from 3.6 GHz to 3.7 GHz, and/or in the range from 4.9 GHz to 5.9 GHz.
- The primary processing unit may be configured to control the first transceiver. The first transceiver and/or the second transceiver may be configured to receive audio streams at frequencies in the range from 2.4 GHz to 2.5 GHz, in the range from 800 MHz to 1 GHz, in the range from 3.6 GHz to 3.7 GHz, and/or in the range from 4.9 GHz to 5.9 GHz. The first transceiver and/or the second transceiver may be configured to receive audio streams at frequencies in the range from 169 MHz to 218 MHz and/or in the range from 480 MHz to 520 MHz.
- Pairing of the hearing device with the audio system, e.g. with the one or more audio sources of the audio system, may comprise transmitting hearing device information from the hearing device to the audio system. The hearing device information may comprise one or more of hearing device idenfication, hearing device manufacturer, hearing device model, and/or hearing device configuration data. Pairing of the hearing device with the audio system may additionally or alternatively comprise transmitting audio system information from the audio system to the hearing device and/or receiving audio system information from the audio system. The audio system information may comprise one or more of audio system identification, audio source identification(s), transceiver identification(s) and/or audio stream configuration data.
- The audio system may be configured to adapt the format of the first audio stream to be sent to the hearing device based on hearing device information received from the hearing device, e.g. model and/or manufacturer, data encoding format etc. Furthermore, pairing of the hearing device with the audio system may comprise exchanging an encryption key, such as to enable encryption of an audio stream, such as the first audio stream, before sending from the audio system, and decryption of the received wireless input signal, e.g. the first wireless input signal, in the hearing device.
- Pairing of the hearing device with the audio system may initiate sending of an audio stream, e.g. the first audio stream, from the audio system.
- Pairing of the hearing device with the audio system may cause the hearing device to inititate receiving a wireless input signal, e.g. the first wireless input signal, which is sent by the audio system.
- The position controller may be coupled to the primary processing unit for receiving and/or sending position control signal(s) from/to the primary processing unit.
- The hearing device may comprise a second transceiver and/or a second antenna. The second transceiver and/or the second antenna may be coupled to the position controller.
- The second transceiver and/or the second antenna may be coupled to the primary processing unit.
- Pairing of the hearing device with the audio system may be performed via the first transceiver and/or the second transceiver of the hearing device.
- The hearing device sends position data indicative of hearing device position to the audio system or device(s) in the audio system. The hearing device may send position data indicative of hearing device position to the audio system via another hearing device or a peripheral or external device, such as a smartphone or a tablet computer. Thus, the first transceiver and/or the second transceiver may be configured to transmit and/or receive wireless signals from/to one or more audio systems or other external devices, such as other hearing devices, mobile phones, or tablet computers.
- The hearing device may send position data indicative of hearing device position to the audio system with the first transceiver and/or the second transceiver.
- The position controller of the hearing device may comprise a motion detector and optionally a position estimator connected to the motion detector. The position estimator may be configured to estimate position data of the hearing device, e.g. based on motion detector output from a motion detector. The motion detector may comprise a plurality of sensors e.g. one or more gyroscopes and/or one or more accelerometers.
- Position data of the hearing device and/or position data from audio source(s) may be sent continuously or regularly with a fixed time interval, e.g. every 100 ms, or every 500 ms, or every 1000 ms. Alternatively, or additionally, position data may be sent to the audio system if a position change criterion is fulfilled. For example, the hearing device may detect that the position is changing, and when the hearing device detects that the position is changing, the new position is sent to the first external device. The change criterion may e.g. be a change in the azimuth angle larger than a change threshold. The change threshold may be more than 0.5 degrees, or more than 1 degree, or more than 2 degrees, or more than 5 degrees. In another exemplary method and/or hearing device, the position data may be sent to the audio system if the position change criterion is fulfilled and at least by a fixed time interval, e.g. every 100 ms, or every 500 ms, or every 1000 ms. Thereby, the position data are updated regularly, but more frequently by a detection of change.
- The audio system may receive position data correspondingly to the sending of position data from the hearing device.
- It may be beneficial to limit the amount of data transmitted from the hearing device due to the limited battery capabilities of a hearing device. Reducing the necessity of sending position data, e.g. by sending position data only if a position change criterion is fulfilled, reduces the power consumption of the hearing device.
- Initiating detection of hearing device motion may comprise setting up a reference coordinate system. Initiating detection of hearing device motion may comprise a calibration procedure where information of an initial alignment of the hearing device and the audio source(s) are determined. The calibration procedure may comprise setting up a reference coordinate system. The setting up of a reference coordinate system may comprise registering absolute position data of the hearing device and/or the audio source(s), e.g. while the user of the hearing device is looking in a predetermined (reference) direction and/or at a given (reference) distance, such as looking at a specific audio source or other reference at a predetermined distance. The reference coordinate system may have an origin at a specific point in relation to an audio source and/or the hearing device.
- The position data may comprise angular position data of the hearing device. For example, the position data may comprise information of the azimuth angle of the hearing device. The azimuth angle may be indicative of the looking direction of the user wearing the hearing device. Alternatively, or additionally, the position data may comprise information of the altitude angle.
- The position data may comprise distance position data of the hearing device. For example, the position data may comprise information of the distance of the hearing device relative to an audio source and/or a reference point.
- The position data may be sent via the first transceiver and/or the first antenna of the hearing device. Alternatively or additionally, the position data may be sent via a second transceiver of the hearing device. The hearing device may comprise a second antenna. The second transceiver may be coupled to the second antenna. The position data may be sent via the second antenna. Pairing the hearing device with one or more devices of an audio system may be performed via the second antenna.
- The audio system comprises one or more audio sources including the first audio source and optionally a second audio source. The audio system may comprise a third audio source and/or a fourth audio source. The first audio source may comprise a first microphone. The second audio source may comprise a second microphone. The audio system may comprise a central unit with the processor and the first transceiver. The audio source(s) may be wired and/or wirelessly coupled to the central unit for feeding input signals to the processor.
- The method of operating an audio system may comprise receiving a plurality of input signals from respective plurality of audio sources, such as receiving a first input signal from the first audio source, and/or receiving a second input signal from the second audio source.
- The method of operating an audio system may further comprise processing input signals from the plurality of audio sources, e.g. the first audio source and the second audio source, i.e. the first input signal and the second input signal. Processing of the input signals may be based on position data of the hearing device to form the first wireless output signal.
- Processing the input signal(s) based on position data of the hearing device may comprise determining one or more gains, including a first gain for the first input signal and/or a second gain for the second input signal, based on position data of the hearing device, and applying the gain(s) to respective input signal(s) from audio source(s).
- The method of operating the audio system may comprise obtaining position data of one or more audio source(s), e.g. the first audio source and/or the second audio source. Position data of audio source(s) may be obtained during an audio system setup procedure. This is in particular useful, when the audio sources are stationary.
- Processing of input signal(s) may be based on position data of the audio source(s). Processing of the first input signal may be based on position data of the first audio source and/or processing of the second input signal may be based on position data of the second audio source. For example, a movement detected by the hearing device may be caused by the user looking at a moving audio source, which preferably should not result in a change of the audio received in the hearing device. Processing of an input signal, such as the first input signal and/or the second input signal, to form the first wireless output signal may be based both on position data of the respective audio source(s), such as the first audio source and/or the second audio source, and the position data of the hearing device.
- The hearing device may be a binaural hearing device system comprising a first hearing device and a second hearing device. Processing the input signal(s) may comprise processing the input signal(s) based on position data of the hearing device to form a first primary wireless output signal and a first secondary wireless output signal. The sending of a first audio stream to the hearing device may comprise sending a first primary audio stream to the first hearing device and sending a first secondary audio stream to the second hearing device.
- The first primary wireless output signal and the first secondary wireless output may at least in part be based on a head-related-transfer-function (HRTF). This entails changes in both amplitude and phase differentially to different ears, which are then perceived by the user of the binaural hearing device as a more natural perception of the effect of head movement on hearing.
- Processing the first input signal may comprise applying a first gain G1 to the first input signal. The first gain G1 may be applied to the first input signal to form the first wireless output signal. The first gain G1 may be based on position data for the hearing device, e.g. given by:
- In an audio system with a plurality of audio sources, e.g. a first audio source and a second audio source, processing of the input signals from the plurality of audio sources may comprise applying a vector of gains. The vector of gains may comprise a gain for each of the input signals. The vector of gains G may be given by:
-
- Where PHD is position data for the hearing device, PAS is position data for an audio source. In the case of a plurality of audio sources, PAS may be a vector with position data of each audio source.
- Processing the first input signal may comprise setting the first gain to a first value if a first field of listening (FIL) criterion is fulfilled, i.e. if the position of the hearing aid indicates that the hearing aid user wishes to listen to the first audio source. Processing the first input signal may comprise setting the first gain to a second value if the first FIL criterion is not fulfilled. The first value may be larger than the second value.
- The first FIL criterion may be based on the first view angle being the angle between the users direction of view and the direction from the hearing device to the first audio source. The first FIL criterion may be given by:
- Processing the second input signal may comprise setting the second gain to a first value if a second field of listening (FIL) criterion is fulfilled, i.e. if the position of the hearing aid indicates that the hearing aid user wishes to listen to the second audio source. Processing the second input signal may comprise setting the second gain to a second value if the second FIL criterion is not fulfilled. The first value may be larger than the second value.
- The second FIL criterion may be based on the second view angle being the angle between the users direction of view and the direction from the hearing device to the second audio source. The second FIL criterion may be given by:
-
Fig. 1 schematically illustrates a hearing device. Thehearing device 2 comprises afirst antenna 4, afirst transceiver 6, aprimary processing unit 8, anacoustic output transducer 10, aposition controller 12, amicrophone 14, and asecondary processing unit 16. - The
hearing device 2 is configured to pair with an audio system including one or more audio sources. For example, pairing of thehearing device 2 with the one or more audio sources is achieved by pairing thehearing device 2 with the audio system. - The
first transceiver 6 is coupled to thefirst antenna 4 and configured to wirelessly receive one or more audio streams including afirst audio stream 54 from the audio system. The one or more audio streams are converted to respective wireless input signals, e.g. thefirst audio stream 54 is converted to a firstwireless input signal 56. - The
secondary processing unit 16 has afirst input 18 and asecond input 20. Thefirst input 18 is coupled to an output of thefirst transceiver 6. Thesecond input 20 is coupled to an output of themicrophone 14. Thesecondary processing unit 16 receives the firstwireless input signal 56 from thefirst transceiver 6 on thefirst input 18, and receives theaudio input signal 58 from themicrophone 14 on thesecond input 20. Thesecondary processing unit 16 is configured to mix the firstwireless input signal 56 and theaudio input signal 58 to form the input signal 60 on an output of thesecondary processing unit 16. - The
primary processing unit 8 processes the hearing device input signal 60 to form anoutput signal 62. Theprimary processing unit 8 has an input coupled to the output of thesecondary processing unit 16. Theprimary processing unit 8 may perform signal processing such as noise reduction, filtering, amplification, etc. of the input signal 60. Theprimary processing unit 8 may perform signal processing of the input signal 60 to account for hearing disability of the user of thehearing device 2. - The
secondary processing unit 16 and theprimary processing unit 8 may be embedded as a single processing unit as indicated with the dotted box inFig. 1 - The
acoustic output transducer 10 is coupled to an output of theprimary processing unit 8 and receives anoutput signal 62 of the primary processing unit and converts theoutput signal 62 into anaudio output signal 64. Theaudio output signal 64 is perceptible by the user of thehearing device 2. - The
position controller 12 is configured for detecting and sending position data of thehearing device 2 to an audio system. Theposition controller 12 comprises amotion detector 22 and optionally aposition estimator 24 configured for estimating position data of the hearing device based on an output of themotion detector 22. In an exemplary hearing device, motion detector output may be coupled directly to thetransceiver 6. The position data of thehearing device 2 may be sent to the audio system via a first external device. In the depicted example, theposition controller 12 transmits positiondata output signal 66 indicative of the position data to thefirst transceiver 6. Thefirst transceiver 6 sends a wireless position data signal 68 based on the position data signal 66 via thefirst antenna 4 to the audio system either directly or via a first external device. - In an alternative exemplary hearing device (not shown), the wireless position data signal 68 may be sent to the audio system via a second transceiver and/or a second antenna in the hearing device.
-
Fig. 2 schematically illustrates anexemplary audio system 30. Theaudio system 30 comprises afirst audio source 32, aprocessing unit 38, and afirst transceiver 36 coupled to afirst antenna 34. Theprocessing unit 38 is coupled to thefirst transceiver 36. Thefirst audio source 32 comprises a microphone and is connected to theprocessing unit 38. - The
audio system 30 receives afirst input signal 50 from thefirst audio source 32. Thefirst input signal 50 is processed to form a firstwireless output signal 52. Afirst audio stream 54 is sent to thehearing device 2 based on the firstwireless output signal 52. Thefirst audio stream 54 send by the audio system corresponds to theaudio stream 54 received by the hearing device as illustrated inFig. 1 . - The
audio system 30 receives wireless position data signal 68 from thehearing device 2, the wireless position data signal 68 is indicative of the position of thehearing device 2. The wireless position data signal 68 is received by thefirst transceiver 36 via thefirst antenna 34. The first transceiver transmits a positiondata input signal 70 based on the wireless position data signal 68 to theprocessing unit 38 of theaudio system 30. The processing of theinput signal 50 is based on the position data of thehearing device 2. Hence, the firstwireless output signal 52 is adapted to the position of thehearing device 2. - In an alternative exemplary audio system (not shown), the wireless position data signal 68 may be received by a second transceiver and/or a second antenna in the audio system.
-
Fig. 3 schematically illustrates an exemplary audio system 30' and an exemplary hearing device system comprising a first hearing device 26 and a second hearing device 28. The first hearing device 26 and/or the second hearing device 28 may be a hearing device as disclosed herein, e.g. ahearing device 2. The audio system 30' comprises afirst audio source 32 comprising a first microphone, and a second audio source 32' comprising a second microphone. Thefirst audio source 32 and the second audio source 32' are wirelessly coupled to acentral unit 33 withprocessor 38 viainterface 35 for feeding first and second input signals to the processor.Processor 38 receives position data of one of or both hearing devices 26, 28 viasecond antenna 42 andsecond transceiver 44 in the audio system and processes the first and second input signals fromaudio sources 32, 32' based on the received hearing device position data to form a first primary wireless output signal and a first secondary wireless output signal. Hearing devices 26, 28 each comprisessecond antenna 42 andsecond transceiver 44 coupled to positioncontroller 12 for sending position data to the audio system. Theposition controller 12 may be omitted in one of the hearing devices 26, 28. The first primary wireless output signal and the first secondary wireless output signal are sent to the first hearing device 26 and second hearing device 28, respectively, as a first primary audio stream to the first hearing device and a first secondary audio stream to the second hearing device via thefirst transceiver 36. The first primary wireless output signal and the first secondary wireless output signal may be multiplexed to a single first audio stream comprising first and second channels. -
Fig. 4 shows a flow diagram of amethod 100 of operating a hearing device or a hearing device system. Themethod 100 comprises pairing 102 the hearing device with an audio system, initiating 104 detection of hearing device motion, sending 106 position data, receiving 108 audio stream(s), converting 110 audio stream(s) to wireless input signal(s), mixing 112 the wireless input signal(s) with audio input signal(s) to form input signal(s), processing 114 the input signal(s) to form output signal(s), and converting 116 the output signal(s) to audio output signal(s). - Pairing 102 the hearing device with the audio system may comprise pairing the hearing device with one or more audio sources of the audio system. Pairing of the hearing device with the
audio system 102 may further comprise exchanging hearing device identification data, audio system identification data and/or audio source identification data, between the hearing device and the audio system. - Initiating 104 detection of hearing device motion may comprise activating motion detection in the hearing device. Sending 106 position data may comprise sending position data indicative of hearing device position to the audio system.
- Receiving 108 audio stream(s) may comprise receiving a first audio stream based on the position data of the hearing device and/or receiving a second audio stream based on the position data of the hearing device.
- Converting 110 audio stream(s) may comprise converting the first audio stream to a first wireless input signal and/or converting the second audio stream to a second wireless input signal.
- Mixing 112 of wireless input signal(s) with audio input signal(s) to form input signal(s) may comprise mixing the first wireless input signal with an audio input signal from a microphone of the hearing device to form an input signal. Alternatively, or additionally, mixing 112 of wireless input signal(s) with audio input signal(s) to form input signal(s) may comprise mixing a plurality of wireless input signals, e.g. a first and a second wireless input signal, with one or more audio input signals, to form one or more input signals.
- Processing 114 the input signal(s) to form output signal(s) may comprise applying signal processing to the input signal(s) e.g. filtering, amplification, noise reduction and/or adjustments to compensate for hearing loss or hearing disability of the user of the hearing device. Processing 114 of the input signal(s) to form output signal(s) may comprise processing of a first input signal to form a first output signal.
- Converting 116 the output signal(s) to an audio output signal(s) may comprise converting a first output signal to a first audio output signal. The audio output signal(s) may be perceptible by the user of the hearing device.
- It is emphasized that the
method 100 may be looped, such that the hearing device is continuously sendingposition data 106, receiving audio stream(s) 108, converting audio stream(s) to wireless input signal(s) 110, mixing the wireless input signal(s) with audio input signal(s) to form input signal(s) 112, processing the input signal(s) to form output signal(s) 114, and converting the output signal(s) to audio output signal(s) 116. Further, different parts of the method may be run in parallel and/or sequentially. -
Fig. 5 shows a flow diagram of a method 100' of operating a hearing device. The method 100' comprises the same steps as themethod 100 as shown inFig. 4 . However, the method 100' illustrates the loop of the method, where the hearing device is continuously receiving audio stream(s) 108, converting audio stream(s) to wireless input signal(s) 110, mixing the wireless input signal(s) with audio input signal(s) to form input signal(s) 112, processing the input signal(s) to form output signal(s) 114, and converting the output signal(s) to audio output signal(s) 116. - The method 100' further comprises a
position change criterion 118. If the position change criterion is fulfilled, the method 100' returns to sendingposition data 106. If the position change criterion is not fulfilled the method 100' returns to the receiving of audio stream(s) 108. Theposition change criterion 118 determines if a position change large enough to justify resending of position data in 106 has occurred. Incorporation of aposition change criterion 118 may improve battery life of the hearing device by reducing the amount of wireless data transmission from the transceiver of the hearing device to the audio system. -
Fig. 6 shows a flow diagram of anexemplary method 200 of operating an audio system which is presently not explicitly claimed. Themethod 200 comprise pairing with ahearing device 202, receiving position data of thehearing device 204, receiving input signal(s) from audio source(s) 206, processing input signal(s) based on position data to form wireless output signal(s) 208, sending audio stream(s) to the hearing device based on the wireless output signal(s) 210. - Pairing 202 with a hearing device may comprise pairing the hearing device with one or more audio sources of the audio system. Pairing 202 with the hearing device may further comprise exchanging hearing device identification data, audio system identification data and/or audio source identification data, between the hearing device and the audio system.
- Receiving 204 position data of the hearing device may comprise continuously receiving updated position data. Alternatively or additionally, receiving 204 position data of the hearing device may comprise receiving position data if the position data has changed. The position data received is indicative of the hearing device position.
- Receiving 206 input signal(s) from audio source(s) may include receiving a first input signal from a first audio source and/or receiving a second input signal from a second audio source.
- Processing 208 the input signal(s) based on position data to form wireless output signal(s) may include processing of a first input signal based on position data to form a first wireless output signal and/or processing of a second input signal based on position data to form a second wireless output signal. Alternatively, a plurality of input signals may be processed based on position data to form a first wireless output signal. Sending 210 audio stream(s) to the hearing device based on wireless output signal(s) may include sending a first audio stream to the hearing device based on a first wireless output signal and/or sending a second audio stream to the hearing device based on a second wireless output signal.
- Optionally, the
method 200 comprises obtaining 212 position data of audio source(s), e.g. betweenpairing 202 with a hearing device and receiving 204 position data of thehearing device 204. The audio source(s) may be movable, and thus it may be beneficial to obtain or detect position data of the audio source(s) 212, and include the position data of the audio sources in the processing of the input signal(s) 208. - The
processing 208 of the input signal(s) to form wireless output signal(s) may be based on position data from the hearing device and/or position data from the audio source(s). -
- 2
- hearing device
- 4
- first antenna
- 6
- first transceiver
- 8
- primary processing unit
- 10
- acoustic output transducer
- 12
- position controller
- 14
- microphone
- 16
- secondary processing unit
- 18
- first input of secondary processing unit
- 20
- second input of secondary processing unit
- 22
- motion detector
- 24
- position estimator
- 26
- first hearing device
- 28
- second hearing device
- 30, 30'
- audio system
- 32
- first audio source
- 32'
- second audio source
- 33
- central unit
- 34
- first antenna
- 35
- interface
- 36
- first transceiver
- 38
- processing unit
- 42
- second antenna
- 44
- second transceiver
- 50
- input signal (audio system)
- 52
- wireless output signal
- 54
- audio stream
- 56
- wireless input signal
- 58
- audio input signal
- 60
- input signal (hearing device)
- 62
- output signal
- 64
- audio output signal
- 66
- position data output signal
- 68
- wireless position data signal
- 70
- position data input signal
- 100, 100'
- method for operating a hearing device
- 102
- pair with an audio system
- 104
- initiate detection of hearing device motion
- 106
- send position data
- 108
- receive audio stream(s)
- 110
- convert audio stream(s) to wireless input signal(s)
- 112
- mix wireless input signal(s) with audio input signal(s) to form input signal(s)
- 114
- process the input signal(s) to form output signal(s)
- 116
- converting output signal(s) to audio output signal(s)
- 118
- fulfilment of a position change criterion
- 200
- method for operating an audio system
- 202
- pair with a hearing device
- 204
- receive position data of the hearing device
- 206
- receive input signal(s) from audio source(s)
- 208
- process input signal(s) based on position data to form wireless output signal(s)
- 210
- send audio stream(s) to the hearing device based on wireless output signal(s)
- 212
- detecting position data of audio source(s)
Claims (8)
- Method (100, 100') of operating a hearing device (2) comprising a first antenna (4), a first transceiver (6, 36) coupled to the first antenna (4, 34), an acoustic output transducer (10), a microphone (14), and a primary processing unit (8), the primary processing unit (8) being coupled to the first transceiver (6), the acoustic output transducer (10), and the microphone (14), the method comprising- pairing (102) the hearing device (2) with an audio system (30, 30');- initiating (104) detection of hearing device motion;- sending (106) position data indicative of hearing device position to the audio system (30, 30'), if a position change criterion is fulfilled;- wirelessly receiving (108) a first audio stream based on the position data of the hearing device (2);- converting (110) the first audio stream to a first wireless input signal;- mixing (112) the first wireless input signal with an audio input signal (58) from the microphone (14) to form an input signal;- processing (114) the input signal to form an output signal (62); and- converting (116) the output signal (62) to an audio output signal (64).
- Method according to claim 1, wherein initiating detection of hearing device motion comprises setting up a reference coordinate system.
- Method according to any of the preceding claims, wherein the position data comprises angular position data of the hearing device (2).
- Method according to any of the preceding claims, wherein the position data are sent via the first transceiver (6).
- Method according to any of the preceding claims, wherein the hearing device (2) comprises a second antenna (42), and wherein the position data are sent via the second antenna (42).
- Method according to any of the preceding claims, wherein the hearing device (2) comprises a second antenna (42), and wherein pairing the hearing device (2) with one or more devices of an audio system (30, 30') is performed via the second antenna (42).
- Hearing device (2) comprising- a first antenna (4, 34);- a first transceiver (6, 36) coupled to the first antenna (4, 34) and configured to wirelessly receive one or more audio streams (54) from an audio system (30, 30'), the one or more audio streams (54) including a first audio stream; wherein the hearing device (2) is configured for pairing the hearing device (2) with the audio system (30, 30');- a microphone (14);- a primary processing unit (8) adapted for processing an input signal (60) according to a hearing prescription in order to alleviate a hearing loss;- an acoustic output transducer (10) coupled to an output of the primary processing unit (8) for conversion of an output signal (62) from the primary processing unit (8) into an audio output signal (64);- a secondary processing unit (16) with a first input (18) and a second input (20), wherein the first input (18) is coupled to an output of the first transceiver (6) for receiving a first wireless input signal representative of the first audio stream from the first transceiver (6), and the second input (20) is coupled to an output of the microphone (14) for receiving an audio input signal (58) from the microphone (14), wherein the secondary processing unit (16) is configured to mix the first wireless input signal and the audio input signal (58) to form an input signal on an output of the secondary processing unit (16), wherein the output of the secondary processing unit (16) is coupled to the input of the primary processing unit (8); and- a position controller (12) configured for detecting and sending position data of the hearing device (2) to the audio system (30, 30'), if a position change criterion is fulfilled.
- Hearing device (2) according to claim 7, wherein the position controller (12) comprises a motion detector (22) and a position estimator (24) connected to the motion detector (22), the position estimator (24) configured to estimate position data of the hearing device (2) based on motion detector output.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DK19197521.8T DK3618458T3 (en) | 2013-12-30 | 2013-12-30 | Hearing aid with position data, audio system and related procedures |
EP19197521.8A EP3618458B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP19197521.8A EP3618458B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
EP13199857.7A EP2890157B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13199857.7A Division EP2890157B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3618458A1 EP3618458A1 (en) | 2020-03-04 |
EP3618458B1 true EP3618458B1 (en) | 2021-11-24 |
Family
ID=49880635
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13199857.7A Active EP2890157B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
EP19197521.8A Active EP3618458B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP13199857.7A Active EP2890157B1 (en) | 2013-12-30 | 2013-12-30 | Hearing device with position data, audio system and related methods |
Country Status (2)
Country | Link |
---|---|
EP (2) | EP2890157B1 (en) |
DK (2) | DK3618458T3 (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6845338B1 (en) * | 2003-02-25 | 2005-01-18 | Symbol Technologies, Inc. | Telemetric contextually based spatial audio system integrated into a mobile terminal wireless system |
EP2584794A1 (en) * | 2011-10-17 | 2013-04-24 | Oticon A/S | A listening system adapted for real-time communication providing spatial information in an audio stream |
EP2675189B1 (en) * | 2012-06-14 | 2015-08-26 | Oticon A/s | A binaural listening system with automatic mode switching |
-
2013
- 2013-12-30 EP EP13199857.7A patent/EP2890157B1/en active Active
- 2013-12-30 DK DK19197521.8T patent/DK3618458T3/en active
- 2013-12-30 DK DK13199857T patent/DK2890157T3/en active
- 2013-12-30 EP EP19197521.8A patent/EP3618458B1/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
DK3618458T3 (en) | 2022-01-24 |
EP2890157A1 (en) | 2015-07-01 |
DK2890157T3 (en) | 2019-10-28 |
EP3618458A1 (en) | 2020-03-04 |
EP2890157B1 (en) | 2019-10-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9877116B2 (en) | Hearing device with position data, audio system and related methods | |
US11304013B2 (en) | Assistive listening device systems, devices and methods for providing audio streams within sound fields | |
US9271103B2 (en) | Audio control based on orientation | |
US9584933B2 (en) | Method and apparatus for localization of streaming sources in hearing assistance system | |
WO2007112756A3 (en) | System and method tracking the position of a listener and transmitting binaural audio data to the listener | |
CN104754467B (en) | The method of hearing device and operation hearing device with position data | |
US8761674B2 (en) | Electronic communication system that mimics natural range and orientation dependence | |
CN113228710B (en) | Sound source separation in a hearing device and related methods | |
EP4184944A1 (en) | Terminal for controlling wireless sound device, and method therefor | |
WO2024000853A1 (en) | Wearable device control method and apparatus, terminal device, and storage medium | |
EP3618458B1 (en) | Hearing device with position data, audio system and related methods | |
JP5816312B2 (en) | Audio system and related methods for audio stream delivery | |
WO2019010029A1 (en) | Locating connected devices | |
EP2890156B1 (en) | Hearing device with position data and method of operating a hearing device | |
EP4131984A1 (en) | Sound input/output control device, sound input/output control method, and program | |
DK201370827A1 (en) | Hearing device with position data and method of operating a hearing device | |
US20180318135A1 (en) | Notification service provision method for hearing-impaired person and device for executing same |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AC | Divisional application: reference to earlier application |
Ref document number: 2890157 Country of ref document: EP Kind code of ref document: P |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17P | Request for examination filed |
Effective date: 20200902 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: UDESEN, JESPER Inventor name: WEISSER, ADAM |
|
17Q | First examination report despatched |
Effective date: 20200928 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20210714 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AC | Divisional application: reference to earlier application |
Ref document number: 2890157 Country of ref document: EP Kind code of ref document: P |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602013080234 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1450797 Country of ref document: AT Kind code of ref document: T Effective date: 20211215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20220117 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20211124 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1450797 Country of ref document: AT Kind code of ref document: T Effective date: 20211124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220224 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220324 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220324 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220224 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220225 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602013080234 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20211231 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211230 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211230 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
26N | No opposition filed |
Effective date: 20220825 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20131230 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231215 Year of fee payment: 11 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231215 Year of fee payment: 11 Ref country code: DK Payment date: 20231219 Year of fee payment: 11 Ref country code: DE Payment date: 20231218 Year of fee payment: 11 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20211124 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20240102 Year of fee payment: 11 |