EP1651005B1 - Synchronization of sound generated in binaural hearing system - Google Patents

Synchronization of sound generated in binaural hearing system Download PDF

Info

Publication number
EP1651005B1
EP1651005B1 EP05027729.2A EP05027729A EP1651005B1 EP 1651005 B1 EP1651005 B1 EP 1651005B1 EP 05027729 A EP05027729 A EP 05027729A EP 1651005 B1 EP1651005 B1 EP 1651005B1
Authority
EP
European Patent Office
Prior art keywords
hearing
hearing device
audio signals
signals
sound
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP05027729.2A
Other languages
German (de)
French (fr)
Other versions
EP1651005A2 (en
EP1651005A3 (en
Inventor
Hans-Ueli Roeck
Raoul Glatt
Micha Knaus
Rolf Laich
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sonova Holding AG
Original Assignee
Sonova AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sonova AG filed Critical Sonova AG
Priority to EP05027729.2A priority Critical patent/EP1651005B1/en
Priority to DK05027729.2T priority patent/DK1651005T3/en
Publication of EP1651005A2 publication Critical patent/EP1651005A2/en
Publication of EP1651005A3 publication Critical patent/EP1651005A3/en
Application granted granted Critical
Publication of EP1651005B1 publication Critical patent/EP1651005B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/55Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
    • H04R25/552Binaural
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/40Arrangements for obtaining a desired directivity characteristic
    • H04R25/407Circuits for combining signals of a plurality of transducers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00Stereophonic arrangements
    • H04R5/033Headphones for stereophonic communication

Definitions

  • a monaural hearing device which comprises a signal emission device by means of which a user of the hearing device can be informed that a particular setting of the hearing device has been selected.
  • WO 01/30127 A2 a monaural hearing device is known, which allows to play user-defined sequences for acknowledging an action that has been carried out on the hearing device.
  • a binaural hearing system in which audio signals obtained from digitizing output signals of a microphone of the hearing system are transmitted from one hearing device of the system to the other hearing device of the system via a communication link. Furthermore, control signals can be transmitted via said communication link.
  • control signals can, e.g., be signals for controlling a signal processor of a hearing device, or signals by means of which a first of the two hearing devices acknowledges to the second hearing device that is has received a control signal that was transmitted from the second hearing device to the first hearing device.
  • the two hearing devices are synchronized by a controlled time-multiplexer unit, which is operated by a respective time control unit, wherein the two time control units are synchronized via said communication link.
  • a binaural hearing system with two hearing devices in which a communication link is provided between the two hearing devices for transmitting control and audio signals.
  • a control signal is generated by this hearing device and sent to the other hearing device in order to cause the same adjustment to the other hearing device's operation.
  • audio signals coming from a signal processing unit in either of the hearing devices can be sent to a signal processing unit in the other hearing device via the communication link.
  • a goal of the invention is, to create a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system with enhanced possibilities for giving the user of the hearing system the impression that he perceives signals from a certain location or certain direction.
  • One object of the invention is to provide for a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system with improved possibilities for synchronizing the perception of sound in the user's left and the right ear.
  • Another object of the invention is to provide for a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system providing for a precisely predictable time difference between the user's perception of system-generated signals in the user's left and right ears.
  • the binaural hearing system comprises
  • tick period is typically of the order of 10 ms (some milliseconds to some ten milliseconds).
  • the threads in the two hearing devices are usually not in phase and do usually not even have a constant phase relation, and even the tick periods in the two hearing devices are usually somewhat different.
  • the two hearing devices are switched on at different times; the booting of the hearing device controlling software does not take exactly the same amount of time in both hearing devices; the clock oscillators in the two hearing devices show (different) drift; the clock oscillators in the two hearing devices are (slightly) differently tuned.
  • particular tasks may run along threads which are different from the thread of other parts of the hearing device software.
  • the thread may even have a different tick period than the hearing device control software.
  • some types of processes within a hearing device are deterministic (within the desired timing accuracy), e.g., signal processing, conversion, amplification or transmission of data or signals from one hearing device to the other.
  • the time required for such processes can be determined in advance; accordingly, they do not contribute to a (random) jitter.
  • Only one stream of audio signals (acoustic sound message data) is generated (issued), by only one of the two hearing devices, and this stream is delivered, via said communication link, to the other hearing device, in which it may undergo amplification and other processing and finally presentation to the user with near-negligible latency with respect to the contralateral side.
  • equal or similar processing of the audio signals may take place, and, if required, pre-determinable delays between the two sides may be accomplished for by introducing an according delay on the faster side.
  • An audio signal (or sound message) originating from a single source is presented to the user with a deterministic delay between the two sides.
  • the invention can be used in conjunction with acknowledge signals (or other sound messages) as the signals to be perceived by the hearing device user.
  • Acknowledge sounds also called feedback sounds, are played to the user upon a change in the hearing device's function, e.g., when the user changes the loudness (volume) or another setting or program of one or both hearing devices, or when some other user's manipulation shall be acknowledged, or when the hearing device by itself takes an action, e.g., by making a change, e.g., if, in the case of a hearing aid, the hearing aid chooses, in dependence of the acoustical environment, a different hearing program (frequency-volume settings and the like), or when the hearing device user shall be informed that a hearing device's battery is low.
  • a change in the hearing device's function e.g., when the user changes the loudness (volume) or another setting or program of one or both hearing devices, or when some other user's manipulation shall be acknowledged, or when the hearing device by itself takes an action, e.g., by making a change, e.g., if, in the case of a hearing aid, the hearing
  • Said signals to be perceived by the user are often acoustic signals (sound waves), but may be other signals as well, e.g., in the case of implanted hearing devices.
  • the hearing devices' output transducers can therefore be electro-to-mechanical converters (loudspeakers) or others, e.g., electrical-to-electrical converters.
  • Said audio signals are usually electrical signals, analogue and/or digital.
  • Said communication link may be wireless (typically electromagnetically; e.g., in the radio frequency range via frequency modulation, or Ultra-Wide-Band communication UWB), or wire-bound.
  • Said communication link allows for communication at least from said first hearing device to said second hearing device, but usually allows for bidirectional communication.
  • Said communication link may involve at least one transmitter comprised in said first hearing device and at least one receiver comprised in said second hearing device. It may, in addition, comprise at least one receiver comprised in said first hearing device and at least one transmitter comprised in said second hearing device.
  • Said sound generator may be implemented in software or (partially) in form of hardware.
  • said first and said second hearing devices each comprise an input transducer for receiving incoming signals and for converting said incoming signals into audio signals.
  • incoming signals are incoming sound
  • said input transducers are mechanical-to-electrical converters (microphones), but converters receiving electromagnetical waves and converting these into audio signals are also possible (e.g., in case of a telephone coil or reception of a remote microphone signal via a frequency modulation -FM- receiver).
  • At least one of said hearing devices each comprise a signal processor for processing audio signals.
  • the system comprises, comprised in said first hearing device, a first sound memory, which comprises a first set of data, which describe audio signals (in particular, first audio signals), and, comprised in said second hearing device, a second sound memory, which comprises a second set of data, which describe audio signals, wherein said first and second sets of data are substantially different from each other.
  • both hearing devices contain data describing (different) audio signals, and, accordingly, the required storage space in each hearing device can be (up to) halved (with respect to storing the same data in each hearing device).
  • the sound generator can produce the corresponding audio signal, which finally can be transmitted to the second hearing device and transduced into signals acknowledging said manipulation (on both sides).
  • the data sets can be considered first and second sets of acknowledge sound descriptions (or representations).
  • audio signals may be described, e.g., in a parametrized form (e.g., as a frequency to be played and a time for which it is to be played), or in a sampled form (e.g., compressed, like in MP3 format).
  • a parametrized form e.g., as a frequency to be played and a time for which it is to be played
  • sampled form e.g., compressed, like in MP3 format
  • At least one of said first and said second hearing devices comprises a storage unit containing information on the contents of said first sound memory and/or of said second sound memory.
  • a storage unit containing information on the contents of said first sound memory and/or of said second sound memory.
  • the storage unit may be a table or a look-up table.
  • the system furthermore comprises a second sound generator for generating second audio signals, which is comprised in said second hearing device, and wherein said second hearing device is adapted to transmitting said second audio signals to said first hearing device via said communication link.
  • both hearing devices comprise a sound generator, and sound generated in either sound generator may be transmitted to the other hearing device.
  • Said first audio signals, and also said second audio signals may comprise or be acknowledge sound signals.
  • the corresponding time delays in each hearing device can be calculated in advance, and corresponding delays may be used in order to let the user perceive the sounds in each ear with a well-defined time delay.
  • the features of the method of synchronization in a hearing system corresponds to the features of the method for operating a hearing system.
  • the method allows for a synchronized perception of signals to be perceived by the user in each ear.
  • the method may furthermore comprise the steps of
  • said first hearing device comprises a first sound memory, which comprises a first set of data, which describe audio signals (first and/or second audio signals)
  • said second hearing device comprises a second sound memory, which comprises a second set of data, which describe audio signals (first and/or second audio signals)
  • said first and second sets of data are substantially different from each other, further comprises the step of
  • This method wherein at least one of said first and said second hearing devices comprises a storage unit containing information on the contents of said first sound memory and (or and/or) of said second sound memory, may further comprise the steps of
  • a command can be transmitted (via said communication link) to the other hearing device (which comprises the sound memory comprising said data) in order to retrieve the desired sound memory data.
  • FIGs. 1 and 2 show schematic diagrams of hearing systems 1.
  • a hearing system 1 comprises two hearing devices 10L and 10R, which may be similarly or identically structured.
  • the hearing device 10L of Figs. 1 and 2 may comprise some more elements, in particular such, which are only drawn in the first hearing device 10R.
  • audio signals (audio data streams) are drawn as solid arrows; control signals or data are represented by dashed arrows. Not all possible signal flows are indicated, but only some, which are required to explain the invention.
  • the hearing devices 10R, 10L comprise controllers or control softwares 15L, 15R.
  • the hearing devices 10L, 10R comprise input transducers 11L, 11R, typically microphones, which convert incoming sound 5 into audio signals (analogue and/or digital electrical signals).
  • audio signals may be processed in signal processors 13L, 13R, preferably digital signal processors (DSP), and are possibly amplified in amplifiers 14L, 14R before being converted into signals 7,8 to be perceived by a user of the hearing system 1 by output transducers 12L,12R, typically loudspeakers.
  • DSP digital signal processors
  • Hearing device 10L comprises a receiver Re.
  • Fig. 1 it is sketched, how a concurrent or (practically) jitter-free perception by the user of an acknowledge sound, e.g. upon a volume change in hearing device 10R, can be achieved.
  • the user manipulates the user interface 16R in order to change the volume (loudness) of signals 7,8 he perceives (typically the loudness in both hearing devices shall be changed).
  • This generates a control signal, whereupon the sound generator 17R is instructed by the controller 15R to generate an appropriate sound message (acknowledge signal), which is referred to as first audio signals 30.
  • the first audio signals 30 are, on the one hand, fed to the output transducer 12R (e.g., via DSP 13R and amplifier 14R, or directly, or in between the two) to be converted into signals 7 to be perceived by the user, and on the other hand, fed to the transmitter Tr of the hearing device 10R.
  • the output transducer 12R e.g., via DSP 13R and amplifier 14R, or directly, or in between the two
  • the transmitter Tr of the hearing device 10R instead of adding the first audio signals 30 to audio signals obtained from incoming sound 5, they may as well replace those audio signals.
  • the first audio signals 30 are made available in the hearing device 10L, where they are, in analogy to the hearing device 10R, converted into signals 8 to be perceived by the user.
  • the (first) audio signals may be processed using Head-Related Transfer Functions (HRTF) in the sense that not only interaural time differences (ITD) are applied as described above, but also interaural level differences (ILD), or, more generally speaking, frequency dependent interaural amplitude and phase differences.
  • HRTF Head-Related Transfer Functions
  • Fig. 2 shows, in the manner of Fig. 1 , a schematic diagram of another hearing system 1.
  • the embodiment is similar to the one of Fig. 1 , and mainly the additional or different features will be described.
  • Both hearing devices 10L, 10R comprise a sound memory 18L, 18R, in each of which a set of data describing audio signals (e.g., acknowledge sounds and/or speech sounds) are contained.
  • the sound memory 18R may contain (data describing) acknowledge sounds, which are to be played when the volume is increased, when the volume is decreased, or when the battery level is low, whereas in the sound memory 18L, (data describing) acknowledge sounds may be stored, which acknowledge a change from one hearing program to another hearing program.
  • Both hearing devices comprise a transmitter Tr and a receiver Re each, thus enabling a bidirectional communication link 20.
  • Fig. 2 it is sketched, how a concurrent or (practically) jitter-free perception by the user of an acknowledge sound, e.g. upon a volume change in hearing device 10L, can be achieved.
  • the user manipulates the second hearing device's user interface 16L.
  • the controller checks the storage unit (table, LUT) 19L and finds out, that the appropriate acknowledge sound (first audio signals 30) is not stored in the sound memory 18L, but in the sound memory 18R of the contralateral hearing device 10R. Thereupon, a control signal is transmitted via the second hearing device's transmitter Tr and the first hearing device's receiver Re to the second hearing device's controller 15R.
  • the controller 15R ensures that the requested data 40 representing (describing) the requested first audio signals 30 are loaded from the sound memory 18R and that the requested audio signals 30 are generated.
  • the requested audio signals 30 are fed towards the output transducer 12R and, in addition, transmitted to the second hearing device 10L via the communication link 20, from where it is transmitted to the output transducer 12L
  • the second hearing device 10L may comprise a sound generator.
  • data and/or audio signals have to be transmitted via the communication link 20.
  • Fig. 3 shows a block diagram of a simple method of operating a hearing system 1.
  • the steps 100 - 300 are self-explaining.
  • the invention enables for a deterministic (predictable) latency and negligible jitter for signals to be perceived by the user's two ears.
  • the invention is particularly useful, when speech signals or more complex sounds are to be generated and presented to the user.
  • the complexity of a sound may manifest in its (large) spectral content, its structure in time or rhythmic or percussive structure.
  • Speech sounds may be used for guiding the user, informing the user and acknowledging events in the hearing system. Requirements for latency and jitter are less tough for simple sounds, like sine tones that fade in and fade out.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)
  • Details Of Audible-Bandwidth Transducers (AREA)

Description

    Technical Field
  • The invention relates to a binaural hearing system comprising two hearing devices, one for each ear of a user of the hearing system, and to a method of operating such a hearing system and a method of synchronization in such a hearing system. Such a hearing system is a linked pair of hearing devices. The hearing devices can be hearing aids, worn in or near the ear or implanted, headphones, earphones, hearing protection devices, communication devices or the like.
  • Background of the Invention
  • From EP 0 557 847 B1 a monaural hearing device is known, which comprises a signal emission device by means of which a user of the hearing device can be informed that a particular setting of the hearing device has been selected.
  • From WO 01/30127 A2 a monaural hearing device is known, which allows to play user-defined sequences for acknowledging an action that has been carried out on the hearing device.
  • From US 2002/0131613 A1 a binaural hearing system is known, in which audio signals obtained from digitizing output signals of a microphone of the hearing system are transmitted from one hearing device of the system to the other hearing device of the system via a communication link. Furthermore, control signals can be transmitted via said communication link. Such control signals can, e.g., be signals for controlling a signal processor of a hearing device, or signals by means of which a first of the two hearing devices acknowledges to the second hearing device that is has received a control signal that was transmitted from the second hearing device to the first hearing device. The two hearing devices are synchronized by a controlled time-multiplexer unit, which is operated by a respective time control unit, wherein the two time control units are synchronized via said communication link.
  • From DE 10 2004 035 046 A1 binaural hearing systems are known, which provide for "virtual sound sources" in the sense that system-generated sounds can be perceived by a user of the system as if they were generated in different locations near the user. Said DE 10 2004 035 046 A1 does not disclose, how to achieve the time synchronization (and timing accuracy) between the signals to be perceived by the user with his left and his right ear, respectively, which is necessary in order to predictably achieve such a virtual sound source effect.
  • From EP 0 941 014 A2 a binaural hearing system with two hearing devices is known, in which a communication link is provided between the two hearing devices for transmitting control and audio signals. When a user actuates a control element on one of the hearing devices in order to adjust the hearing device's operation a control signal is generated by this hearing device and sent to the other hearing device in order to cause the same adjustment to the other hearing device's operation. Furthermore, audio signals coming from a signal processing unit in either of the hearing devices can be sent to a signal processing unit in the other hearing device via the communication link.
  • Summary of the Invention.
  • A goal of the invention is, to create a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system with enhanced possibilities for giving the user of the hearing system the impression that he perceives signals from a certain location or certain direction.
  • One object of the invention is to provide for a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system with improved possibilities for synchronizing the perception of sound in the user's left and the right ear.
  • Another object of the invention is to provide for a binaural hearing system and a method of operating a hearing system and a method of synchronization in a hearing system providing for a precisely predictable time difference between the user's perception of system-generated signals in the user's left and right ears.
  • These objects are achieved by a binaural hearing system and by a method for operating a hearing system and by a method of synchronization in a hearing system according to the patent claims.
  • The binaural hearing system comprises
    • a first hearing device and a second hearing device, each comprising an output transducer for converting audio signals into signals to be perceived by a user of the hearing system;
    • a communication link interconnecting said first and said second hearing device; and
    • a sound generator comprised in said first hearing device for generating first audio signals;
    wherein first hearing device is adapted to transmitting said first audio signals to said second hearing device via said communication link.
  • Through this, it is possible to achieve a rather precisely predictable time difference between the perception of signals obtained from said first audio signals in the user's left and right ears.
  • Today, software-controlled digital hearing devices are widely used. Accordingly, commands of such a hearing device software are generated and executed along clocked threads with a certain tick period, which determines discrete points in time at which commands are executed. Such a tick period is typically of the order of 10 ms (some milliseconds to some ten milliseconds). In the case of a hearing system with two hearing devices, the threads in the two hearing devices are usually not in phase and do usually not even have a constant phase relation, and even the tick periods in the two hearing devices are usually somewhat different. This may be due to various reasons: the two hearing devices are switched on at different times; the booting of the hearing device controlling software does not take exactly the same amount of time in both hearing devices; the clock oscillators in the two hearing devices show (different) drift; the clock oscillators in the two hearing devices are (slightly) differently tuned.
  • Accordingly, there will always be a jitter of the order of some or several milliseconds between "synchronized" commands in the two hearing devices, unless great effort is taken to create a stable phase relation between the threads of the control software of the two hearing devices. Note that such a jitter means a practically randomly occuring time difference between left and right hearing device. Thus, signals perceived by a user will, if such a jitter is present, randomly appear to arrive from random directions, to contain echo, or to stutter.
  • In addition, particular tasks, like hearing-device-internal sound generation, may run along threads which are different from the thread of other parts of the hearing device software. The thread may even have a different tick period than the hearing device control software.
  • All this makes it difficult or impossible to provide a hearing system user with signals (typically sound), so that these signals are perceived by the user as coming from a predictable direction. For achieving such an effect, an accuracy of some or some ten microseconds (µs) (at most maybe about 50 µs to 100 µs) is necessary, since the whole range spanning from signals being perceived as stemming from straight ahead or from the back to signals being perceived as stemming from the very left or very right of the user ranges, in terms of time delays between the left and the right side, from 0 µs to about 600 µs or 650 µs. A jitter of more than 0.5 ms or even more than 1 ms is therefore not acceptable.
  • On the other hand, some types of processes within a hearing device are deterministic (within the desired timing accuracy), e.g., signal processing, conversion, amplification or transmission of data or signals from one hearing device to the other. The time required for such processes can be determined in advance; accordingly, they do not contribute to a (random) jitter.
  • Accordingly, for signals, which shall be perceived by the user's ears with a minimal time jitter (of the order of magnitude of 10 µs) between left and right, only deterministic processes shall take place in each of the hearing devices after a highly synchronized origin of the signals (no jitter or below about 20 us). Such an origin can be the generation of said signals in only one of the hearing devices.
  • Only one stream of audio signals (acoustic sound message data) is generated (issued), by only one of the two hearing devices, and this stream is delivered, via said communication link, to the other hearing device, in which it may undergo amplification and other processing and finally presentation to the user with near-negligible latency with respect to the contralateral side. In both hearing devices, equal or similar processing of the audio signals may take place, and, if required, pre-determinable delays between the two sides may be accomplished for by introducing an according delay on the faster side.
  • Thus, random delays from a lack of (precise) synchronization between the two sides are effectively avoided.
  • An audio signal (or sound message) originating from a single source is presented to the user with a deterministic delay between the two sides.
  • Typically, the invention can be used in conjunction with acknowledge signals (or other sound messages) as the signals to be perceived by the hearing device user.
  • Acknowledge sounds, also called feedback sounds, are played to the user upon a change in the hearing device's function, e.g., when the user changes the loudness (volume) or another setting or program of one or both hearing devices, or when some other user's manipulation shall be acknowledged, or when the hearing device by itself takes an action, e.g., by making a change, e.g., if, in the case of a hearing aid, the hearing aid chooses, in dependence of the acoustical environment, a different hearing program (frequency-volume settings and the like), or when the hearing device user shall be informed that a hearing device's battery is low.
  • Said signals to be perceived by the user are often acoustic signals (sound waves), but may be other signals as well, e.g., in the case of implanted hearing devices. The hearing devices' output transducers can therefore be electro-to-mechanical converters (loudspeakers) or others, e.g., electrical-to-electrical converters.
  • Said audio signals are usually electrical signals, analogue and/or digital.
  • Said communication link may be wireless (typically electromagnetically; e.g., in the radio frequency range via frequency modulation, or Ultra-Wide-Band communication UWB), or wire-bound. Said communication link allows for communication at least from said first hearing device to said second hearing device, but usually allows for bidirectional communication.
  • Said communication link may involve at least one transmitter comprised in said first hearing device and at least one receiver comprised in said second hearing device. It may, in addition, comprise at least one receiver comprised in said first hearing device and at least one transmitter comprised in said second hearing device.
  • Said sound generator may be implemented in software or (partially) in form of hardware.
  • Typically, said first and said second hearing devices each comprise an input transducer for receiving incoming signals and for converting said incoming signals into audio signals. Typically, such incoming signals are incoming sound, and said input transducers are mechanical-to-electrical converters (microphones), but converters receiving electromagnetical waves and converting these into audio signals are also possible (e.g., in case of a telephone coil or reception of a remote microphone signal via a frequency modulation -FM- receiver).
  • Typically, at least one of said hearing devices, usually both (said first and said second hearing devices), each comprise a signal processor for processing audio signals. In one embodiment, the system comprises, comprised in said first hearing device, a first sound memory, which comprises a first set of data, which describe audio signals (in particular, first audio signals), and, comprised in said second hearing device, a second sound memory, which comprises a second set of data, which describe audio signals, wherein said first and second sets of data are substantially different from each other. This way, both hearing devices contain data describing (different) audio signals, and, accordingly, the required storage space in each hearing device can be (up to) halved (with respect to storing the same data in each hearing device). If, e.g., the user makes, at the second hearing device, a change, which is to be acknowledged (on both sides), and the appropriate acknowledge sound is stored in the second hearing device, and the data describing the corresponding audio signals can be transmitted to the first hearing device, the sound generator can produce the corresponding audio signal, which finally can be transmitted to the second hearing device and transduced into signals acknowledging said manipulation (on both sides). In the case that said data describe acknowledge sounds, the data sets can be considered first and second sets of acknowledge sound descriptions (or representations).
  • In said data of said first and second sets, audio signals may be described, e.g., in a parametrized form (e.g., as a frequency to be played and a time for which it is to be played), or in a sampled form (e.g., compressed, like in MP3 format).
  • In one embodiment, at least one of said first and said second hearing devices comprises a storage unit containing information on the contents of said first sound memory and/or of said second sound memory. By means of such a storage unit, it can be found out, without any communication between the first and the second hearing devices, whether data describing a desired sound are stored in the first or in the second hearing device. It is, e.g., possible to have in said storage unit a list of the contents of the first and of the second sound memory indicating where to find data of desired audio signals. And it is also possible to have in said storage unit a list of the contents of the sound memory of only one hearing device (which may be the hearing device containing said storage unit, or the other hearing device). In the latter case, the existence and not-existence in the list can indicate where (i.e., in which hearing device) data of the desired audio signals can be found.
  • The storage unit may be a table or a look-up table.
  • Accordingly, said storage unit may be considered to contain information on whether audio signals to be sent to said output transducers can be obtained from data contained in said first sound memory (i.e., in said first set of data) or from data contained in said second sound memory (i.e., in said second set of data).
  • In one embodiment, the system furthermore comprises a second sound generator for generating second audio signals, which is comprised in said second hearing device, and wherein said second hearing device is adapted to transmitting said second audio signals to said first hearing device via said communication link. This way, both hearing devices comprise a sound generator, and sound generated in either sound generator may be transmitted to the other hearing device.
  • Said first audio signals, and also said second audio signals, may comprise or be acknowledge sound signals.
  • When only (sufficiently) deterministic processes are involved after the sound generation, the corresponding time delays in each hearing device can be calculated in advance, and corresponding delays may be used in order to let the user perceive the sounds in each ear with a well-defined time delay.
  • The method for operating a hearing system comprising a first and a second hearing device and a communication link connecting the two hearing devices comprises the steps of
    • generating first audio signals in said first hearing device;
    • transmitting said first audio signals to said second hearing device via said communication link; and
    • converting, in each of the two hearing devices, said first audio signals into signals to be perceived by a user of the hearing system.
  • The features of the method of synchronization in a hearing system corresponds to the features of the method for operating a hearing system. The method allows for a synchronized perception of signals to be perceived by the user in each ear.
  • The method may furthermore comprise the steps of
    • generating second audio signals in said second hearing device; and
    • transmitting said second audio signals to said first hearing device via said communication link.
  • In another method, wherein said first hearing device comprises a first sound memory, which comprises a first set of data, which describe audio signals (first and/or second audio signals), and said second hearing device comprises a second sound memory, which comprises a second set of data, which describe audio signals (first and/or second audio signals), and wherein said first and second sets of data are substantially different from each other, further comprises the step of
    • transmitting data from said second sound memory via said communication link to said first hearing device.
  • This method, wherein at least one of said first and said second hearing devices comprises a storage unit containing information on the contents of said first sound memory and (or and/or) of said second sound memory, may further comprise the steps of
    • retrieving from said storage unit the information whether audio signals to be converted into signals to be perceived by a user of the hearing system are to be obtained from data contained in said first sound memory or from data contained in said second sound memory;
    • retrieving, from the appropriate sound memory, data describing the appropriate audio signals.
  • If said sound memory data are contained in that one sound memory, which is located in that one hearing device, which does not comprise said storage unit, a command (control signal) can be transmitted (via said communication link) to the other hearing device (which comprises the sound memory comprising said data) in order to retrieve the desired sound memory data.
  • The advantages of the methods correspond to the advantages of corresponding systems.
  • Further preferred embodiments and advantages emerge from the dependent claims and the figures.
  • Brief Description of the Drawings
  • Below, the invention is described in more detail by means of examples and the included drawings. The figures show:
  • Fig. 1
    a schematic diagram of a hearing system;
    Fig. 2
    a schematic diagram of a hearing system;
    Fig. 3
    a block diagram of a method of operating a hearing system.
  • The reference symbols used in the figures and their meaning are summarized in the list of reference symbols. Generally, alike or alike-functioning parts are given the same or similar reference symbols. The described embodiments are meant as examples and shall not confine the invention.
  • Detailed Description of the Invention
  • Figs. 1 and 2 show schematic diagrams of hearing systems 1. A hearing system 1 comprises two hearing devices 10L and 10R, which may be similarly or identically structured. The hearing device 10L of Figs. 1 and 2 may comprise some more elements, in particular such, which are only drawn in the first hearing device 10R.
  • In the Figs. 1 and 2, audio signals (audio data streams) are drawn as solid arrows; control signals or data are represented by dashed arrows. Not all possible signal flows are indicated, but only some, which are required to explain the invention.
  • The hearing devices 10R, 10L comprise controllers or control softwares 15L, 15R. For normal operation, the hearing devices 10L, 10R comprise input transducers 11L, 11R, typically microphones, which convert incoming sound 5 into audio signals (analogue and/or digital electrical signals). These audio signals may be processed in signal processors 13L, 13R, preferably digital signal processors (DSP), and are possibly amplified in amplifiers 14L, 14R before being converted into signals 7,8 to be perceived by a user of the hearing system 1 by output transducers 12L,12R, typically loudspeakers.
  • In Fig. 1, a user interface 16R, a sound generator 17R and a transmitter Tr of hearing device 10R are drawn. Hearing device 10L comprises a receiver Re.
  • Although drawn as separate units, the DSP 13R and the amplifier 14R may be integrated within the same software or electrical unit. In addition, also the controller or control software 15R may be integrally formed with DSP 13R and/or amplifier 14R. The hearing device controlling software 15L, 15R may be implemented in a separate processor, but does not have to. The same applies for the corresponding elements of the hearing device 10L.
  • In Fig. 1, it is sketched, how a concurrent or (practically) jitter-free perception by the user of an acknowledge sound, e.g. upon a volume change in hearing device 10R, can be achieved.
  • The user manipulates the user interface 16R in order to change the volume (loudness) of signals 7,8 he perceives (typically the loudness in both hearing devices shall be changed). This generates a control signal, whereupon the sound generator 17R is instructed by the controller 15R to generate an appropriate sound message (acknowledge signal), which is referred to as first audio signals 30. The first audio signals 30 are, on the one hand, fed to the output transducer 12R (e.g., via DSP 13R and amplifier 14R, or directly, or in between the two) to be converted into signals 7 to be perceived by the user, and on the other hand, fed to the transmitter Tr of the hearing device 10R. Instead of adding the first audio signals 30 to audio signals obtained from incoming sound 5, they may as well replace those audio signals.
  • By means of a communication link 20 involving the transmitter Tr of hearing device 10R and the receiver Re of hearing device 10L, the first audio signals 30 are made available in the hearing device 10L, where they are, in analogy to the hearing device 10R, converted into signals 8 to be perceived by the user.
  • It is possible to foresee a delay unit in the signal path (at least in the hearing device 10R), which allows for compensating and/or introducing time delays between the two hearing devices 10L,10R. For example, if processing (and amplifying) steps are the same for the.first audio signals 30 in both hearing devices 10L, 10R, the only step causing a time delay between the two hearing devices is the transmission of the first audio signals 30 via the communication link 20. For encoding (in transmitter Tr), transmitting and decoding (in receiver Re), a pre-determinable delay of the order of, e.g., some milliseconds is introduced, which may have a jitter of the order of some microseconds to 10 µs or 30 µs. This delay can by compensated by such a delay unit. And additional delays between both sides (typically of the order of some 10 µs to some 100 µs) can be willingly introduced in order to simulate that the perceived signal 7,8 originates from a certain (predetermined) direction.
  • Of course, in order to improve such a simulation (virtual sound source), the (first) audio signals may be processed using Head-Related Transfer Functions (HRTF) in the sense that not only interaural time differences (ITD) are applied as described above, but also interaural level differences (ILD), or, more generally speaking, frequency dependent interaural amplitude and phase differences.
  • Fig. 2 shows, in the manner of Fig. 1, a schematic diagram of another hearing system 1. The embodiment is similar to the one of Fig. 1, and mainly the additional or different features will be described.
  • Both hearing devices 10L, 10R comprise a sound memory 18L, 18R, in each of which a set of data describing audio signals (e.g., acknowledge sounds and/or speech sounds) are contained. For example, the sound memory 18R may contain (data describing) acknowledge sounds, which are to be played when the volume is increased, when the volume is decreased, or when the battery level is low, whereas in the sound memory 18L, (data describing) acknowledge sounds may be stored, which acknowledge a change from one hearing program to another hearing program.
  • Furthermore, both hearing devices 10L, 10R contain storage units 19L,19R, by means of which the respective controller 15L,15R can find out, whether a desired audio signal is stored in the sound memory 18L of the hearing device 10L or in the sound memory 18R of hearing device 10R.
  • Both hearing devices comprise a transmitter Tr and a receiver Re each, thus enabling a bidirectional communication link 20.
  • In Fig. 2, it is sketched, how a concurrent or (practically) jitter-free perception by the user of an acknowledge sound, e.g. upon a volume change in hearing device 10L, can be achieved.
  • The user manipulates the second hearing device's user interface 16L. This is noted by the controller 15L. The controller checks the storage unit (table, LUT) 19L and finds out, that the appropriate acknowledge sound (first audio signals 30) is not stored in the sound memory 18L, but in the sound memory 18R of the contralateral hearing device 10R. Thereupon, a control signal is transmitted via the second hearing device's transmitter Tr and the first hearing device's receiver Re to the second hearing device's controller 15R. The controller 15R ensures that the requested data 40 representing (describing) the requested first audio signals 30 are loaded from the sound memory 18R and that the requested audio signals 30 are generated. The requested audio signals 30 are fed towards the output transducer 12R and, in addition, transmitted to the second hearing device 10L via the communication link 20, from where it is transmitted to the output transducer 12L
  • Of course, also the second hearing device 10L may comprise a sound generator.
  • In conjunction with one sound memory or with sound memories 18L,18R, at least the following embodiments are possible:
    • only the first hearing device 10R comprises a sound generator 17R, and only the first hearing device 10R comprises a sound memory 18R;
    • only the first hearing device 10R comprises a sound generator 17R, and only the second hearing device 10L comprises a sound memory 18R (this requires always the transmission of data from hearing device 10L to hearing device 10R when a stored audio signal is to be played);
    • only the first hearing device 10R comprises a sound generator 17R, and both hearing devices 10L,10R comprise a sound memory 18L,18R;
    • both hearing devices 10L,10R comprise a sound generator 17L, 17R, and only the first hearing device 10R comprises a sound memory 18R;
    • both hearing devices 10L, 10R comprise a sound generator 17L,17R, and both hearing devices 10L, 10R comprise a sound memory 1BL, 18R.
  • Depending on where data are stored and where they are generated, data and/or audio signals have to be transmitted via the communication link 20.
  • It is possible to foresee a storage unit 19 in only one hearing device instead of in both.
  • Fig. 3 shows a block diagram of a simple method of operating a hearing system 1. The steps 100 - 300 are self-explaining.
  • The invention enables for a deterministic (predictable) latency and negligible jitter for signals to be perceived by the user's two ears.
  • The invention is particularly useful, when speech signals or more complex sounds are to be generated and presented to the user. The complexity of a sound may manifest in its (large) spectral content, its structure in time or rhythmic or percussive structure. Speech sounds may be used for guiding the user, informing the user and acknowledging events in the hearing system. Requirements for latency and jitter are less tough for simple sounds, like sine tones that fade in and fade out.
  • List of Reference Symbol
  • An appended "R" in a reference symbol indicates that the respective element is part of the first hearing device 10R; an appended "L" in a reference symbol indicates that the respective element is part of the second hearing device 10L.
  • 1
    hearing system, binaural hearing system
    5
    incoming sound, sound waves
    7
    signals to be perceived by a user of the hearing system (from the first hearing device 10R)
    8
    signals to be perceived by a user of the hearing system (from the second hearing device 10L)
    10
    hearing device
    11
    input transducer, microphone
    12
    output transducer, loudspeaker
    13
    signal processor, digital signal processor, DSP
    14
    amplifier
    15
    controller, control software, control part of hearing device software, hearing device controlling software
    16
    user interface
    17
    sound generator
    18
    sound memory
    19
    storage unit, table, lookup table
    20
    communication link
    30
    first audio signals
    40
    data describing first audio signals
    100,200,300
    steps
    LUT
    lookup table
    Tr
    transmitter
    Re
    receiver

Claims (11)

  1. Binaural hearing system (1) comprising
    - a first hearing device (10R) and a second hearing device (10L), each comprising an input transducer (11L, 11R) for receiving incoming signals (5) and for converting said incoming signals (5) into input audio signals and an output transducer (12L,12R) for converting audio signals into signals (7,8) to be perceived by a user of the hearing system;
    - a communication link (20) interconnecting said first (10R) and said second (10L) hearing device; and
    - a sound generator (17R) comprised in said first hearing device (10R) for generating sound messages, in particular acknowledge sound signals, as first audio signals (30) in said first hearing device (10R);
    characterized in that
    said first hearing device (10R) is adapted to transmit said first audio signals (30) to said second hearing device (10L) via said communication link (20).
  2. System (1) according to claim 1, adapted such that either one or both of said first audio signals (30) and said input audio signals are fed as audio signals to the output transducer (12R) of the first hearing device (10R) to be converted into said signals (7) to be perceived by the user of the hearing system.
  3. System (1) according to claim 1 or claim 2, comprising, comprised in said first hearing device (10R), a first sound memory (18R), which comprises a first set of data, which describe audio signals, and, comprised in said second hearing device (10L), a second sound memory (18L), which comprises a second set of data, which describe audio signals, wherein said first and second sets of data are substantially different from each other.
  4. System (1) according to claim 3, wherein at least one of said first (10R) and said second (10L) hearing devices comprises a storage unit (19L,19R) containing information on the contents of said first sound memory (18R) and/or of said second sound memory (18L).
  5. System (1) according to one of the preceding claims, furthermore comprising a second sound generator (17L) for generating second audio signals, which is comprised in said second hearing device (10L), and wherein said second hearing device is adapted to transmitting said second audio signals to said first hearing device via said communication link (20).
  6. System (1) according to claim 5, wherein said second audio signals comprise acknowledge sound signals.
  7. Method for operating a hearing system (1) comprising a first (10R) and a second (10L) hearing device and a communication link (20) connecting the two hearing devices (10L, 10R), said method comprising the steps of
    - converting incoming signals (5) into input audio signals by an input transducer (11R) comprised in said first hearing device (10R);
    - generating by a sound generator (17R) sound messages, in particular acknowledge sound signals, as first audio signals (30) in said first hearing device (10R);
    - transmitting said first audio signals (30) to said second hearing device (10L) via said communication link (20); and
    - converting, in each of the two hearing devices (10L,10R), said first audio signals (30) into signals (7,8) to be perceived by a user of the hearing system.
  8. Method according to claim 7, further comprising the step of
    - feeding either one or both of said first audio signals (30) and said input audio signals as audio signals to an output transducer (12R) of the first hearing device (10R) to be converted into said signals (7) to be perceived by the user of the hearing system.
  9. Method according to claim 7 or 8, further comprising the steps of
    - generating second audio signals in said second hearing device (10L); and
    - transmitting said second audio signals to said first hearing device (10R) via said communication link (20).
  10. Method according to one of claims 7 to 9, wherein said first hearing device (10R) comprises a first sound memory (18R), which comprises a first set of data, which describe audio signals, and said second hearing device (10L) comprises a second sound memory (18L), which comprises a second set of such data, which describe audio signals, and wherein said first and second sets of data are substantially different from each other, further comprising the step of
    - transmitting data from said second sound memory (18L) via said communication link (20) to said first hearing device (10R).
  11. Method according to claim 10, wherein at least one of said first (10R) and said second (10L) hearing devices comprises a storage unit (19R;19L) containing information on the contents of said first sound memory (18R) and/or of said second sound memory (18L), further comprising the steps of
    - retrieving from said storage unit (19R;19L) the information whether audio signals (7,8) to be converted into signals to be perceived by a user of the hearing System (1) are to be obtained from data contained in said first sound memory (18R) or from data contained in said second sound memory (18L);
    - retrieving from the appropriate sound memory data (18R;18L) describing the appropriate audio signals.
EP05027729.2A 2005-12-19 2005-12-19 Synchronization of sound generated in binaural hearing system Active EP1651005B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP05027729.2A EP1651005B1 (en) 2005-12-19 2005-12-19 Synchronization of sound generated in binaural hearing system
DK05027729.2T DK1651005T3 (en) 2005-12-19 2005-12-19 Synchronization of sound generated in binaural hearing aid system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP05027729.2A EP1651005B1 (en) 2005-12-19 2005-12-19 Synchronization of sound generated in binaural hearing system

Publications (3)

Publication Number Publication Date
EP1651005A2 EP1651005A2 (en) 2006-04-26
EP1651005A3 EP1651005A3 (en) 2006-07-05
EP1651005B1 true EP1651005B1 (en) 2017-03-22

Family

ID=35789290

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05027729.2A Active EP1651005B1 (en) 2005-12-19 2005-12-19 Synchronization of sound generated in binaural hearing system

Country Status (2)

Country Link
EP (1) EP1651005B1 (en)
DK (1) DK1651005T3 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008006772A2 (en) * 2006-07-12 2008-01-17 Phonak Ag Method for operating a binaural hearing system as well as a binaural hearing system
EP1885156B1 (en) 2006-08-04 2013-04-24 Siemens Audiologische Technik GmbH Hearing-aid with audio signal generator
DK2119310T3 (en) 2007-01-22 2017-02-13 Sonova Ag SYSTEM AND METHOD INTENDED TO PROVIDE HEARING AID TO A USER
DE102007015223B4 (en) 2007-03-29 2013-08-22 Siemens Audiologische Technik Gmbh Method and device for reproducing synthetically generated signals by a binaural hearing system
US8477975B2 (en) 2007-12-20 2013-07-02 Phonak Ag Hearing system with joint task scheduling
EP2547122A4 (en) * 2010-03-08 2013-01-16 Panasonic Corp Hearing aid

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE383730T1 (en) * 1998-02-18 2008-01-15 Widex As BINAURAL DIGITAL HEARING AID SYSTEM
EP0941014B1 (en) 1998-03-03 2006-01-04 Siemens Audiologische Technik GmbH Hearing aid system with two hearing aid devices
US7254246B2 (en) 2001-03-13 2007-08-07 Phonak Ag Method for establishing a binaural communication link and binaural hearing devices
DE102004035046A1 (en) 2004-07-20 2005-07-21 Siemens Audiologische Technik Gmbh Hearing aid or communication system with virtual signal sources providing the user with signals from the space around him
DE102004035256B3 (en) * 2004-07-21 2005-09-22 Siemens Audiologische Technik Gmbh Hearing aid system and method for operating a hearing aid system with audio reception

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP1651005A2 (en) 2006-04-26
DK1651005T3 (en) 2017-07-10
EP1651005A3 (en) 2006-07-05

Similar Documents

Publication Publication Date Title
US8712063B2 (en) Synchronization of sound generated in binaural hearing system
US10469960B2 (en) Method and apparatus for a binaural hearing assistance system using monaural audio signals
EP1843633A2 (en) Time-delay hearing instrument system and method
US8019386B2 (en) Companion microphone system and method
EP1651005B1 (en) Synchronization of sound generated in binaural hearing system
US7167571B2 (en) Automatic audio adjustment system based upon a user's auditory profile
US20110188662A1 (en) Method of rendering binaural stereo in a hearing aid system and a hearing aid system
CN117596516A (en) Audio playing system
WO2005125269A1 (en) First person acoustic environment system and method
US9294861B2 (en) Audio signal processing device
JP5872449B2 (en) System, hearing aid, and method for improving synchronicity of audio signal to video display
US10952005B1 (en) Stereo paired speaker system with center extraction
EP2273800A1 (en) Hearing device with virtual sound source
US10104480B2 (en) Method and facility for reproducing synthetically generated signals by means of a binaural hearing system
WO2008119122A1 (en) An acoustically transparent earphone
US20070127750A1 (en) Hearing device with virtual sound source
CN114946194A (en) Wireless MIDI earphone
JP2021536207A (en) Hearing device environment Methods, systems, and hearing devices for enhancing audio signals
US12008287B2 (en) Audio output device with hardware volume control
CN216291417U (en) Head stereo device
JP3076813U (en) Ear monitor system
CN114390388A (en) Audio playing device and related control method
CN112437385A (en) Earphone set
CN113709655A (en) Mobile terminal, playing system and playing method
JPH08237764A (en) Remote control transmitter-receiver

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

RIN1 Information on inventor provided before grant (corrected)

Inventor name: GLATT, RAOUL

Inventor name: LAICH, ROLF

Inventor name: KNAUS, MICHA

Inventor name: ROECK, HANS-UELI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

17P Request for examination filed

Effective date: 20061014

17Q First examination report despatched

Effective date: 20061120

AKX Designation fees paid

Designated state(s): CH DE DK LI

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SONOVA AG

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 5/033 20060101ALN20160928BHEP

Ipc: H04R 25/00 20060101AFI20160928BHEP

INTG Intention to grant announced

Effective date: 20161019

RIN1 Information on inventor provided before grant (corrected)

Inventor name: LAICH, ROLF

Inventor name: KNAUS, MICHA

Inventor name: ROECK, HANS-UELI

Inventor name: GLATT, RAOUL

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): CH DE DK LI

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602005051554

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20170704

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602005051554

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20180102

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171231

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171231

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DK

Payment date: 20221227

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20221228

Year of fee payment: 18