EP3461148A2 - Hearing assistance system with own voice detection - Google Patents
Hearing assistance system with own voice detection Download PDFInfo
- Publication number
- EP3461148A2 EP3461148A2 EP18195310.0A EP18195310A EP3461148A2 EP 3461148 A2 EP3461148 A2 EP 3461148A2 EP 18195310 A EP18195310 A EP 18195310A EP 3461148 A2 EP3461148 A2 EP 3461148A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- voice
- wearer
- microphone
- output information
- hearing assistance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 230000003044 adaptive effect Effects 0.000 claims description 69
- 238000000034 method Methods 0.000 claims description 39
- 210000000613 ear canal Anatomy 0.000 description 35
- 230000008569 process Effects 0.000 description 18
- 230000005236 sound signal Effects 0.000 description 12
- 230000004044 response Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000009467 reduction Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 208000032041 Hearing impaired Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 210000003625 skull Anatomy 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 238000013022 venting Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/552—Binaural
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/40—Arrangements for obtaining a desired directivity characteristic
- H04R25/407—Circuits for combining signals of a plurality of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/40—Arrangements for obtaining a desired directivity characteristic
- H04R25/405—Arrangements for obtaining a desired directivity characteristic by combining a plurality of transducers
Definitions
- This application relates to hearing assistance systems, and more particularly, to hearing assistance systems with own voice detection.
- Hearing assistance devices are electronic devices that amplify sounds above the audibility threshold to is hearing impaired user. Undesired sounds such as noise, feedback and the user's own voice may also be amplified, which can result in decreased sound quality and benefit for the user. It is undesirable for the user to hear his or her own voice amplified. Further, if the user is using an ear mold with little or no venting, he or she will experience an occlusion effect where his or her own voice sounds hollow ("talking in a barrel"). Thirdly, if the hearing aid has a noise reduction/environment classification algorithm, the user's own voice can be wrongly detected as desired speech.
- One proposal to detect voice adds a bone conductive microphone to the device.
- the bone conductive microphone can only be used to detect the user's own voice, has to make a good contact to the skull in order to pick up the own voice, and has a low signal-to-noise ratio.
- Another proposal to detect voice adds a directional microphone to the hearing aid, and orients the microphone toward the mouth of the user to detect the user's voice.
- the effectiveness of the directional microphone depends on the directivity of the microphone and the presence of other sound sources, particularly sound sources in the same direction as the mouth.
- Another proposal to detect voice provides a microphone in the ear-canal and only uses the microphone to record an occluded signal.
- Another proposal attempts to use a filter to distinguish the user's voice from other sound. However, the filter is unable to self correct to accommodate changes in the user's voice and for changes in the environment of the user.
- the present subject matter provides apparatus and methods to use a hearing assistance device to detect a voice of the wearer of the hearing assistance device.
- Embodiments use an adaptive filter to provide a self-correcting voice detector, capable of automatically adjusting to accommodate changes in the wearer's voice and environment.
- Examples are provided, such as an apparatus configured to be worn by a wearer who has an ear and an ear canal.
- the apparatus includes a first microphone adapted to be worn about the ear of the person, a second microphone adapted to be worn about the ear canal of the person and at a different location than the first microphone, a sound processor adapted to process signals from the first microphone to produce a processed sound signal, and a voice detector to detect the voice of the wearer.
- the voice detector includes an adaptive filter to receive signals from the first microphone and the second microphone.
- an apparatus includes a housing configured to be worn behind the ear or over the ear, a first microphone in the housing, and an ear piece configured to be positioned in the ear canal, wherein the ear piece includes a microphone that receives sound from the outside when positioned near the ear canal.
- Various voice detection systems employ an adaptive filter that receives signals from the first microphone and the second microphone and detects the voice of the wearer using a peak value for coefficients of the adaptive filter and an error signal from the adaptive filter.
- the present subject matter also provides methods for detecting a voice of a wearer of a hearing assistance device where the hearing assistance device includes a first microphone and a second microphone.
- An example of the method is provided and includes using a first electrical signal representative of sound detected by the first microphone and a second electrical signal representative of sound detected by the second microphone as inputs to a system including an adaptive filter, and using the adaptive filter to detect the voice of the wearer of the hearing assistance device.
- the present subject matter further provides apparatus and methods to use a pair of left and right hearing assistance devices to detect a voice of the wearer of the pair of left and right hearing assistance devices.
- Embodiments use outcome of detection of the voice of the wearer performed by the left hearing assistance device and the outcome of detection of the voice of the wearer performed the right hearing assistance device to determine whether to declare a detection of the voice of the wearer.
- Various embodiments disclosed herein provide a self-correcting voice detector, capable of reliably detecting the presence of the user's own voice through automatic adjustments that accommodate changes in the user's voice and environment.
- the detected voice can be used, among other things, to reduce the amplification of the user's voice, control an anti-occlusion process and control an environment classification process.
- the present subject matter provides, among other things, an "own voice" detector using two microphones in a standard hearing assistance device.
- standard hearing aids include behind-the-ear (BTE), over-the-ear (OTE), and receiver-in-canal (RIC) devices.
- BTE behind-the-ear
- OFT over-the-ear
- RIC receiver-in-canal
- RIC devices have a housing adapted to be worn behind the ear or over the ear.
- the RIC electronics housing is called a BTE housing or an OTE housing.
- one microphone is the microphone as usually present in the standard hearing assistance device, and the other microphone is mounted in an ear bud or ear mold near the user's ear canal.
- the microphone is directed to detection of acoustic signals outside and not inside the ear canal.
- the two microphones can be used to create a directional signal.
- FIG. 1A illustrates a hearing assistance device with a voice detector according to one embodiment of the present subject matter.
- the figure illustrates an ear with a hearing assistance device 100, such as a hearing aid.
- the illustrated hearing assistance device includes a standard housing 101 (e.g. behind-the-ear (BTE) or on-the-ear (OTE) housing) with an optional ear hook 102 and an ear piece 103 configured to fit within the ear canal.
- a first microphone (MIC 1) is positioned in the standard housing 101
- a second microphone (MIC 2) is positioned near the ear canal 104 on the air side of the ear piece.
- FIG. 1B schematically illustrates a cross section of the ear piece 103 positioned near the ear canal 104, with the second microphone on the air side of the ear piece 103 to detect acoustic signals outside of the ear canal.
- the first microphone (M1) is adapted to be worn about the ear of the person and the second microphone (M2) is adapted to be worn about the ear canal of the person.
- the first and second microphones are at different locations to provide a time difference for sound from a user's voice to reach the microphones. As illustrated in FIG. 2 , the sound vectors representing travel of the user's voice from the user's mouth to the microphones are different.
- the first microphone (MIC 1) is further away from the mouth than the second microphone (MIC 2). Sound received by MIC 2 will be relatively high amplitude and will be received slightly sooner than sound detected by MIC 1. And when the wearer is speaking, the sound of the wearer's voice will dominate the sounds received by both MIC 1 and MIC 2. The differences in received sound can be used to distinguish the own voice from other sound sources.
- FIG. 3 illustrates a hearing assistance device according to one embodiment of the present subject matter.
- the illustrated device 305 includes the first microphone (MIC 1), the second microphone (MIC 2), and a receiver (speaker) 306.
- each microphone is an omnidirectional microphone.
- each microphone is a directional microphone.
- the microphones may be both directional and omnidirectional.
- Various order directional microphones can be employed.
- Various embodiments incorporate the receiver in a housing of the device (e.g. behind-the-ear or on-the-ear housing).
- a sound conduit can be used to direct sound from the receiver toward the ear canal.
- Various embodiments use a receiver configured to fit within the user's ear canal. These embodiments are referred to as receiver-in-canal (RIC) devices.
- RIC receiver-in-canal
- a digital sound processing system 308 processes the acoustic signals received by the first and second microphones, and provides a signal to the receiver 306 to produce an audible signal to the wearer of the device 305.
- the illustrated digital sound processing system 308 includes an interface 307, a sound processor 308, and a voice detector 309.
- the illustrated interface 307 converts the analog signals from the first and second microphones into digital signals for processing by the sound processor 308 and the voice detector 309.
- the interface may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by the sound processor and voice detector.
- the illustrated sound processor 308 processes a signal representative of a sound received by one or both of the first microphone and/or second microphone into a processed output signal 310, which is provided to the receiver 306 to produce the audible signal.
- the sound processor 308 is capable of operating in a directional mode in which signals representative of sound received by the first microphone and sound received by the second microphone are processed to provide the output signal 310 to the receiver 306 with directionality.
- the voice detector 309 receives signals representative of sound received by the first microphone and sound received by the second microphone.
- the voice detector 309 detects the user's own voice, and provides an indication 311 to the sound processor 308 regarding whether the user's own voice is detected. Once the user's own voice is detected any number of possible other actions can take place.
- the sound processor 308 can perform one or more of the following, including but not limited to reduction of the amplification of the user's voice, control of an anti-occlusion process, and/or control of an environment classification process. Those skilled in the art will understand that other processes may take place without departing from the scope of the present subject matter.
- the voice detector 309 includes an adaptive filter.
- adaptive filters include Recursive Least Square error (RLS), Least Mean Squared error (LMS), and Normalized Least Mean Square error (NLMS) adaptive filter processes.
- the desired signal for the adaptive filter is taken from the first microphone (e.g., a standard behind-the-ear or over-the-ear microphone), and the input signal to the adaptive filter is taken from the second microphone. If the hearing aid wearer is talking, the adaptive filter models the relative transfer function between the microphones.
- Voice detection can be performed by comparing the power of the error signal to the power of the signal from the standard microphone and/or looking at the peak strength in the impulse response of the filter.
- the amplitude of the impulse response should be in a certain range in order to be valid for the own voice. If the user's own voice is present, the power of the error signal will be much less than the power of the signal from the standard microphone, and the impulse response has a strong peak with an amplitude above a threshold (e.g. above about 0.5 for normalized coefficients). In the presence of the user's own voice, the largest normalized coefficient of the filter is expected to be within the range of about 0.5 to about 0.9. Sound from other noise sources would result in a much smaller difference between the power of the error signal and the power of the signal from the standard microphone, and a small impulse response of the filter with no distinctive peak
- FIG. 4 illustrates a voice detector according to one embodiment of the present subject matter.
- the illustrated voice detector 409 includes an adaptive filter 412, a power analyzer 413 and a coefficient analyzer 414.
- the output 411 of the voice detector 409 provides an indication to the sound processor indicative of whether the user's own voice is detected.
- the illustrated adaptive filter includes an adaptive filter process 415 and a summing junction 416.
- the desired signal 417 for the filter is taken from a signal representative of sound from the first microphone, and the input signal 418 for the filter is taken from a signal representative of sound from the second microphone.
- the filter output signal 419 is subtracted from the desired signal 417 at the summing junction 416 to produce an error signal 420 which is fed back to the adaptive filter process 415.
- the illustrated power analyzer 413 compares the power of the error signal 420 to the power of the signal representative of sound received from the first microphone. According to various embodiments, a voice will not be detected unless the power of the signal representative of sound received from the first microphone is much greater than the power of the error signal. For example, the power analyzer 413 compares the difference to a threshold, and will not detect voice if the difference is less than the threshold.
- the illustrated coefficient analyzer 414 analyzes the filter coefficients from the adaptive filter process 415. According to various embodiments, a voice will not be detected unless a peak value for the coefficients is significantly high. For example, some embodiments will not detect voice unless the largest normalized coefficient is greater than a predetermined value (e.g. 0.5).
- a predetermined value e.g. 0.5
- FIGS. 5-7 illustrate various processes for detecting voice that can be used in various embodiments of the present subject matter.
- the power of the error signal from the adaptive filter is compared to the power of a signal representative of sound received by the first microphone.
- the threshold is selected to be sufficiently high to ensure that the power of the first microphone is much greater than the power of the error signal.
- voice is detected at 523 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold, and voice is not detected at 524 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold.
- coefficients of the adaptive filter are analyzed.
- voice is detected at 623 if the largest normalized coefficient is greater than a predetermined value, and voice is not detected at 624 if the largest normalized coefficient is not greater than a predetermined value.
- the power of the error signal from the adaptive filter is compared to the power of a signal representative of sound received by the first microphone.
- voice is not detected at 724 if the power of the first microphone is not greater than the power of the error signal by a predetermined threshold. If the power of the error signal is too large, then the adaptive filter has not converged.
- the coefficients are not analyzed until the adaptive filter converges.
- coefficients of the adaptive filter are analyzed if the power of the first microphone is greater than the power of the error signal by a predetermined threshold.
- the largest normalized coefficient is greater than a predetermined value, such as greater than 0.5.
- voice is not detected at 724 if the largest normalized coefficient is not greater than a predetermined value.
- Voice is detected at 723 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold and if the largest normalized coefficient is greater than a predetermined value.
- FIG. 8 illustrates one embodiment of the present subject matter with an "own voice detector" to control active noise canceller for occlusion reduction.
- the active noise canceller filters microphone M2 with filter h and sends the filtered signal to the receiver.
- the microphone M2 and the error microphone M3 (in the ear canal) are used to calculate the filter update for filter h.
- the own voice detector which uses microphone M1 and M2, is used to steer the stepsize in the filter update.
- FIG. 9 illustrates one embodiment of the present subject matter offering a multichannel expansion, compression and output control limiting algorithm (MECO) which uses the signal of microphone M2 to calculate the desired gain and subsequently applies that gain to microphone signal M2 and then sends the amplified signal to the receiver. Additionally, the gain calculation can take into account the outcome of the own voice detector (which uses M1 and M2) to calculate the desired gain. If the wearer's own voice is detected, the gain in the lower channels (typically below 1 KHz) will be lowered to avoid occlusion. Note: the MECO algorithm can use microphone signal M1 or M2 or a combination of both.
- MECO multichannel expansion, compression and output control limiting algorithm
- FIG. 10 illustrates one embodiment of the present subject matter which uses an "own voice detector" in an environment classification scheme. From the microphone signal M2, several features are calculated. These features together with the result of the own voice detector, which uses M1 and M2, are used in a classifier to determine the acoustic environment. This acoustic environment classification is used to set the gain in the hearing aid.
- the hearing aid may use M2 or M1 or M1 and M2 for the feature calculation.
- FIG. 11 illustrates a pair of hearing assistance devices according to one embodiment of the present subject matter.
- the pair of hearing assistance devices includes a left hearing assistance device 1105L and a right hearing assistance device 1105R, such as a left hearing aid and a right hearing aid.
- the left hearing assistance device 1105L is configured to be worn in or about the left ear of a wearer for delivering sound to the left ear canal of the wearer.
- the right hearing assistance device 1105R is configured to be worn in or about the right ear of the wearer for delivering sound to the right ear canal of the wearer.
- the left and right hearing assistance devices 1105L and 1105R each represent an embodiment of the device 305 as discussed above with capability of performing wireless communication between each other and uses voice detection capability of both devices to determine whether voice of the wearer is present.
- the illustrated left hearing assistance device 1105L includes a first microphone MIC 1L, a second microphone MIC 2L, an interface 1107L, a sound processor 1108L, a receiver 1106L, a voice detector 1109L, and a communication circuit 1130L.
- the first microphone MIC 1L produces a first left microphone signal.
- the second microphone MIC 2L produces a second left microphone signal.
- the first microphone MIC 1L is positioned about the left ear or the wearer
- the second microphone MIC 2L is positioned about the left ear canal of wearer, at a different location than the first microphone MIC 1L, on an air side of the left ear canal to detect signals outside the left ear canal.
- Interface 1107L converts the analog versions of the first and second left microphone signals into digital signals for processing by the sound processor 1108L and the voice detector 1109L.
- the interface 1107L may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by the sound processor 1108L and the voice detector 1109L.
- the sound processor 1108L produces a processed left sound signal 1110L.
- the left receiver 1106L produces a left audible signal based on the processed left sound signal 1110L and transmits the left audible signal to the left ear canal of the wearer.
- the sound processor 1108L produces the processed left sound signal 1110L based on the first left microphone signal.
- the sound processor 1108L produces the processed left sound signal 1110L based on the first left microphone signal and the second left microphone signal.
- the left voice detector 1109L detects a voice of the wearer using the first left microphone signal and the second left microphone signal. In one embodiment, in response to the voice of the wearer being detected based on the first left microphone signal and the second left microphone signal, the left voice detector 1109L produces a left detection signal indicative of detection of the voice of the wearer. In one embodiment, the left voice detector 1109L includes a left adaptive filter configured to output left information and identifies the voice of the wearer from the output left information. In various embodiments, the output left information includes coefficients of the left adaptive filter and/or a left error signal. In various embodiments, the left voice detector 1109L includes the voice detector 309 or the voice detector 409 as discussed above.
- the left communication circuit 1130L receives information from, and transmits information to, the right hearing assistance device 1105R via a wireless communication link 1132.
- the information transmitted via wireless communication link 1132 includes information associated with the detection of the voice of the wearer as performed by each of the left and right hearing assistance devices 1105L and 1105R.
- the illustrated right hearing assistance device 1105R includes a first microphone MIC 1R, a second microphone MIC 2R, an interface 1107R, a sound processor 1108R, a receiver 1106R, a voice detector 1109R, and a communication circuit 1130R.
- the first microphone MIC 1R produces a first right microphone signal.
- the second microphone MIC 2R produces a second right microphone signal.
- the first microphone MIC 1R is positioned about the right ear or the wearer
- the second microphone MIC 2R is positioned about the right ear canal of wearer, at a different location than the first microphone MIC 1R, on an air side of the right ear canal to detect signals outside the right ear canal.
- Interface 1107R converts the analog versions of the first and second right microphone signals into digital signals for processing by the sound processor 1108R and the voice detector 1109R.
- the interface 1107R may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by the sound processor 1108R and the voice detector 1109R.
- the sound processor 1108R produces a processed right sound signal 1110R.
- the right receiver 1106R produces a right audible signal based on the processed right sound signal 1110R and transmits the right audible signal to the right ear canal of the wearer.
- the sound processor 1108R produces the processed right sound signal 1110R based on the first right microphone signal.
- the sound processor 1108R produces the processed right sound signal 1110R based on the first right microphone signal and the second right microphone signal.
- the right voice detector 1109R detects the voice of the wearer using the first right microphone signal and the second right microphone signal. In one embodiment, in response to the voice of the wearer being detected based on the first right microphone signal and the second right microphone signal, the right voice detector 1109R produces a right detection signal indicative of detection of the voice of the wearer. In one embodiment, the right voice detector 1109R includes a right adaptive filter configured to output right information and identifies the voice of the wearer from the output right information. In various embodiments, the output right information includes coefficients of the right adaptive filter and/or a right error signal. In various embodiments, the right voice detector 1109R includes the voice detector 309 or the voice detector 409 as discussed above.
- the right communication circuit 1130R receives information from, and transmits information to, the right hearing assistance device 1105R via a wireless communication link 1132.
- At least one of the left voice detector 1109L and the right voice detector 1109R is configured to detect the voice of wearer using the first left microphone signal, the second left microphone signal, the first right microphone signal, and the second right microphone signal.
- signals produced by all of the microphones MIC 1L, MIC 2L, MIC 1R, and MIC 2R are used for determining whether the voice of the wearer is present.
- the left voice detector 1109L and/or the right voice detector 1109R declares a detection of the voice of the wearer in response to at least one of the left detection signal and the second detection signal being present.
- the left voice detector 1109L and/or the right voice detector 1109R declares a detection of the voice of the wearer in response to the left detection signal and the second detection signal both being present. In one embodiment, the left voice detector 1109L and/or the right voice detector 1109R determines whether to declare a detection of the voice of the wearer using the output left information and output right information.
- the output left information and output right information are each indicative of one or more detection strength parameters each being a measure of likeliness of actual existence of the voice of wearer. Examples of the one or more detection strength parameters include the difference between the power of the error signal and the power of the first microphone signal and the largest normalized coefficient of the adaptive filter.
- the left voice detector 1109L and/or the right voice detector 1109R determines whether to declare a detection of the voice of the wearer using a weighted combination of the output left information and the output eight information.
- the weighted combination of the output left information and the output right information can include a weighted sum of the detection strength parameters.
- the one or more detection strength parameters produced by each of the left and right voice detectors can be multiplied by one or more corresponding weighting factors before being added to produce the weighted sum.
- the weighting factors may be determined using a priori information such as estimates of the background noise and/or position(s) of other sound sources in a room.
- the detection of the voice of the wearer is performed using both the left and the right voice detectors such as detectors 1109L and 1109R.
- whether to declare a detection of the voice of the wearer may be determined by each of the left voice detector 1109L and the right voice detector 1109R, determined by the left voice detector 1109L and communicated to the right voice detector 1109R via wireless link 1132, or determined by the right voice detector 1109R and communicated to the left voice detector 1109L via wireless link 1132.
- the left voice detector 1109L transmits an indication 1111L to the sound processor 1108L
- the right voice detector 1109R transmits an indication 1111R to the sound processor 1108R.
- the sound processors 1108L and 1108R produce the processed sound signals 1110L and 1110R, respectively, using the indication that the voice of the wearer is detected.
- FIG. 12 illustrates a process for detecting voice using a pair of hearing assistance devices including a left hearing assistance device and a right hearing assistance device, such as the left and right hearing assistance devices 1105L and 1105R.
- voice of a wearer is detected using the left hearing assistance device.
- voice of a wearer is detected using the right hearing assistance device.
- steps 1241 and 1242 are performed concurrently or simultaneously. Examples for each of steps 1241 and 1242 include the processes illustrated in each of FIGS. 5-7 .
- whether to declare a detection of the voice of the wearer is determining using an outcome of both of the detections at 1241 and 1242.
- the left and right hearing assistance devices each include first and second microphones. Electrical signals produced by the first and second microphones of the left hearing assistance device are used as inputs to a voice detector of the left hearing assistance device at 1241.
- the voice detector of the left hearing assistance device includes a left adaptive filter. Electrical signals produced by the first and second microphones of the right hearing assistance device are used as inputs to a voice detector of the right hearing assistance device at 1242.
- the voice detector of the right hearing assistance device includes a right adaptive filter.
- the voice of the wearer is detected using information output from the left adaptive filter and information output from the right adaptive filter at 1243. In one embodiment, the voice of the wearer is detected using left coefficients of the left adaptive filter and right coefficients of the right adaptive filter.
- the voice of the wearer is detected using a left error signal produced by the left adaptive filter and a right error signal produced by the right adaptive filter.
- the voice of the wearer is detected using a left detection strength parameter of the information output from the left adaptive filter and a right detection strength parameter of the information output from the right adaptive filter.
- the left and right detection strength parameters are each a measure of likeliness of actual existence of the voice of wearer. Examples of the left detection strength parameter include the difference between the power of a left error signal produced by the left adaptive filter and the power of the electrical signal produced by the first microphone of the left hearing assistance device and the largest normalized coefficient of the left adaptive filter.
- Examples of the right detection strength parameter include the difference between the power of a right error signal produced by the right adaptive filter and the power of the electrical signal produced by the first microphone of the right hearing assistance device and the largest normalized coefficient of the right adaptive filter.
- the voice of the wearer is detected using a weighted combination of the information output from the left adaptive filter and the information output from the right adaptive filter.
- the voice of the wearer is detected using the left hearing assistance device based on the electrical signals produced by the first and second microphones of the left hearing assistance device, and a left detection signal indicative of whether the voice of the wearer is detected by the left hearing assistance device is produced, at 1241.
- the voice of the wearer is detected using the right hearing assistance device based on the electrical signals produced by the first and second microphones of the right hearing assistance device, and a right detection signal indicative of whether the voice of the wearer is detected by the right hearing assistance device is produced, at 1242. Whether to declare the detection of the voice of the wearer is determined using the left detection signal and the right detection signal at 1243.
- the detection of the voice of the wearer is declared in response to both of the left detection signal and the right detection signal being present. In another embodiment, the detection of the voice of the wearer is declared in response to at least one of the left detection signal and the right detection signal being present. In one embodiment, whether to declare the detection of the voice of the wearer is determined using the left detection signal, the right detection signal, and weighting factors applied to the left and right detection signals.
- each device of a pair of hearing assistance devices can be applied to each device of a pair of hearing assistance devices, with the declaration of the detection of the voice of the wearer being a result of detection using both devices of the pair of hearing assistance devices, as discussed with reference to FIGS. 11 and 12 .
- Such binaural voice detection will likely improve the acoustic perception of the wearer because both hearing assistance devices worn by the wearer are acting similarly when the wearer speaks.
- whether to declare a detection of the voice of the wearer may be determined based on the detection performed by either one device of the pair of hearing assistance devices or based on the detection performed by both devices of the pair of hearing assistance devices.
- An example of the pair of hearing assistance devices includes a pair of hearing aids.
- the present subject matter includes hearing assistance devices, and was demonstrated with respect to BTE, OTE, and RIC type devices, but it is understood that it may also be employed in cochlear implant type hearing devices. It is understood that other hearing assistance devices not expressly stated herein may fall within the scope of the present subject matter.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
Description
- This application relates to hearing assistance systems, and more particularly, to hearing assistance systems with own voice detection.
- Hearing assistance devices are electronic devices that amplify sounds above the audibility threshold to is hearing impaired user. Undesired sounds such as noise, feedback and the user's own voice may also be amplified, which can result in decreased sound quality and benefit for the user. It is undesirable for the user to hear his or her own voice amplified. Further, if the user is using an ear mold with little or no venting, he or she will experience an occlusion effect where his or her own voice sounds hollow ("talking in a barrel"). Thirdly, if the hearing aid has a noise reduction/environment classification algorithm, the user's own voice can be wrongly detected as desired speech.
- One proposal to detect voice adds a bone conductive microphone to the device. The bone conductive microphone can only be used to detect the user's own voice, has to make a good contact to the skull in order to pick up the own voice, and has a low signal-to-noise ratio. Another proposal to detect voice adds a directional microphone to the hearing aid, and orients the microphone toward the mouth of the user to detect the user's voice. However, the effectiveness of the directional microphone depends on the directivity of the microphone and the presence of other sound sources, particularly sound sources in the same direction as the mouth. Another proposal to detect voice provides a microphone in the ear-canal and only uses the microphone to record an occluded signal. Another proposal attempts to use a filter to distinguish the user's voice from other sound. However, the filter is unable to self correct to accommodate changes in the user's voice and for changes in the environment of the user.
- The present subject matter provides apparatus and methods to use a hearing assistance device to detect a voice of the wearer of the hearing assistance device. Embodiments use an adaptive filter to provide a self-correcting voice detector, capable of automatically adjusting to accommodate changes in the wearer's voice and environment.
- Examples are provided, such as an apparatus configured to be worn by a wearer who has an ear and an ear canal. The apparatus includes a first microphone adapted to be worn about the ear of the person, a second microphone adapted to be worn about the ear canal of the person and at a different location than the first microphone, a sound processor adapted to process signals from the first microphone to produce a processed sound signal, and a voice detector to detect the voice of the wearer. The voice detector includes an adaptive filter to receive signals from the first microphone and the second microphone.
- Another example of an apparatus includes a housing configured to be worn behind the ear or over the ear, a first microphone in the housing, and an ear piece configured to be positioned in the ear canal, wherein the ear piece includes a microphone that receives sound from the outside when positioned near the ear canal. Various voice detection systems employ an adaptive filter that receives signals from the first microphone and the second microphone and detects the voice of the wearer using a peak value for coefficients of the adaptive filter and an error signal from the adaptive filter.
- The present subject matter also provides methods for detecting a voice of a wearer of a hearing assistance device where the hearing assistance device includes a first microphone and a second microphone. An example of the method is provided and includes using a first electrical signal representative of sound detected by the first microphone and a second electrical signal representative of sound detected by the second microphone as inputs to a system including an adaptive filter, and using the adaptive filter to detect the voice of the wearer of the hearing assistance device.
- The present subject matter further provides apparatus and methods to use a pair of left and right hearing assistance devices to detect a voice of the wearer of the pair of left and right hearing assistance devices. Embodiments use outcome of detection of the voice of the wearer performed by the left hearing assistance device and the outcome of detection of the voice of the wearer performed the right hearing assistance device to determine whether to declare a detection of the voice of the wearer.
- This Summary is an overview of some of the teachings of the present application and is not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description. The scope of the present invention is defined by the appended claims and their legal equivalents.
-
-
FIGS. 1A and 1B illustrate a hearing assistance device with a voice detector according to one embodiment of the present subject matter. -
FIG. 2 demonstrates how sound can travel from the user's mouth to the first and second microphones illustrated inFIG. 1A . -
FIG. 3 illustrates a hearing assistance device according to one embodiment of the present subject matter. -
FIG. 4 illustrates a voice detector according to one embodiment of the present subject matter. -
FIGS. 5-7 illustrate various processes for detecting voice that can be used in various embodiments of the present subject matter. -
FIG. 8 illustrates one embodiment of the present subject matter with an "own voice detector" to control active noise canceller for occlusion reduction. -
FIG. 9 illustrates one embodiment of the present subject matter offering a multichannel expansion, compression and output control limiting algorithm (MECO). -
FIG. 10 illustrates one embodiment of the present subject matter which uses an "own voice detector" in an environment classification scheme. -
FIG. 11 illustrates a pair of hearing assistance devices according to one embodiment of the present subject matter. -
FIG. 12 illustrates a process for detecting voice using the pair of hearing assistance devices. - The following detailed description refers to subject matter in the accompanying drawings which show, by way of illustration, specific aspects and embodiments in which the present subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present subject matter. References to "an", "one", or "various" embodiments in this disclosure are not necessarily to the same embodiment, and such references contemplate more than one embodiment. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope is defined only by the appended claims, along with the full scope of legal equivalents to which such claims are entitled.
- Various embodiments disclosed herein provide a self-correcting voice detector, capable of reliably detecting the presence of the user's own voice through automatic adjustments that accommodate changes in the user's voice and environment. The detected voice can be used, among other things, to reduce the amplification of the user's voice, control an anti-occlusion process and control an environment classification process.
- The present subject matter provides, among other things, an "own voice" detector using two microphones in a standard hearing assistance device. Examples of standard hearing aids include behind-the-ear (BTE), over-the-ear (OTE), and receiver-in-canal (RIC) devices. It is understood that RIC devices have a housing adapted to be worn behind the ear or over the ear. Sometimes the RIC electronics housing is called a BTE housing or an OTE housing. According to various embodiments, one microphone is the microphone as usually present in the standard hearing assistance device, and the other microphone is mounted in an ear bud or ear mold near the user's ear canal. Hence, the microphone is directed to detection of acoustic signals outside and not inside the ear canal. The two microphones can be used to create a directional signal.
-
FIG. 1A illustrates a hearing assistance device with a voice detector according to one embodiment of the present subject matter. The figure illustrates an ear with ahearing assistance device 100, such as a hearing aid. The illustrated hearing assistance device includes a standard housing 101 (e.g. behind-the-ear (BTE) or on-the-ear (OTE) housing) with anoptional ear hook 102 and anear piece 103 configured to fit within the ear canal. A first microphone (MIC 1) is positioned in thestandard housing 101, and a second microphone (MIC 2) is positioned near theear canal 104 on the air side of the ear piece.FIG. 1B schematically illustrates a cross section of theear piece 103 positioned near theear canal 104, with the second microphone on the air side of theear piece 103 to detect acoustic signals outside of the ear canal. - Other embodiments may be used in which the first microphone (M1) is adapted to be worn about the ear of the person and the second microphone (M2) is adapted to be worn about the ear canal of the person. The first and second microphones are at different locations to provide a time difference for sound from a user's voice to reach the microphones. As illustrated in
FIG. 2 , the sound vectors representing travel of the user's voice from the user's mouth to the microphones are different. The first microphone (MIC 1) is further away from the mouth than the second microphone (MIC 2). Sound received byMIC 2 will be relatively high amplitude and will be received slightly sooner than sound detected byMIC 1. And when the wearer is speaking, the sound of the wearer's voice will dominate the sounds received by bothMIC 1 andMIC 2. The differences in received sound can be used to distinguish the own voice from other sound sources. -
FIG. 3 illustrates a hearing assistance device according to one embodiment of the present subject matter. The illustrateddevice 305 includes the first microphone (MIC 1), the second microphone (MIC 2), and a receiver (speaker) 306. It is understood that different types of microphones can be employed in various embodiments. In one embodiment, each microphone is an omnidirectional microphone. In one embodiment, each microphone is a directional microphone. In various embodiments, the microphones may be both directional and omnidirectional. Various order directional microphones can be employed. Various embodiments incorporate the receiver in a housing of the device (e.g. behind-the-ear or on-the-ear housing). A sound conduit can be used to direct sound from the receiver toward the ear canal. Various embodiments use a receiver configured to fit within the user's ear canal. These embodiments are referred to as receiver-in-canal (RIC) devices. - A digital
sound processing system 308 processes the acoustic signals received by the first and second microphones, and provides a signal to thereceiver 306 to produce an audible signal to the wearer of thedevice 305. The illustrated digitalsound processing system 308 includes aninterface 307, asound processor 308, and avoice detector 309. The illustratedinterface 307 converts the analog signals from the first and second microphones into digital signals for processing by thesound processor 308 and thevoice detector 309. For example, the interface may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by the sound processor and voice detector. The illustratedsound processor 308 processes a signal representative of a sound received by one or both of the first microphone and/or second microphone into a processedoutput signal 310, which is provided to thereceiver 306 to produce the audible signal. According to various embodiments, thesound processor 308 is capable of operating in a directional mode in which signals representative of sound received by the first microphone and sound received by the second microphone are processed to provide theoutput signal 310 to thereceiver 306 with directionality. - The
voice detector 309 receives signals representative of sound received by the first microphone and sound received by the second microphone. Thevoice detector 309 detects the user's own voice, and provides anindication 311 to thesound processor 308 regarding whether the user's own voice is detected. Once the user's own voice is detected any number of possible other actions can take place. For example, in various embodiments when the user's voice is detected, thesound processor 308 can perform one or more of the following, including but not limited to reduction of the amplification of the user's voice, control of an anti-occlusion process, and/or control of an environment classification process. Those skilled in the art will understand that other processes may take place without departing from the scope of the present subject matter. - In various embodiments, the
voice detector 309 includes an adaptive filter. Examples of processes implemented by adaptive filters include Recursive Least Square error (RLS), Least Mean Squared error (LMS), and Normalized Least Mean Square error (NLMS) adaptive filter processes. The desired signal for the adaptive filter is taken from the first microphone (e.g., a standard behind-the-ear or over-the-ear microphone), and the input signal to the adaptive filter is taken from the second microphone. If the hearing aid wearer is talking, the adaptive filter models the relative transfer function between the microphones. Voice detection can be performed by comparing the power of the error signal to the power of the signal from the standard microphone and/or looking at the peak strength in the impulse response of the filter. The amplitude of the impulse response should be in a certain range in order to be valid for the own voice. If the user's own voice is present, the power of the error signal will be much less than the power of the signal from the standard microphone, and the impulse response has a strong peak with an amplitude above a threshold (e.g. above about 0.5 for normalized coefficients). In the presence of the user's own voice, the largest normalized coefficient of the filter is expected to be within the range of about 0.5 to about 0.9. Sound from other noise sources would result in a much smaller difference between the power of the error signal and the power of the signal from the standard microphone, and a small impulse response of the filter with no distinctive peak -
FIG. 4 illustrates a voice detector according to one embodiment of the present subject matter. The illustratedvoice detector 409 includes an adaptive filter 412, apower analyzer 413 and acoefficient analyzer 414. Theoutput 411 of thevoice detector 409 provides an indication to the sound processor indicative of whether the user's own voice is detected. The illustrated adaptive filter includes anadaptive filter process 415 and a summingjunction 416. The desiredsignal 417 for the filter is taken from a signal representative of sound from the first microphone, and theinput signal 418 for the filter is taken from a signal representative of sound from the second microphone. Thefilter output signal 419 is subtracted from the desiredsignal 417 at the summingjunction 416 to produce anerror signal 420 which is fed back to theadaptive filter process 415. - The illustrated
power analyzer 413 compares the power of theerror signal 420 to the power of the signal representative of sound received from the first microphone. According to various embodiments, a voice will not be detected unless the power of the signal representative of sound received from the first microphone is much greater than the power of the error signal. For example, thepower analyzer 413 compares the difference to a threshold, and will not detect voice if the difference is less than the threshold. - The illustrated
coefficient analyzer 414 analyzes the filter coefficients from theadaptive filter process 415. According to various embodiments, a voice will not be detected unless a peak value for the coefficients is significantly high. For example, some embodiments will not detect voice unless the largest normalized coefficient is greater than a predetermined value (e.g. 0.5). -
FIGS. 5-7 illustrate various processes for detecting voice that can be used in various embodiments of the present subject matter. InFIG. 5 , as illustrated at 521, the power of the error signal from the adaptive filter is compared to the power of a signal representative of sound received by the first microphone. At 522, it is determined whether the power of the first microphone is greater than the power of the error signal by a predetermined threshold. The threshold is selected to be sufficiently high to ensure that the power of the first microphone is much greater than the power of the error signal. In some embodiments, voice is detected at 523 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold, and voice is not detected at 524 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold. - In
FIG. 6 , as illustrated at 625, coefficients of the adaptive filter are analyzed. At 626, it is determined whether the largest normalized coefficient is greater than a predetermined value, such as greater than 0.5. In some embodiments, voice is detected at 623 if the largest normalized coefficient is greater than a predetermined value, and voice is not detected at 624 if the largest normalized coefficient is not greater than a predetermined value. - In
FIG. 7 , as illustrated at 721, the power of the error signal from the adaptive filter is compared to the power of a signal representative of sound received by the first microphone. At 722, it is determined whether the power of the first microphone is greater than the power of the error signal by a predetermined threshold. In some embodiments, voice is not detected at 724 if the power of the first microphone is not greater than the power of the error signal by a predetermined threshold. If the power of the error signal is too large, then the adaptive filter has not converged. In the illustrated method, the coefficients are not analyzed until the adaptive filter converges. As illustrated at 725, coefficients of the adaptive filter are analyzed if the power of the first microphone is greater than the power of the error signal by a predetermined threshold. At 726, it is determined whether the largest normalized coefficient is greater than a predetermined value, such as greater than 0.5. In some embodiments, voice is not detected at 724 if the largest normalized coefficient is not greater than a predetermined value. Voice is detected at 723 if the power of the first microphone is greater than the power of the error signal by a predetermined threshold and if the largest normalized coefficient is greater than a predetermined value. -
FIG. 8 illustrates one embodiment of the present subject matter with an "own voice detector" to control active noise canceller for occlusion reduction. The active noise canceller filters microphone M2 with filter h and sends the filtered signal to the receiver. The microphone M2 and the error microphone M3 (in the ear canal) are used to calculate the filter update for filter h. The own voice detector, which uses microphone M1 and M2, is used to steer the stepsize in the filter update. -
FIG. 9 illustrates one embodiment of the present subject matter offering a multichannel expansion, compression and output control limiting algorithm (MECO) which uses the signal of microphone M2 to calculate the desired gain and subsequently applies that gain to microphone signal M2 and then sends the amplified signal to the receiver. Additionally, the gain calculation can take into account the outcome of the own voice detector (which uses M1 and M2) to calculate the desired gain. If the wearer's own voice is detected, the gain in the lower channels (typically below 1 KHz) will be lowered to avoid occlusion. Note: the MECO algorithm can use microphone signal M1 or M2 or a combination of both. -
FIG. 10 illustrates one embodiment of the present subject matter which uses an "own voice detector" in an environment classification scheme. From the microphone signal M2, several features are calculated. These features together with the result of the own voice detector, which uses M1 and M2, are used in a classifier to determine the acoustic environment. This acoustic environment classification is used to set the gain in the hearing aid. In various embodiments, the hearing aid may use M2 or M1 or M1 and M2 for the feature calculation. -
FIG. 11 illustrates a pair of hearing assistance devices according to one embodiment of the present subject matter. The pair of hearing assistance devices includes a lefthearing assistance device 1105L and a righthearing assistance device 1105R, such as a left hearing aid and a right hearing aid. The lefthearing assistance device 1105L is configured to be worn in or about the left ear of a wearer for delivering sound to the left ear canal of the wearer. The righthearing assistance device 1105R is configured to be worn in or about the right ear of the wearer for delivering sound to the right ear canal of the wearer. In one embodiment, the left and righthearing assistance devices device 305 as discussed above with capability of performing wireless communication between each other and uses voice detection capability of both devices to determine whether voice of the wearer is present. - The illustrated left
hearing assistance device 1105L includes afirst microphone MIC 1L, asecond microphone MIC 2L, aninterface 1107L, asound processor 1108L, areceiver 1106L, avoice detector 1109L, and acommunication circuit 1130L. Thefirst microphone MIC 1L produces a first left microphone signal. Thesecond microphone MIC 2L produces a second left microphone signal. In one embodiment, when the left and righthearing assistance devices first microphone MIC 1L is positioned about the left ear or the wearer, and thesecond microphone MIC 2L is positioned about the left ear canal of wearer, at a different location than thefirst microphone MIC 1L, on an air side of the left ear canal to detect signals outside the left ear canal.Interface 1107L converts the analog versions of the first and second left microphone signals into digital signals for processing by thesound processor 1108L and thevoice detector 1109L. For example, theinterface 1107L may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by thesound processor 1108L and thevoice detector 1109L. Thesound processor 1108L produces a processedleft sound signal 1110L. Theleft receiver 1106L produces a left audible signal based on the processedleft sound signal 1110L and transmits the left audible signal to the left ear canal of the wearer. In one embodiment, thesound processor 1108L produces the processedleft sound signal 1110L based on the first left microphone signal. In another embodiment, thesound processor 1108L produces the processedleft sound signal 1110L based on the first left microphone signal and the second left microphone signal. - The
left voice detector 1109L detects a voice of the wearer using the first left microphone signal and the second left microphone signal. In one embodiment, in response to the voice of the wearer being detected based on the first left microphone signal and the second left microphone signal, theleft voice detector 1109L produces a left detection signal indicative of detection of the voice of the wearer. In one embodiment, theleft voice detector 1109L includes a left adaptive filter configured to output left information and identifies the voice of the wearer from the output left information. In various embodiments, the output left information includes coefficients of the left adaptive filter and/or a left error signal. In various embodiments, theleft voice detector 1109L includes thevoice detector 309 or thevoice detector 409 as discussed above. Theleft communication circuit 1130L receives information from, and transmits information to, the righthearing assistance device 1105R via awireless communication link 1132. In the illustrated embodiment, the information transmitted viawireless communication link 1132 includes information associated with the detection of the voice of the wearer as performed by each of the left and righthearing assistance devices - The illustrated right
hearing assistance device 1105R includes afirst microphone MIC 1R, asecond microphone MIC 2R, aninterface 1107R, asound processor 1108R, areceiver 1106R, avoice detector 1109R, and acommunication circuit 1130R. Thefirst microphone MIC 1R produces a first right microphone signal. Thesecond microphone MIC 2R produces a second right microphone signal. In one embodiment, when the left and righthearing assistance devices first microphone MIC 1R is positioned about the right ear or the wearer, and thesecond microphone MIC 2R is positioned about the right ear canal of wearer, at a different location than thefirst microphone MIC 1R, on an air side of the right ear canal to detect signals outside the right ear canal.Interface 1107R converts the analog versions of the first and second right microphone signals into digital signals for processing by thesound processor 1108R and thevoice detector 1109R. For example, theinterface 1107R may include analog-to-digital converters, and appropriate registers to hold the digital signals for processing by thesound processor 1108R and thevoice detector 1109R. Thesound processor 1108R produces a processedright sound signal 1110R. Theright receiver 1106R produces a right audible signal based on the processedright sound signal 1110R and transmits the right audible signal to the right ear canal of the wearer. In one embodiment, thesound processor 1108R produces the processedright sound signal 1110R based on the first right microphone signal. In another embodiment, thesound processor 1108R produces the processedright sound signal 1110R based on the first right microphone signal and the second right microphone signal. - The
right voice detector 1109R detects the voice of the wearer using the first right microphone signal and the second right microphone signal. In one embodiment, in response to the voice of the wearer being detected based on the first right microphone signal and the second right microphone signal, theright voice detector 1109R produces a right detection signal indicative of detection of the voice of the wearer. In one embodiment, theright voice detector 1109R includes a right adaptive filter configured to output right information and identifies the voice of the wearer from the output right information. In various embodiments, the output right information includes coefficients of the right adaptive filter and/or a right error signal. In various embodiments, theright voice detector 1109R includes thevoice detector 309 or thevoice detector 409 as discussed above. Theright communication circuit 1130R receives information from, and transmits information to, the righthearing assistance device 1105R via awireless communication link 1132. - In various embodiments, at least one of the
left voice detector 1109L and theright voice detector 1109R is configured to detect the voice of wearer using the first left microphone signal, the second left microphone signal, the first right microphone signal, and the second right microphone signal. In other words, signals produced by all of themicrophones MIC 1L,MIC 2L,MIC 1R, andMIC 2R are used for determining whether the voice of the wearer is present. In one embodiment, theleft voice detector 1109L and/or theright voice detector 1109R declares a detection of the voice of the wearer in response to at least one of the left detection signal and the second detection signal being present. In another embodiment, theleft voice detector 1109L and/or theright voice detector 1109R declares a detection of the voice of the wearer in response to the left detection signal and the second detection signal both being present. In one embodiment, theleft voice detector 1109L and/or theright voice detector 1109R determines whether to declare a detection of the voice of the wearer using the output left information and output right information. The output left information and output right information are each indicative of one or more detection strength parameters each being a measure of likeliness of actual existence of the voice of wearer. Examples of the one or more detection strength parameters include the difference between the power of the error signal and the power of the first microphone signal and the largest normalized coefficient of the adaptive filter. In one embodiment, theleft voice detector 1109L and/or theright voice detector 1109R determines whether to declare a detection of the voice of the wearer using a weighted combination of the output left information and the output eight information. For example, the weighted combination of the output left information and the output right information can include a weighted sum of the detection strength parameters. The one or more detection strength parameters produced by each of the left and right voice detectors can be multiplied by one or more corresponding weighting factors before being added to produce the weighted sum. In various embodiments, the weighting factors may be determined using a priori information such as estimates of the background noise and/or position(s) of other sound sources in a room. - In various embodiments when a pair of left and right hearing assistance device is worn by the wearer, the detection of the voice of the wearer is performed using both the left and the right voice detectors such as
detectors left voice detector 1109L and theright voice detector 1109R, determined by theleft voice detector 1109L and communicated to theright voice detector 1109R viawireless link 1132, or determined by theright voice detector 1109R and communicated to theleft voice detector 1109L viawireless link 1132. Upon declaration of the detection of the voice of the wearer, theleft voice detector 1109L transmits anindication 1111L to thesound processor 1108L, and theright voice detector 1109R transmits anindication 1111R to thesound processor 1108R. Thesound processors sound signals -
FIG. 12 illustrates a process for detecting voice using a pair of hearing assistance devices including a left hearing assistance device and a right hearing assistance device, such as the left and righthearing assistance devices steps steps FIGS. 5-7 . At 1243, whether to declare a detection of the voice of the wearer is determining using an outcome of both of the detections at 1241 and 1242. - In one embodiment, the left and right hearing assistance devices each include first and second microphones. Electrical signals produced by the first and second microphones of the left hearing assistance device are used as inputs to a voice detector of the left hearing assistance device at 1241. The voice detector of the left hearing assistance device includes a left adaptive filter. Electrical signals produced by the first and second microphones of the right hearing assistance device are used as inputs to a voice detector of the right hearing assistance device at 1242. The voice detector of the right hearing assistance device includes a right adaptive filter. The voice of the wearer is detected using information output from the left adaptive filter and information output from the right adaptive filter at 1243. In one embodiment, the voice of the wearer is detected using left coefficients of the left adaptive filter and right coefficients of the right adaptive filter. In one embodiment, the voice of the wearer is detected using a left error signal produced by the left adaptive filter and a right error signal produced by the right adaptive filter. In one embodiment, the voice of the wearer is detected using a left detection strength parameter of the information output from the left adaptive filter and a right detection strength parameter of the information output from the right adaptive filter. The left and right detection strength parameters are each a measure of likeliness of actual existence of the voice of wearer. Examples of the left detection strength parameter include the difference between the power of a left error signal produced by the left adaptive filter and the power of the electrical signal produced by the first microphone of the left hearing assistance device and the largest normalized coefficient of the left adaptive filter. Examples of the right detection strength parameter include the difference between the power of a right error signal produced by the right adaptive filter and the power of the electrical signal produced by the first microphone of the right hearing assistance device and the largest normalized coefficient of the right adaptive filter. In one embodiment, the voice of the wearer is detected using a weighted combination of the information output from the left adaptive filter and the information output from the right adaptive filter.
- In one embodiment, the voice of the wearer is detected using the left hearing assistance device based on the electrical signals produced by the first and second microphones of the left hearing assistance device, and a left detection signal indicative of whether the voice of the wearer is detected by the left hearing assistance device is produced, at 1241. The voice of the wearer is detected using the right hearing assistance device based on the electrical signals produced by the first and second microphones of the right hearing assistance device, and a right detection signal indicative of whether the voice of the wearer is detected by the right hearing assistance device is produced, at 1242. Whether to declare the detection of the voice of the wearer is determined using the left detection signal and the right detection signal at 1243. In one embodiment, the detection of the voice of the wearer is declared in response to both of the left detection signal and the right detection signal being present. In another embodiment, the detection of the voice of the wearer is declared in response to at least one of the left detection signal and the right detection signal being present. In one embodiment, whether to declare the detection of the voice of the wearer is determined using the left detection signal, the right detection signal, and weighting factors applied to the left and right detection signals.
- The various embodiments of the present subject matter discussed with reference to
FIGS. 1-10 can be applied to each device of a pair of hearing assistance devices, with the declaration of the detection of the voice of the wearer being a result of detection using both devices of the pair of hearing assistance devices, as discussed with reference toFIGS. 11 and12 . Such binaural voice detection will likely improve the acoustic perception of the wearer because both hearing assistance devices worn by the wearer are acting similarly when the wearer speaks. In various embodiments in which a pair of hearing assistance devices is worn by the wearer, whether to declare a detection of the voice of the wearer may be determined based on the detection performed by either one device of the pair of hearing assistance devices or based on the detection performed by both devices of the pair of hearing assistance devices. An example of the pair of hearing assistance devices includes a pair of hearing aids. - The present subject matter includes hearing assistance devices, and was demonstrated with respect to BTE, OTE, and RIC type devices, but it is understood that it may also be employed in cochlear implant type hearing devices. It is understood that other hearing assistance devices not expressly stated herein may fall within the scope of the present subject matter.
- This application is intended to cover adaptations or variations of the present subject matter. It is to be understood that the above description is intended to be illustrative, and not restrictive. The scope of the present subject matter should be determined with reference to the appended claims, along with the full scope of legal equivalents to which such claims are entitled.
- The invention can be further described with regards to the following consistory clauses:
- 1. An apparatus configured to be worn by a wearer having a left ear with a left ear canal and a right ear with a right ear canal, comprising:
- a left hearing assistance device and a right hearing assistance device communicatively coupled to each other, the left and right hearing assistance devices each including:
- a first microphone configured to produce a first microphone signal;
- a second microphone configured to produce a second microphone signal; and
- a voice detector,
- wherein at least one voice detector of the voice detectors of the left and right hearing assistance devices is configured to detect a voice of the wearer using the first and second microphone signals produced by the left hearing assistance device and the first and second microphone signals produced by the right hearing assistance device.
- a left hearing assistance device and a right hearing assistance device communicatively coupled to each other, the left and right hearing assistance devices each including:
- 2. The apparatus according to
clause 1, wherein the voice detector of each of the left and right hearing assistance device comprises an adaptive filter configured to output information and the at least one of the voice detectors of the left and right hearing assistance devices is configured to detect the voice of the wearer using the output information from each of the left and right hearing assistance devices. - 3. The apparatus according to
clause 2, wherein the at least one voice detector is configured to detect the voice of the wearer using coefficients of the adaptive filter of each of the left and right hearing assistance devices. - 4. The apparatus according to any of
clauses 2 and 3, wherein the at least one voice detector is configured to detect the voice of the wearer using an error signal produced by the adaptive filter of each of the left and right hearing assistance devices. - 5. The apparatus according to any of
clauses 2 to 4, wherein the at least one voice detector is configured to detect the voice of the wearer using a detection strength parameter of the output information from each of the left and right hearing assistance, the detection strength parameter being a measure of likeliness of actual existence of the voice of wearer. - 6. The apparatus according to any of
clauses 2 to 5, wherein the at least one voice detector is configured to detect the voice of the wearer using a weighted combination of the output information from the left hearing assistance device and the output information from the right hearing assistance device. - 7. The apparatus according to any of the preceding clauses, wherein the left and right hearing assistance devices each comprise a hearing aid configured such that when being worn by the wearer, the first microphone is positioned about one of the left and right ears and the second microphone is positioned about one of the left and right ear canals, at a different location than the first microphone, on an air side of the one of the left and left ear canals to detect signals outside the one of the left and right ear canals.
- 8. The apparatus according to clause 7, wherein the hearing aid comprises:
- a sound processor configured to produce a processed sound signal based on at least the first microphone signal and whether the voice of the wearer is detected; and
- a receiver configured to produce an audible signal based on the processed sound signal and transmit the audible signal to the one of the left and right ear canals.
- 9. A method for detecting a voice of a wearer of a pair of left and right hearing assistance devices each including a first microphone and a second microphone, the wearer having a left ear with a left ear canal and a right ear with a right ear canal, the method comprising:
- positioning the first microphone and the second microphone of the left hearing assistance device about the left ear to each detect sound outside the left ear canal;
- positioning the first microphone and the second microphone of the right hearing assistance device about the right ear to each detect sound outside the right ear canal; and
- detecting a voice of the wearer using electrical signals produced by the first and second microphones of the left hearing assistance device and the electrical signals produced by the first and second microphones of the right hearing assistance device.
- 10. The method according to clause 9, wherein detecting the voice of the wearer comprises:
- using electrical signals produced by the first and second microphones of the left hearing assistance device as inputs to a voice detector of the left hearing assistance device including a left adaptive filter;
- using electrical signals produced by the first and second microphones of the right hearing assistance device as inputs to a voice detector of the right hearing assistance device including a right adaptive filter; and
- detecting the voice of the wearer using information output from the left adaptive filter and information output from the right adaptive filter.
- 11. The method according to clause 10, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using left coefficients of the left adaptive filter and right coefficients of the right adaptive filter.
- 12. The method according to any of clauses 10 and 11, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using a left error signal produced by the left adaptive filter and a right error signal produced by the right adaptive filter.
- 13. The method according to any of clauses 10 to 12, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using a left detection strength parameter of the information output from the left adaptive filter and a right detection strength parameter of the information output from the right adaptive filter, the left and right detection strength parameters each being a measure of likeliness of actual existence of the voice of wearer.
- 14. The method according to any of clauses 10 to 13, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using a weighted combination of the information output from the left adaptive filter and the information output from the right adaptive filter.
- 15. The method according to any of clauses 9 to 14, wherein detecting the voice of the wearer comprises:
- detecting the voice of the wearer using the left hearing assistance device based on the electrical signals produced by the first and second microphones of the left hearing assistance device;
- producing a left detection signal indicative of whether the voice of the wearer is detected by the left hearing assistance device;
- detecting the voice of the wearer using the right hearing assistance device based on the electrical signals produced by the first and second microphones of the right hearing assistance device;
- producing a right detection signal indicative of whether the voice of the wearer is detected by the right hearing assistance device; and
- determining whether to declare a detection of the voice of the wearer using the left detection signal and the right detection signal.
Claims (13)
- A hearing device for detecting a voice of a wearer of the hearing device and a second hearing device, each hearing device having a plurality of microphones, the hearing device comprising:a voice detector comprising an adaptive filter configured to receive microphone signals from the plurality of microphones of the hearing device and produce first output information indicative of detection of the voice of the wearer by the hearing device, the voice detector further configured to receive second output information from the second hearing device that indicates detection of the voice of the wearer by the second hearing device, wherein the voice detector is programmed to determine the wearer's own voice based on the first and second output information.
- The hearing device of claim 1, wherein the voice detector is configured to determine the wearer's own voice using a first detection strength parameter of the first output information and a second detection strength parameter of the second output information, the first and second detection strength parameters each being a measure of likeliness of actual existence of the voice of wearer.
- The hearing device of claim 2, wherein the voice detector is configured to produce the first output information using a largest normalized coefficient of the adaptive filter.
- The hearing device of claim 3, wherein the voice detector is configured to produce the first output information using the largest normalized coefficient of the adaptive filter, a difference between a power of an error signal produced by the adaptive filter and a power of a microphone signal of the microphone signals.
- The hearing device of claim 2, wherein the voice detector is configured to produce the first output information using a difference between a power of an error signal produced by the adaptive filter and a power of a microphone signal of the microphone signals.
- The hearing device of claim 2, wherein the voice detector is configured to determine the wearer's own voice using a weighted combination of the first output information and the second output information.
- A method for operating a hearing device configured for being worn by a wearer of the hearing device and a second hearing device, the method comprising:receiving electrical signals from a first microphone and a second microphone of the hearing device;producing first output information indicative a voice of the wearer based on the received electrical signals using an adaptive filter of the hearing device;receiving second output information indicative the voice of the wearer from the second hearing device; anddetecting the voice of the wearer using the first output information and the second output information.
- The method of claim 7, comprising positioning the first and second microphones to provide a time difference for the voice of the wearer to reach each of the first and second microphones when the first and second hearing devices are worn by the wearer.
- The method of claim 8, wherein producing the first output information comprises producing the first output information using coefficients of the adaptive filter.
- The method of claim 9, wherein producing the first output information comprises producing the first output information using an error signal produced by the adaptive filter.
- The method of claim 8, wherein producing the first output information comprises producing the first output information using an error signal produced by the adaptive filter.
- The method of claim 7, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using a first detection strength parameter of the first output information and a right detection strength parameter of the second output information, the first and second detection strength parameters each being a measure of likeliness of actual existence of the voice of wearer.
- The method of claim 7, wherein detecting the voice of the wearer comprises detecting the voice of the wearer using a weighted combination of the first output information and the second output information.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/464,149 US9219964B2 (en) | 2009-04-01 | 2014-08-20 | Hearing assistance system with own voice detection |
EP15181620.4A EP2988531B1 (en) | 2014-08-20 | 2015-08-19 | Hearing assistance system with own voice detection |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15181620.4A Division EP2988531B1 (en) | 2014-08-20 | 2015-08-19 | Hearing assistance system with own voice detection |
Publications (3)
Publication Number | Publication Date |
---|---|
EP3461148A2 true EP3461148A2 (en) | 2019-03-27 |
EP3461148A3 EP3461148A3 (en) | 2019-04-17 |
EP3461148B1 EP3461148B1 (en) | 2023-03-22 |
Family
ID=53879441
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15181620.4A Revoked EP2988531B1 (en) | 2014-08-20 | 2015-08-19 | Hearing assistance system with own voice detection |
EP18195310.0A Active EP3461148B1 (en) | 2014-08-20 | 2015-08-19 | Hearing assistance system with own voice detection |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15181620.4A Revoked EP2988531B1 (en) | 2014-08-20 | 2015-08-19 | Hearing assistance system with own voice detection |
Country Status (2)
Country | Link |
---|---|
EP (2) | EP2988531B1 (en) |
DK (1) | DK2988531T3 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3672281B1 (en) * | 2018-12-20 | 2023-06-21 | GN Hearing A/S | Hearing device with own-voice detection and related method |
CN114449427B (en) * | 2020-11-02 | 2024-06-25 | 达发科技股份有限公司 | Hearing assistance device and method for adjusting output sound of hearing assistance device |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7027607B2 (en) * | 2000-09-22 | 2006-04-11 | Gn Resound A/S | Hearing aid with adaptive microphone matching |
NL1021485C2 (en) * | 2002-09-18 | 2004-03-22 | Stichting Tech Wetenschapp | Hearing glasses assembly. |
EP1599742B1 (en) * | 2003-02-25 | 2009-04-29 | Oticon A/S | Method for detection of own voice activity in a communication device |
WO2008006772A2 (en) | 2006-07-12 | 2008-01-17 | Phonak Ag | Method for operating a binaural hearing system as well as a binaural hearing system |
US8477973B2 (en) | 2009-04-01 | 2013-07-02 | Starkey Laboratories, Inc. | Hearing assistance system with own voice detection |
DE102010012622B4 (en) | 2010-03-24 | 2015-04-30 | Siemens Medical Instruments Pte. Ltd. | Binaural method and binaural arrangement for voice control of hearing aids |
US9124984B2 (en) * | 2010-06-18 | 2015-09-01 | Panasonic Intellectual Property Management Co., Ltd. | Hearing aid, signal processing method, and program |
DK2563045T3 (en) * | 2011-08-23 | 2014-10-27 | Oticon As | Method and a binaural listening system to maximize better ear effect |
DE102011087984A1 (en) | 2011-12-08 | 2013-06-13 | Siemens Medical Instruments Pte. Ltd. | Hearing apparatus with speaker activity recognition and method for operating a hearing apparatus |
US9094749B2 (en) | 2012-07-25 | 2015-07-28 | Nokia Technologies Oy | Head-mounted sound capture device |
EP2920980B1 (en) | 2012-11-15 | 2016-10-05 | Sonova AG | Own voice shaping in a hearing instrument |
-
2015
- 2015-08-19 DK DK15181620.4T patent/DK2988531T3/en active
- 2015-08-19 EP EP15181620.4A patent/EP2988531B1/en not_active Revoked
- 2015-08-19 EP EP18195310.0A patent/EP3461148B1/en active Active
Non-Patent Citations (1)
Title |
---|
None |
Also Published As
Publication number | Publication date |
---|---|
EP3461148A3 (en) | 2019-04-17 |
EP2988531A1 (en) | 2016-02-24 |
EP3461148B1 (en) | 2023-03-22 |
EP2988531B1 (en) | 2018-09-19 |
DK2988531T3 (en) | 2019-01-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11388529B2 (en) | Hearing assistance system with own voice detection | |
US10715931B2 (en) | Hearing assistance system with own voice detection | |
US9749754B2 (en) | Hearing aids with adaptive beamformer responsive to off-axis speech | |
EP3005731B1 (en) | Method for operating a hearing device and a hearing device | |
EP3188508B1 (en) | Method and device for streaming communication between hearing devices | |
CN105898651B (en) | Hearing system comprising separate microphone units for picking up the user's own voice | |
US9641942B2 (en) | Method and apparatus for hearing assistance in multiple-talker settings | |
KR101689339B1 (en) | Earphone arrangement and method of operation therefor | |
EP3799444A1 (en) | A hearing aid comprising a directional microphone system | |
EP3461148B1 (en) | Hearing assistance system with own voice detection |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AC | Divisional application: reference to earlier application |
Ref document number: 2988531 Country of ref document: EP Kind code of ref document: P |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 25/00 20060101AFI20190311BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191015 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20201022 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20220512 |
|
GRAJ | Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted |
Free format text: ORIGINAL CODE: EPIDOSDIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTC | Intention to grant announced (deleted) | ||
INTG | Intention to grant announced |
Effective date: 20221021 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AC | Divisional application: reference to earlier application |
Ref document number: 2988531 Country of ref document: EP Kind code of ref document: P |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602015082927 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1555976 Country of ref document: AT Kind code of ref document: T Effective date: 20230415 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20230322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230622 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230624 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1555976 Country of ref document: AT Kind code of ref document: T Effective date: 20230322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230623 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230724 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230722 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230920 Year of fee payment: 9 Ref country code: DE Payment date: 20230920 Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R026 Ref document number: 602015082927 Country of ref document: DE |
|
PLBI | Opposition filed |
Free format text: ORIGINAL CODE: 0009260 |
|
PLAX | Notice of opposition and request to file observation + time limit sent |
Free format text: ORIGINAL CODE: EPIDOSNOBS2 |
|
26 | Opposition filed |
Opponent name: OTICON A/S Effective date: 20231221 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230819 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230819 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230831 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20230831 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20230322 |
|
PLBB | Reply of patent proprietor to notice(s) of opposition received |
Free format text: ORIGINAL CODE: EPIDOSNOBS3 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230819 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230819 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230831 |