WO2026013307A1

WO2026013307A1 - Method of operating a hearing aid system and a hearing aid system

Info

Publication number: WO2026013307A1
Application number: PCT/EP2025/070020
Authority: WO
Inventors: David PELEGRIN-GARCIA
Original assignee: Widex AS
Current assignee: Widex AS
Priority date: 2024-07-11
Filing date: 2025-07-11
Publication date: 2026-01-15
Anticipated expiration: 2027-01-11

Abstract

A method (100) of operating a hearing aid system based on determining a time varying and frequency dependent masking level for a given specific hearing loss. The invention also refers to a hearing aid system adapted to carry out said method and a method of training a neural network.

Description

METHOD OF OPERATING A HEARING AID SYSTEM AND A HEARING AID SYSTEM The present invention relates to a method of operating a hearing aid system. The present invention also relates to a hearing aid system adapted to carry out said method. BACKGROUND OF THE INVENTION Generally a hearing aid system according to the invention is understood as meaning any device which provides an output signal that can be perceived as an acoustic signal by a user or contributes to providing such an output signal, and which has means which are customized to compensate for an individual hearing loss of the user or contribute to compensating for the hearing loss of the user. They are, in particular, hearing aids which can be worn on the body or by the ear, in particular on or in the ear, and which can be fully or partially implanted. However, some devices whose main aim is not to compensate for a hearing loss, may also be regarded as hearing aid systems, for example consumer electronic devices (televisions, hi-fi systems, mobile phones, MP3 players etc.) provided they have, however, measures for compensating for an individual hearing loss. Within the present context a traditional hearing aid can be understood as a small, battery-powered, microelectronic device designed to be worn behind or in the human ear by a hearing-impaired user. Prior to use, the hearing aid is adjusted by a hearing aid fitter according to a prescription. The prescription is based on a hearing test, resulting in a so-called audiogram, of the performance of the hearing-impaired user’s unaided hearing. The prescription is developed to reach a setting where the hearing aid will alleviate a hearing loss by amplifying sound at frequencies in those parts of the audible frequency range where the user suffers a hearing deficit. A hearing aid comprises one or more microphones, a battery, a microelectronic circuit comprising a signal processor, and an acoustic output transducer. The signal processor is preferably a digital signal processor. The hearing aid is enclosed in a casing suitable for fitting behind or in a human ear. Within the present context a hearing aid system may comprise a single hearing aid (a so called monaural hearing aid system) or comprise two hearing aids, one for each ear of the hearing aid user (a so called binaural hearing aid system). Furthermore, the hearing aid system may comprise an external device, such as a smart phone having software applications adapted to interact with other devices of the hearing aid system. Thus within the present context the term “hearing aid system device” may denote a hearing aid or an external device. However, the audio device system may also include a remote microphone system (which generally can also be considered an external device) comprising additional microphones and/or may even include a remote server providing abundant processing resources and generally these additional devices will also include link means adapted to operationally connect to the various other devices of the hearing aid system. The mechanical design has developed into a number of general categories. As the name suggests, Behind-The-Ear (BTE) hearing aids are worn behind the ear. To be more precise, an electronics unit comprising a housing containing the major electronics parts thereof is worn behind the ear. An earpiece for emitting sound to the hearing aid user is worn in the ear, e.g. in the concha or the ear canal. In a traditional BTE hearing aid, a sound tube is used to convey sound from the output transducer, which in hearing aid terminology is normally referred to as the receiver, located in the housing of the electronics unit and to the ear canal. In some modern types of hearing aids, a conducting member comprising electrical conductors conveys an electric signal from the housing and to a receiver placed in the earpiece in the ear. Such hearing aids are commonly referred to as Receiver-In-The-Ear (RITE) hearing aids. In a specific type of RITE hearing aids the receiver is placed inside the ear canal. This category is sometimes referred to as Receiver-In-Canal (RIC) hearing aids. In-The-Ear (ITE) hearing aids are designed for arrangement in the ear, normally in the funnel-shaped outer part of the ear canal. In a specific type of ITE hearing aids the hearing aid is placed substantially inside the ear canal. This category is sometimes referred to as Completely-In-Canal (CIC) hearing aids. This type of hearing aid requires an especially compact design in order to allow it to be arranged in the ear canal, while accommodating the components necessary for operation of the hearing aid. With time new functionality, such as advanced noise reduction algorithms, is implemented in hearing aid systems. One example of a contemporary advanced noise reduction algorithm is the Speech Intelligibility Index (SII) based noise reduction that seeks to select the optimum gain based on the frequency dependent hearing threshold of a specific user and based on a model of speech intelligibility for a given frequency dependent signal to noise ratio. However, the gain hereby selected may not be an optimum selection for all users. WO 2001/069504 discloses a hearing aid system wherein an additional personalized gain is applied on top of and independent of the various other gains (including e.g. adaptive gains from noise reduction algorithms) applied as part of the signal processing. However, this additional personalized gain is directed at adjustments of the prescribed gain curve and as such does not provide control of the actual hearing aid output level in any given sound environment because this also depends on the various other applied gains. It is therefore an object of the present invention to provide a method of operating a hearing aid system capable of providing improved sound quality and speech intelligibility for the hearing aid user. SUMMARY OF THE INVENTION The invention is set out in the appended set of claims. BRIEF DESCRIPTION OF THE DRAWINGS By way of example, there is shown and described a preferred embodiment of this invention. As will be realized, the invention is capable of other embodiments, and its several details are capable of modification in various, obvious aspects all without departing from the invention. Accordingly, the drawings and descriptions will be regarded as illustrative in nature and not as restrictive. In the drawings: Fig.1 illustrates highly schematically a method of operating a hearing aid system; DETAILED DESCRIPTION In the present context the terms “received sound” and ”processed sound” will generally be construed to mean an electrical (analog or digital) signal representing a sound. A beamformed signal (either monaural or binaural) is one example of such an electrical signal representing a sound. Another example is an electrical signal wirelessly streamed to the audio device system. In the following the terms “mask”, “masking level” and “masking threshold” may generally be used interchangeably, despite that it may be argued that very strictly speaking the term “mask” represents the general phenomenon of one sound obscuring another, the term “masking level” represents the intensity of a masked signal and the term “masking threshold” represents the minimum level of any added signal before it can be heard. Masking models are widely used in consumer electronics for a variety of purposes (e.g. audio watermarking and audio codecs). These are targeted to a wide user audience and playback conditions, and therefore the masking models operate on the conservative side (assuming normal hearing users playing at soft levels). In contrast, hearing aids are calibrated devices that are fitted to an individual user. It is one inventive aspect of the present invention to exploit the knowledge about the absolute levels and the individual hearing abilities (i.e. the specific frequency dependent hearing loss) to optimize a masking model and hereby the performance of a plurality of hearing aid system applications. Thus in the present context an “optimized” masking model is construed to mean that the masking model is based on knowledge of the hearing aid system users (specific) hearing loss Such an optimized masking model is preferably based on a sound (which in the following may be denoted an input signal or a signal of interest or just “signal”) that is estimated or measured at or in the hearing system user’s ear. Based hereon the optimized masking model (which in the following may be denoted (only) masking _{model) calculates a time-frequency mask (or mask level) ^^^^^^^^(^^^^, ^^^^).} Distortions or signals whose level is below the mask will be inaudible to the user. This can be used in a variety of hearing-aid features: • Perceptual audio codec for CROS/BiCROS configurations: Quantization noise of transmitted audio streams can be shaped under the mask. Hereby it becomes possible to reduce the transmission bit-rate and hereby decreasing power consumption, while still achieving a good sound quality of the transmitted signal. • Probe noise generation for feedback cancellation: A random noise signal can be shaped under the mask and added to the signal of interest to continuously measure the acoustic/mechanical feedback path and hereby perform digital feedback cancellation. • Audio Watermarking: Non-acoustic information can be encoded under the mask. As an example, sequences with information on the device number and/or platform ID and/or fitting data could be played back with regular intervals. This type of information would be conveyed to in-ear recordings with fitted hearing aids (e.g. on a dummy head). This could then be used internally in the company for automatic labelling, archival and retrieval of audio recordings with hearing aids. Another application would be to spot audio recordings in public or commercial databases performed with our hearing aids. • Application of noise reduction with minimal distortions. Whenever a noise reduction is applied, there is a risk that the speech component is degraded, typically by introducing various sound artifacts such as speech artifacts. By limiting the noise reduction (i.e. constraining the magnitude) as a function of a time-varying and frequency-dependent masking threshold such that noise suppression is selectively limited in regions where a target signal is perceptually dominant – then speech artifacts can be minimized. Thus all the above hearing aid features will benefit from the higher masking levels that can be determined by the optimized masking model according to the present invention. In addition, there are other applications which are not related to the injection or shaping of signals under the masking levels: • Scene detection / classification: A property of the time-varying and frequency- dependent masking threshold calculated by the optimized masking model is that the distance to the corresponding signal level is greater for tonal than for noisy signal components. This way, the masking threshold and signal level information can be used to improve a scene detection / classification algorithm. • Hearing loss compensation: It is known that the loudness of a sound grows from the masked threshold (see e.g. Lochner, J. P. A., & Burger, J. F. (1961). Form of the loudness function in the presence of masking noise. The Journal of the Acoustical Society of America, 33(12), 1705-1707). Therefore, an improved hearing loss compensation scheme to provide an, at least partial, loudness restoration can be obtained by modelling the loudness growth for a normal hearing person and a hearing impaired hearing system user respectively based on the time-varying and frequency-dependent masking thresholds and the associated signal (i.e. sound) levels for a normal hearing person and a hearing impaired hearing system user. Thus the optimized masking model that provides the basis for the various embodiments according to the present invention is a masking model based on individual hearing impairment characterization that profits from the calibrated operation of hearing aids (that enables the absolute threshold of hearing to be determined sufficiently precise). The optimized masking model is especially advantageous because it can account for (at least): - the individual absolute threshold of hearing in quiet; - the individual broadening of auditory filters; and - the individual loss of temporal resolution. This is obtained by: - modelling forward masking and including an exponentially decay in logarithmic domain towards the hearing threshold; - the absolute hearing threshold, the spectral masking spread (broadening) and the forward masking can be tuned individually to perceptual measures; - the architecture of the optimized masking model has a low algorithmic delay and can be deployed into a hearing aid platform. Overall the predicted time-varying and frequency-dependent masking thresholds are generally higher than the masking thresholds for normal hearing persons and consequently the effectiveness of all applications relying on the use of the optimized masking model will increase when compared to existing models of masking that are directed at (and based on) normal hearing persons. Reference is first given to Fig.1, that highly schematically illustrates a method 100 of operating a hearing aid system, according to an embodiment, and more specifically the steps required to determine, based on a received sound, a time-varying and frequency- dependent masking threshold for a specific hearing loss. In a first step 101, excitation patterns in auditory bands are determined, using a filterbank with envelope extraction. This enables the determination of signal envelope levels in auditory bands (based on e.g. the Bark scale or the equivalent rectangular bandwidth (ERB) scale. Thus the filterbank is used to split the signal in different bands, which can be carried out based on either of the two basic approaches to this i.e.: - using a high-resolution filterbank and grouping components in auditory bands at a later processing stage (similar to what the MPEG psychoacoustic model does) - Using an auditorily-inspired filterbank that splits the signal in auditory bands. According to the present embodiment a complex gammatone filterbank is applied (see e.g.: “Frequency analysis and synthesis using a Gammatone filterbank” by V. Hohmann, Acust. Acta Acust., 88(3):433–442, 2002) with bands having a default width of one ERB while also being spaced one ERB apart. Specific advantages of that filterbank includes that: - the frequency shaping of the (filterbank) filters is asymmetric and resembles the shape of auditory filters measured with psychoacoustic measurements; and that - the modulus of the complex output in each band corresponds to the Hilbert envelope. According to an embodiment the bandwidth of the (filterbank) filters is updated to be higher than one ERB depending on an individual measure of frequency selectivity, e.g. extracted from psychoacoustic measurements of auditory bandwidth, which may be obtained by using a software application (i.e. an “app) or based on an audiogram. In an embodiment the ERB bandwidth of the (filterbank) filters can dynamically be changed based on the broadband level of the input signal so that the filters are wider when the input signals are louder. In a second step 102, spectral mask computation incorporating cross frequency masking is carried out, based on signal envelope levels for a person with a given (i.e. specific) hearing loss. The spectral mask computation incorporates the cross-frequency masking based on the signal envelope levels on the different frequency channels, and potentially also on the recent history of the signal. The spectral mask computation is based on the article: “A new psychoacoustical masking model for audio coding applications”, by S. Van De Par, A. Kohlrausch, G. Charestan, and R. Heusdens, from ICASSP, pages 1805–1808, 2002. However, the spectral mask computation differs from the above mentioned reference at least in that: _{- there is no calculation of running segment length, so ^�^^^ = 1;} - depending on the application, a correction is applied for the threshold obtained from the model (which is valid for tonal masked signals) so that it corresponds to a masked signal that is spread throughout the spectrum (as e.g. quantization noise); - the parameters of the spectral mask computation (i.e. the filterbank spectral shapes and the associated calibration terms C_a and C_b) are adjusted together with the ERB bandwidth used in the filterbank, to account for the user’s loss of frequency selectivity. One of the key features in the optimized masking model is that, by construction, the mask for a tonal signal is at a lower level relative to the signal envelope, compared to that of a broad band signal. In a third step 103 broader level dependent auditory filters are determined for said (specific) hearing loss. As explained above, and in an embodiment, this can be done by dynamically changing the ERB bandwidth of the (filterbank) filters based on the broadband level of the input signal so that the filters are wider when the input signals are louder. In another embodiment and according to an alternative approach, which especially is advantageous in a case the filterbank is static, said spectral broadening (i.e. said modelling of broader level dependent auditory filters) can be determined as explained below: This approach applies a broadening of the mask by limiting the maximum upward and downward slopes of the mask. For normal hearing persons listening to soft sounds, this step is not required but for hearing aid system users it is. Now, assuming that the masking model has N frequency bands ranging from 0 to N-1. The frequency bands have an auditory spacing, following the ERB scale, and with the spacing between said frequency bands being one ERB. The frequency index is ^^^^, the input mask is referred to as ^^^^_^^^^ and the broadened mask as ^^^^_^^^^. Upward and downward slopes are ^^^^_^^^^ and ^^^^_^^^^. The downward and upward slopes are applied in parallel. The downward spread is calculated as follows: •_{Initial condition:} •_{Recursion for ^^^^ = ^^^^ − 2 ∶ −1: 0} _{o ^^^^ = ^^^^^^^^,^^^^(^^^^ + 1) − ^^^^^^^^} _{o ^^^^ < ^^^^^^^^(^^^^)?} _{^ True: ^^^^^^^^,^^^^(^^^^) = ^^^^^^^^(^^^^)} _{^ False: ^^^^^^^^,^^^^(^^^^) = ^^^^} The upward spread is calculated similarly: •_{Initial condition: ^^^^^^^^,^^^^(0) = ^^^^^^^^(0)} _{• Recursion for ^^^^ = 1 ∶ 1: ^^^^ − 1} _{o ^^^^ = ^^^^^^^^,^^^^(^^^^ − 1) − ^^^^^^^^} _{o ^^^^ < ^^^^^^^^(^^^^)?} _{^ True: ^^^^^^^^,^^^^(^^^^) = ^^^^^^^^(^^^^)} _{^ False: ^^^^^^^^,^^^^(^^^^) = ^^^^} Finally, the broadened mask ^^^^₀(^^^^) is determined as the maximum of the two previous intermediate results: To introduce level-dependency slopes, it is possible to make ^^^^_^^^^ and ^^^^_^^^^ dependent on the broadband input level, so that the resulting values vary from steep slopes at low input levels to more gradual at high levels. The exact values can be derived from psychoacoustic data. In a fourth step 104, the hearing threshold of said specific hearing loss is incorporated in the resulting masking level (i.e. the masked threshold) based on: Wherein ^^^^_^^^^(^^^^) represents said resulting masked threshold, wherein ℎ(^^^^) represents said specific hearing threshold (as a function of the frequency band ^^^^) and expressed in dB SPL@ED (i.e. decibels Sound Pressure Level at the Ear Drum). As in the previous equation ^^^^₀ ⁽^^^^⁾ represents the broadened mask. In a fifth step 105 the forward masking is determined by applying a temporal decay to the determined resulting masking level ^^^^_^^^^(^^^^). The applied temporal decay to the determined resulting masking level ^^^^_^^^^ ⁽^^^^⁾ is based on the psychoacoustic data given e.g. in the article by Fastl, H., & Zwicker, E.: “Psychoacoustics: facts and models”, (Vol.22), Springer Science & Business Media (2007). The masked threshold follow an exponential decay in the logarithmic (level) domain towards the absolute threshold, unless a higher instantaneous value of the mask is observed, in which case the output tracks this new value. The smoothing time constant ^^^^ is chosen so that it corresponds to a time constant of 20 ms for a normal hearing configuration. It can nevertheless be configured based on _{individual measurements. At time ^^^^ and frequency band ^^^^, the output mask ^^^^^^^^(^^^^, ^^^^)} (i.e the resulting mask prediction) is given by following equation: ^_{^^^^^^^(^^^^, ^^^^) = max�^^^^^^^^(^^^^, ^^^^),^^^^ ℎ(^^^^) + (1 − ^^^^) ^^^^^^^^(^^^^, ^^^^ − 1)�,^^^^ = 0 …^^^^ − 1} The initial value is chosen very low so that the maximum condition takes the current input as the output. According to a more specific embodiment, said filterbank, with envelope extraction, that is used to determine excitation patterns in auditory bands is a complex gammatone filterbank. The complex gammatone filterbank is especially advantageous for determining excitation patterns in auditory bands because it closely models the human cochlear filtering process while additionally offering computational and analytical benefits that makes the complex gammatone filterbank very well suited for estimating (i.e. determining) said excitation patterns that represent how much energy is present in each auditory band. Thus a gammatone filter mimics the frequency selectivity of the human auditory system and the complex gammatone filterbank extends this by using analytic (complex- valued) filters, which preserve both amplitude and phase information of the signal in each auditory band. More specifically, the complex output allows separation of: - the envelope (i.e. the slow amplitude fluctuations, that are important for speech intelligibility), and - the fine structure (i.e. the rapid oscillations, that are important for pitch and localization). This dual representation is crucial for obtaining a level of realistic auditory modeling, that is required for use in a hearing aid feature such as the one described herein. Furthermore the magnitude of the complex output directly provides the instantaneous energy in each band , which avoids the need for squaring and low-pass filtering, as required in real-valued filterbanks. Thus it enables time-varying excitation patterns to be computed with high temporal resolution. Thus, the complex gammatone filterbank provides, through the use of envenlope extraction (i.e. by taking the magnitude of the complex output) the instantaneous amplitude (or energy) of the signal in a given auditory band. Thus by using the complex gammatone filterbank, with envelope extraction, according to this specific embodiment, accurate modeling of auditory excitation patterns, crucial for perceptual signal processing, is obtained. According to a more specific embodiment of said method of operating a hearing aid system, said step of minimizing speech artifacts by adaptively constraining the magnitude of noise reduction as a function of a time-varying and frequency-dependent masking level such that noise suppression is selectively limited in regions where the target signal is perceptually dominant comprises the following steps: - a) receiving an input audio signal comprising a mixture of a target signal and noise; - b) processing the input audio signal using a neural network, wherein said neural network is trained to: - - estimate a noise-reduced output signal based on said input audio signal; and - - provide a noise-reduced output signal; - c) dynamically adjusting the operation of the neural network to provide a predefined target signal-to-noise ratio in the output signal; wherein the neural network is further adapted to minimize the perceptual signal artifacts introduced during noise reduction by: - - i) incorporating a loss function during training of said neural network that penalizes perceptual distortion in the output signal relative to the input signal and/or - - ii) conditioning the neural network on an artifact control parameter indicative of a trade-off between noise suppression and audio fidelity of the output signal. Hereby significantly improved performance of (e.g. off-the-shelf) neural networks can be obtained because the noise reduction provided by the neural network is deliberately tuned to maintain a relatively low output signal to noise ratio (SNR) and hereby prioritizing the preservation of signal quality and minimizing artifacts, even if it means allowing more residual noise. In an embodiment a method of training a neural network to provide noise reduction for a specific hearing aid system (i.e. a hearing aid system for a person with a specific (i.e. given) hearing loss is provided wherein said noise reduction is adapted to suppress the noise level to be at a target difference around a time varying and frequency dependent masking level for a specific hearing aid system user, and wherein said time varying and frequency dependent masking level for a specific hearing aid system user is determined by: - determining excitation patterns in auditory bands, using a filterbank with envelope extraction; - carrying out spectral mask computation incorporating cross frequency masking for the specific hearing loss of said hearing aid system user; - determining broader level dependent auditory filters for said specific hearing loss of said hearing aid system user; - incorporating the hearing threshold of said specific hearing loss in the resulting masking level; and - determining forward masking by applying a temporal decay to the determined resulting masking level. In still other variations a non‐transitory computer readable medium carrying instructions which, when executed by a computer, cause the methods of the disclosed embodiments to be performed.

Claims

CLAIMS 1. A method of operating a hearing aid system comprising the steps of: - receiving sound by the hearing system; - generating a processed sound adapted to at least alleviate a specific hearing loss; - presenting the processed sound; - determining, based on the received sound, a time varying and frequency dependent masking level for said specific hearing loss, wherein said determining comprises the steps of: - - a) determining excitation patterns in auditory bands, using a filterbank with envelope extraction; - - b) carrying out spectral mask computation incorporating cross frequency masking for said specific hearing loss; - - c) determining broader level dependent auditory filters for said specific hearing loss; - - d) incorporating the hearing threshold of said specific hearing loss in the resulting masking level; and - - e) determining forward masking by applying a temporal decay to the determined resulting masking level; and - using said time varying and frequency dependent masking level, to obtain optimized hearing aid system performance with respect to at least one of: - - i) minimizing speech artifacts by adaptively constraining the magnitude of noise reduction as a function of said time-varying and frequency-dependent masking level, such that noise suppression is selectively limited in regions where the target signal is perceptually dominant; - - ii) reduction of power consumption for audio streaming by reducing the transmission bit-rate and shaping the resulting quantization noise to be below the level of said time varying and frequency dependent masking level; and - - iii) improving probe noise based feedback cancellation performance by enabling use of a higher level of masked probe noise based on the higher level of said time varying and frequency dependent masking level for said specific hearing loss.

2. The method according to claim 1, wherein step a) comprises the use of a complex gammatone filterbank.

3. The method according to claim 1, comprising the further step of determining whether the input signal is tonal or a broadband signal and in response hereto adjusting the time varying and frequency dependent masking level for said specific hearing loss.

4. The method according to claim 1, wherein said step of minimizing speech artifacts by adaptively constraining the magnitude of noise reduction as a function of a time-varying and frequency-dependent masking level such that noise suppression is selectively limited in regions where the target signal is perceptually dominant comprises the following steps: - a) receiving an input audio signal comprising a mixture of a target signal and noise; - b) processing the input audio signal using a neural network, wherein said neural network is trained to: - - estimate a noise-reduced output signal based on said input audio signal; and - - provide a noise-reduced output signal; - c) dynamically adjusting the operation of the neural network to provide a predefined target signal-to-noise ratio in the output signal; wherein the neural network is further adapted to minimize the perceptual signal artifacts introduced during noise reduction by: - - i) incorporating a loss function during training of said neural network that penalizes perceptual distortion in the output signal relative to the input signal and/or - - ii) conditioning the neural network on an artifact control parameter indicative of a trade-off between noise suppression and audio fidelity of the output signal.

5. A hearing aid system adapted to carry out the method steps according to any of the preceding claims 1 – 4.

6. A method of training a neural network to provide noise reduction, for a specific hearing aid system, wherein said noise reduction is adapted to suppress the noise level to be at a target difference around a time varying and frequency dependent masking level for a specific hearing aid system user, wherein said time varying and frequency dependent masking level for a specific hearing aid system user is determined by: - determining excitation patterns in auditory bands, using a filterbank with envelope extraction; - carrying out spectral mask computation incorporating cross frequency masking for the specific hearing loss of said hearing aid system user; - determining broader level dependent auditory filters for said specific hearing loss of said hearing aid system user; - incorporating the hearing threshold of said specific hearing loss in the resulting masking level; and - determining forward masking by applying a temporal decay to the determined resulting masking level.