GB2586817A - A method for automatically adjusting a hearing aid device based on a machine learning - Google Patents

Abstract

A signal processing parameter of a hearing aid 2 is adjusted by a user 1 operating a user control 4 which may be provided on a smartphone 3. The adjustments are recorded in a memory together with time information provided by a clock. The recorded adjustment information, volume, gain, loudness compression, noise cancellation, beam forming, is analysed based on the assumption that there are periodic patterns, for example patterns reoccurring each day, week or month. Based on the analysis adjustments are carried out automatically. For example, it is possible to detect if the hearing aid user 1 prefers soft sounds in the morning and loud sounds in the afternoon and to adjust the hearing aid accordingly (fig 2). The analysis may include calculating a polynomial trendline, a moving average or spline interpolation (figs 3a-4c). The adjustment information may also be sent to a server 8 over the internet 7 allowing the aggregation of big data and analysis across multiple hearing aid users.

Description

A method for automatically adiustina a hearina aid device based on machine learning

Technical Field

The invention relates to the field of hearing aid devices. More particularly, it relates to a method for automatically adjusting a hearing aid device by machine learning and a hearing system for carrying out the method.

Background of the Invention

A hearing aid device is a device for aiding an individual in regard to its hearing. It 10 may be a hearing prosthesis for compensating a hearing loss, namely a conventional acoustic hearing aid amplifying sound or a cochlear implant electrically stimulating nerve cells.

Hearing aid devices need to be adjusted to the needs and preferences of their users, a process also referred to as personalization. Conventionally, the adjustment is done either manually or automatically. The manual adjustment is typically done by the user and/or by a hearing care professional. The automatic adjustment is typically done based on a sound environment classifier which determines the current acoustic scene such as "speech in noise" or "music". It is also known to adapt hearing aids based on a location, e.g. determined by GPS.

However, there are also hearing aid devices which employ a so called "preference learning" approach where an automatic adjustment is carried out based on previous manual adjustment by the user.

The EP 1 802 169 A2 by Fischer discloses a special method for controlling the amplification of a hearing aid. Amplification is adjusted as a function of the switch-25 off duration. An acceptable volume can thus be achieved for instance when the device is switched on for the first time in the morning.

The US 2002 / 044 669 by Meyer discloses a hearing aid with an automatic choice of a hearing program. The current time of day and the day of the week can influence the choice of the active hearing program.

The DE 10 2012 / 212 172 by Hannemann discloses a hearing aid with an automatic configuration based on calendar entries.

The US 6,330,339 by Ishige discloses a hearing aid with a condition detection unit. Conditions such as sleep, tension and motion are detected The EP 0 964 603 by Hansen discloses generating control parameters using parameters comprising time of day, ambient temperature, ambient air humidity, ambient light, telecoil detection, voice recognized spoken control words, pulse rate of the user etc.

Summary of the Invention

It is an object of the invention to provide a method for automatically adjusting a hearing aid device by machine learning, which is able to take into account that a user may have different preference during a certain period, for example different preferences during a day, namely in the morning and in the evening.

This object is achieved by the method as defined in claim 1. The user is allowed to adjust a signal processing parameter, which is then stored together with context information.

The method of claim 5 is advantageous in that the context information does not only comprise the time but further context parameters such as the location.

The method of claim 7 is advantageous in that opens up the possibility to set the clock in the hearing aid device in a user friendly manner.

It is a further object of the invention to provide a hearing system for carrying out 25 the method of claim 1.

This object is achieved by the device as defined in claim 10.

Further embodiments and advantages of the invention are described with reference to the attached drawings.

Brief Description of the Drawings

Below, the invention is described in more detail by referring to the drawings 5 showing exemplified embodiments.

Fig. 1 is a schematic view of a system for implementing the method according to the invention, Fig. 2 is a diagram showing a volume setting in dependence of the time at the beginning of the system usage; Fig. 3a, 3h and 3c are a diagrams showing a volume setting in dependence of the time; Fig. 3a shows the situation at the beginning with no recordings yet; Fig. 3b shows recorded settings after two weeks of usage; Fig. 3b shows recorded settings after four weeks of usage; Fig. 4a, 4b and 4c are a diagrams showing a volume setting in dependence of the time with interpolation.

Fig. 4a shows an interpolation by polynomial regression. Fig. 3b shows an interpolation by splines.

Fig. 4c shows an interpolation by moving average.

The described embodiments are meant as examples and shall not confine the invention.

Detailed Description of the Invention

Fig. 1 is a schematic view of a system for implementing the method according to the invention. A user 1 wears a hearing aid device 2. The user 1 adjusts the volume of the hearing aid device 2 by a volume control 4 on a smartphone 3. The smartphone is connected to the internet 7. Data in regard to adjustments is sent to a sever 8.

Fig. 2 is a diagram showing a volume setting 23 in dependence of the time. The horizontal axis 21 is the time, the vertical axis 22 the volume. The diagram illustrates a reoccurring pattern. A user prefers softer settings in the morning 25 and louder settings in the afternoon 26. The period of the reoccurrence is one day 24.

Fig. 3a, 3h and 3c are a diagrams showing a volume setting in dependence of the time. The horizontal axis 31 is the time, labeled with hours. The vertical axis is a volume setting in the range from -4 to 6. Fig. 3a shows the situation at the beginning with no recordings yet. Fig. 3b shows recorded settings 33 after two weeks of usage with an interpolation curve 34. Fig. 3b shows recorded settings 35 after four weeks of usage with an interpolation curve 36.

Fig. 4a, 4b and 4c are a diagrams showing a volume setting in dependence of the time together with different interpolations. The horizontal axis 41 is the time and 20 the vertical axis 42 the volume setting. The dots 43 represent points or raw data.

Fig. 4a shows an interpolation 44 by polynomial regression, which is based on the following equation: Y = Po ± P2x2 P3x3 firixn e Polynomial interpolation has the advantage of being easy to derive and resulting in an optimal curve relatively quickly. However, the precision of the adaptation depends largely on the degree of the polynomial. If there are few data points or if the data points are located in an unfavorable way there may by false maxima (or oscillations) resulting from the interpolation (Extreme example: adaptation of a polynomial of degree four to two data points). The reduction of the influence of old data points is to be implemented by the deletion of these old data points.

Fig. 4b shows an interpolation by splines, which is based on the following equation: 51(x) = a1 + (x -xi) + c (x -xi) + (x -xi)3 Splines have the advantage of being calculatable with a low linear computational effort and resulting in relatively well usable curve shapes. However, splines have a lower order of convergence than polynomials (i.e. the speed with which the members of a convergent sequence approach the limit value, or the speed with which the optimum curve is reached). Data fluctuating quickly over time can be relatively well represented, however it takes longer than in the case of polynomials until a good fit of the interpolation curve is achieved. As in the case of polynomials, the reduction of the influence of old data points is to be implemented by the deletion of these old data points.

Fig. 4c shows an interpolation by moving average, which is based on the following equation: n-1 Pm + Pm-i+ +PM-(n-1) 1\ 15sm Pm-L 1-0 A moving average corresponds to a low-pass filtering and is generally also easy to be carried out. However, depending on the application ( [tO;tn] vs. [t-n;tn] ), there may result curve movements (lags). The width of the window influences the effect of outliers or the resolution of the curve fitting. Data varying quickly over time can be better represented with a small window indeed, however in this case also outliers have a greater effect. Again, as in the case of polynomials, the reduction of the influence of old data points is to be implemented by the deletion of these old data points.

Alternatively a weighted moving average may be applied based on the following 25 equation: average(o= average(n_i)-(w -1) + NewValuei The advantage of such a leaky average is that current data is weighted more heavily and thereby the change of the underlying structure of the data is regarded, i.e. the forgetting of old data is done implicitly.

Original fitting (or basic setting): The hearing aid device is adapted to the needs of a hearing aid device user by a so called fitting. Usually an audiogram is measured and a so called fitting formula is applied to determine a gain model specifying gain in a frequency and input level dependent manner.

User controllable signal processing parameter: The most basic user controllable signal processing parameter is the volume setting, which may also be referred to as gain setting. Other parameters are bass, treble, loudness compression, sound cleaning actuator settings such as noise canceller strength and beam former strength, beam former direction, gain for specific frequencies or bands and left-right balance. The parameter may be "global" or sound type specific, e.g. just for the hearing program "music" or just for "loud sounds".

Parameter Range. The user controllable signal processing parameter may have certain range and a certain resolution. A volume setting may for example be between -10 and +10 with an initial setting of 0 and a step size of 1. The parameter specifies a difference to the original fitting and can therefore be referred to as delta. The minimum may be defined such that the hearing aid device produces sounds near to the hearing threshold (HL) of the user. The maximum may be defined such, that the hearing aid device produces sounds near to the uncomfortable level (UCL) of the user. The invention is especially beneficial for continuous or near continuous parameters. However, it may also be applied to parameters specifying certain discrete classes, such as "speech", "quiet" and "music".

Step size: The steps size is preferably selected such that a change of one step is well perceivable by the user. Accordingly, a volume control may have a step size of about 5dB.

User control: The user controllable signal processing parameter is adjusted by a 30 user interface or user control. The most traditional interface is an adjustment wheel at the hearing aid device coupled to a potentiometer. The hearing aid device may also be provided with push buttons, rocker switch or a scroll wheel. The adjustment can also be done by an external device such as a remote control or a smartphone with a remote control app or function.

Clock: For optimally implementing the invention it is desirable to obtain a real clock information, i.e. not only the operation time, but the date and the actual time of the day. Thereby it can be detected if the user has different preferences in the morning and the evening or on weekdays and weekends. Date and time may be obtained from a hearing aid, from a smartphone, from a fitting computer, from the internet, i.e. from a server or cloud. Date and time may also be received from a radio transmitter, from a hearing aid charger or via a user interface on the hearing aid. To synchronize the time information, for example between a smartphone and the hearing aid, it is sufficient if there is a temporary communication link. The transmitted data may not only contain the date and time, but also information regarding the next daylight-saving time adjustments.

Context information: The most basic context information in regard to the present invention is the time. However, there are other context indicators, such as location, movement, sound environment, physiological condition, hearing intention and calendar information. The context information can be derived from the environment microphone signals, from sensor data in general, from date received from an external device and/or from data received from the user. The context information may be provided as continuous parameters or divided into classes.

Location: The location may be an explicit location as defined by a GPS coordinate. However, it may also be a location class such as "home", "outdoors", "train station", "in a building", "in an office", "in a mall", "in a theater" etc. Movement: The movement may be an explicit movement as defined by a velocity or acceleration vector. However, it may also be a movement class such as "stationary (sitting, standing)", "waking", "running", "riding on a train/vehicle" etc. Sound environment: The sound environment may be characterized by various 30 acoustic measures, such as the overall level, the noise level, spectral characteristics, signal-to-noise ratio as well as measures regarding a sound-source analysis, such as the number of sound sources, the number of conversational partners, the number of interesting sound sources, the number of disturbing sound sources.

Physiological condition: The context information may comprise physiological information, for example the pulse or heart rate of the user, the blood oxygenation, the skin temperature, the skin blood perfusion, the respiration rate, the heart rate variability, the blood pulse wave, the blood glucose, electrophysiological information, the brain activity, skin conductance and derived psychophysiological measures such as activity, arousal and stress. The information may be determined not only by sensors, but also by conducting a questionnaire. Physiological parameters are likely to show a circadian rhythm.

Hearing intention (also referred to as hearing activity): The user may want to listen to a specific speaker. The user may also just want to relax without any specific listening. It remains a big challenge for hearing systems to detect the hearing intention. The most reliable way to do that is an explicit input by the user. However, it is also possible to derive the hearing intention from indicators such as the heart rate.

Joint context information: Two or more types of context information may be applied 20 jointly, resulting in classes such as "Monday morning in the office" and "Monday morning at home" or "Evening with stress" and "Evening without stress".

Time period (or time frame): The most basic time period in regard to the present invention is one day. However, the present invention can also be implemented based on other time periods such as one week, one month or one year. The time 25 periods can also be combined.

Combined time periods: Different time periods can be combined, for example by detecting both, daily and weekly patterns. There may be, for example, a separate handling of workdays and weekend days. There may also be two separate interpolations, one for the daily pattern and one for the weekly pattern.

Minimum learning period / sufficient number of inputs: Before automatic modifications can be applied there must be a sufficient amount of data. If the time period is one day, the system may collect data during one week before the first automatic modification. Alternatively, the start of automatic modifications may not be determined by a time period but by a minimum number of user inputs instead. The system may for example require a minimum number of twenty inputs. The start of the automatic modification may also depend on the variability of the modifications. If there is a clear trend the automatic modification is started. If the inputs are purely random, there may never by an automatic modification. Finally, the start of the automatic modification can be initiated by the user or the hearing car professional.

Maximum learning period / learning termination: The system may be configured to learn infinitively. However, it is also possible to specify a learning termination time. For example, learning may be only active for one year. After that, the learnt modifications are applied, but there is no more change to them. Learning can also be terminated automatically if there are fewer and less significant (e.g. smaller than 2dB) adjustments. Learning can also be terminated, if the delta differs from the basic setting by more than 20dB. Terminating the learning has the advantage that processing resources and power can be saved.

Variability of modifications: Modifications may be grouped in time segments. If the modifications within the time segment are similar, they are marked as homogeneous. If the modification within the time segment differ by more than a pre-defined threshold (e.g. 6dB) they are marked as heterogeneous.

Time segment: The time period (for example one day) may for the statistical 25 evaluation be divided into time segments (for example one hour).

Logging (or data storage or memory): According to the invention data is collected in regard to user actions and/or the state of the user controlled parameter. The data can be stored in the hearing aid device and/or in an external device. The trigger for storing data may be a user action or a clock impulse.

User action based logging: The system may save the action "user changes volume from +5 to +6" together with a time stamp and/or context information shortly after the user has actuated the user control. A duration information indicative of how long the modification was applied may be added when there is the next user action or during a logging data preprocessing.

Clock based logging: The system may save the information "volume is +6" optionally together with a time stamp and/or context information, with a certain interval or frequency, for example each minute.

External device: The external device may be a smartphone, a tablet computer, a 10 smartwatch and/or on a server accessed over the internet also referred to as cloud.

Data processing: The stored data is processed for deriving automatic adjustments. The processing may be a statistical analysis, in particular an interpolation and averaging. The processing may be carried out on the hearing aid, in an external 15 device such as a smartphone and/or on a server accessed over the internet Data preprocessing: In a first processing step, the data may be mapped to a single scale, for example data from several days is mapped to a single day.

Interpolation and averaging: Interpolation and averaging can be done by different mathematical methods, such as polynomial regression, polynomial trendline, moving average, weighted moving average and spline interpolation.

Exceptions: Exceptions may be defined for certain events such as concerts, meetings or vacation. The exceptions may be derived from calendar information. This may happen automatically or manually on request of the user.

Learning and forgetting: It can be assumed that the user preferences change over time, for example because the user is getting used to the hearing aid device or because there is an additional hearing loss. This can be accounted for by adding a weighting parameter. A weighting time constant of one year may specify that logged values are only accounted with half the weight after one year.

Preferred embodiment: According to a preferred embodiment the user controllable parameter is the volume, the range is -10 to +10, the context information the time, the period is one day, the time segment is one hour and the weighting time constant one year. After a learning period of about a week the volume is steered automatically based on the time.

Alternative solution without learning: Alternatively, the fitting software may be provided with a functionality which allows to define time dependent parameters. If it is known that the client socially interacts mostly on the weekend, speech enhancement features such as sound cleaning may be set to be stronger on weekends.

Claims (1)

1. Claims 1. A method for automatically adjusting a hearing aid device (2) by machine learning comprising the steps: A) Predefining a time period (22); B) Manually adjusting a signal processing parameter (21) by a user control (4); C) Obtaining context information comprising at least the step of obtaining time information from a clock; D) Storing data regarding said signal processing parameter (21) together or in dependence of said context information or data derived from said context information in a memory thereby obtaining stored data; E) Repeating steps C and D at least once; F) Processing said stored data thereby obtaining a result; G) Automatically adjusting said signal processing parameter (8) based on said result; 2 The method of claim 1, wherein the signal processing parameter (21) is one of the following: volume, gain, loudness compression, noise canceller strength, beam former strength.3. The method of one of the preceding claims, wherein the time period (22) is one of the following: a day, a week, a month.4 The method of one of the preceding claims, wherein said processing is one of the following: calculating a polynomial trendline, calculating a moving average, calculating a spline interpolation.The method of one of the preceding claims, wherein said context information further comprises one or more of the following: location, movement, environment sound class, a number of environment sound sources, physiological information, psychophysiological information, pulse, electrophysiological information, data obtained from an external device (3), in particular calendar information, data entered by said user (1), in particular data regarding his or her hearing intentions.6 The method of one of the preceding claims, wherein said clock is comprised in one or more of the following: said hearing aid device (2), an external device, a smartphone (3), a charger, a fitting computer, the internet (7), a server (8), a cloud.7 The method of claim 6, further comprising the step of synchronizing a clock between said hearing aid device (2) and said external device by transmitting data from said external device to said hearing aid device (2).8 The method of one of the preceding claims, wherein said user control (4) is comprised in one of the following: said hearing aid device (2), an external device (3), a smartphone (3).9 The method of one of the preceding claims, wherein said memory (6) is comprised in one of the following: said hearing aid device (2), an external device, a smartphone (3), a server (8), a cloud (9); 10. A hearing system for carrying out the method according to one of the previous claims, comprising at least one hearing aid device (2), a user control (4), a clock and a memory.