EP3761668B1

EP3761668B1 - Hearing device for providing position data and method of its operation

Info

Publication number: EP3761668B1
Application number: EP19183970.3A
Authority: EP
Inventors: Andreas Breitenmoser; Simon Berger
Original assignee: Sonova AG
Current assignee: Sonova Holding AG
Priority date: 2019-07-02
Filing date: 2019-07-02
Publication date: 2023-06-07
Anticipated expiration: 2039-07-02
Also published as: EP3761668A1

Description

TECHNICAL FIELD

This disclosure relates to a hearing device comprising a housing and a processing unit configured to generate position data of the housing with respect to a reference point, according to the preamble of claim 1. The disclosure further relates to a method of operating a hearing device for generating position data, according to the preamble of claim 14, and a computer-readable medium according to claim 15.

BACKGROUND

Hearing devices may be used to improve the hearing capability or communication capability of a user, for instance by compensating a hearing loss of a hearing-impaired user, in which case the hearing device is commonly referred to as a hearing instrument such as a hearing aid, or hearing prosthesis. A hearing device may also be used to produce a sound in a user's ear canal. Sound may be communicated by a wire or wirelessly to a hearing device, which may reproduce the sound in the user's ear canal. Hearing devices are often employed in conjunction with communication devices, such as smartphones, for instance when listening to sound data processed by the communication device and/or during a phone conversation operated by the communication device. More recently, communication devices have been integrated with hearing devices such that the hearing devices at least partially comprise the functionality of those communication devices.
Different types of hearing devices can be distinguished by the position at which they are intended to be worn at an ear of a user. Some types of hearing devices comprise a behind-the-ear part (BTE part) including a housing configured to be worn at a wearing position behind the ear of the user. The housing of the BTE part can accommodate functional components of the hearing device. Hearing devices with a BTE part can comprise, for instance, receiver-in-the-canal (RIC) hearing aids and behind-the-ear (BTE) hearing aids. Other functional components of such a hearing device may be intended to be worn at a different position at the ear, in particular at least partially inside an ear canal. For instance, a RIC hearing aid may comprise a receiver intended to be worn at least partially inside the ear canal. The receiver may be implemented in a separate housing, for instance an earpiece adapted for an insertion and/or a partial insertion into the ear canal. A BTE hearing aid may further comprise a sound conduit intended to be worn at least partially inside the ear canal. Other types of hearing devices, for instance earbuds, earphones, and hearing instruments such as in-the-ear (ITE) hearing aids, invisible-in-the-canal (IIC) hearing aids, and completely-in-the-canal (CIC) hearing aids, commonly comprise a housing intended to be worn at a position at the ear such that they are at least partially inserted inside the ear canal. A displacement detector may be integrated with a housing of a BTE part and/or a housing intended to be worn at least partially inside the ear canal and/or any other housing of the hearing device worn at the user's ear, such as a beam or bracket of a headphone.
Position data of a hearing device housing with respect to a reference point can be useful for a variety of applications. For instance, hearing devices are often employed to reproduce or amplify a sound originating from a sound source generally located at a distance to the housing. The reference point can be defined by a momentary position of the sound source. The reference point may be stationary or moving relative to a reference frame of the earth's surface. To illustrate, the sound source may be another person, such as a conversation partner, an audio reproduction device, such as a portable radio, a transmission unit for transmitting audio signals, such as signals captured by a remote microphone, and/or the like. At the same time, the housing worn by the user may also be moving relative to the earth's reference frame. In particular, the user may turn his head or shake his head or move his body, for instance by walking around. Those user displacements, however, generally deviate from the displacement behavior of the sound source at the reference point. For instance, the user displacements can be often more irregular and/or faster than the sound source displacements, at least as compared to a long term average of the sound source position. Consequently, the angular orientation and spatial location of the housing relative to the sound source varies.
This can lead to adverse effects for reproducing the sound provided by the sound source. As an example, a signal reception by the hearing device of a signal received from the sound source may be compromised at different positions relative to the reference point. As another example, optimized configurations of a beamformer implemented in the hearing device may depend on the sound source position relative to the hearing device. Thus, changing the position can reduce the quality of the beamformed signal. As yet another example, the hearing device may be configured to process audio signals in a manner to artificially create a spatial hearing perception depending on the sound source position. Changing the relative position can then produce a distorted perception of the intended spatial resolution, for instance when the artificially created audio signal is based on inaccurate or incorrect position data. Information about a momentary position of the housing with respect to the sound source at the reference point during or after the displacements could be employed to counteract those adverse effects. For instance, optimized configurations related to the signal perception and/or beamforming and/or spatially resolved audio signal could then be adjusted accordingly.
Other applications may employ position data relative to a reference point other than a momentary sound source position. For instance, a signal transmission from a wireless signal source to the hearing device may be enhanced when the position data relative to the source is taken into account. Furthermore, position data relative to a reference point that is stationary with respect to the earth's reference frame can be useful, for instance, to track a momentary position of the user wearing the hearing device relative to the stationary reference point. In some applications, the position information can be transmitted to an external device for a further evaluation.
Position data can be provided in a hearing device by employing a plurality of microphones. In particular, audio signals from a sound source detected by the microphones can be analyzed with respect to a relative angle of the hearing device to the sound source. European Patent No. EP 3 248 393 B1 discloses such a hearing device comprising two hearing units in a binaural configuration. The hearing units include a microphone arrangement for audio detection. Audio signals detected locally at opposite ears by each hearing unit can be exchanged via a binaural link. Determining an interaural difference between those audio signals allows to estimate an angular orientation of the hearing units relative to the sound source. The required signal evaluation, however, can be rather processing intensive. Thus, a rather fast and uninterrupted detection of the position, which would be desirable in many applications, can exhaust the technical limits of the processor. In addition, the audio signals obtained by the microphones can be affected by environmental disturbances limiting a reliability of the position data.
A displacement of the hearing device relative to the earth's reference frame can also be determined by an inertial sensor. European patent application No. EP 19166417.6 discloses a hearing device comprising an inertial sensor included in the housing configured to provide such displacement data. An orientation of the housing can be estimated from the displacement data by providing, during a walking activity of the user, calibration data relative to a reference orientation, and determining a deviation of the housing position from the reference orientation based on the calibration data and the displacement data. Integrating the displacement data over time can thus provide continuous information about the degree of the deviation of the housing orientation with respect to the reference orientation. The integration, however, can be flawed by numerical errors rendering the position data rather unprecise, at least when the integration is performed for a prolonged time span.
Conventional methods of providing the position data in a hearing device have a further drawback that a displacement behavior of the user wearing the hearing device is not adequately taken into account. Typically, the user frequently exerts quite irregular and rather fast rotational and translational movements of his head and body during daily usage of a hearing device. Those user movements may comprise a number of individual short-term movement actions, such as shaking the head and/or temporarily turning the body, which can at least partially cancel out or balance each other on a more long-term average. As a consequence, the user movements can lead to negative side effects for the position data generation. For instance, the user movements can mask or disturb a recognition of a rather homogeneous average position over an extended time, in particular a rather uniform steady state position, of the hearing device with respect to the reference point. Many applications of the position data obtained in a hearing device, however, such as the examples illustrated above, rely on a relative position to the reference point independent from said irregular user movements. The position data disregarding the user's movement behavior can therefore limit those applications in an undesired way.
US 2019/0110137 A1 discloses a binaural hearing system comprising two hearing aids each comprising a set of microphones, an electronic monaural signal transducer configured to receive an electronic monaural signal provided by an external device linked with a sound source, such as a spouse microphone and a television (TV), a direction of arrival (DOA) estimator configured to correlate the output signals provided by each set of microphones with the electronic monoaural signal to provide directional transfer functions, and a binaural filter configured to process the electronic monaural signal based on the directional transfer functions such that the electronic monaural signal is perceivable by a user wearing the hearing aids as arriving from the sound source. The binaural hearing system further comprises head tracker including an inertial measurement unit, such as an accelerometer and a gyroscope, for determining head yaw, head pitch, head roll, and head displacements when the user wears the binaural hearing system. In a case in which the head tracker has detected no, or insignificant, head movements, the directional transfer functions are determined in the above described way, and, in a case in which the head tracker has detected head movements, the determined directional transfer functions are modified in accordance with the detected change of orientation of the user's head. In this way, when the user is moving his head, the DOA of the emitted sound can be determined based on a tracking signal provided by the head tracker, which is calibrated based on the electronic monaural signal whenever the head of the user is kept still.
EP 2 891 898 A1 discloses a mobile device configured to determine a distance between a loudspeaker and itself based on a link quality indicator (LQI) which is included in digital data received from the loudspeaker, and to adjust a transmission power level (TPL) of a signal transmitted to the loudspeaker depending on the distance. The TPL is based on a bit error rate (BER) of a wireless signal transmitted from the mobile device to the loudspeaker, which may not be enough to correlate the LQI to the distance d because the LQI is also subject to other losses. The mobile device thus further includes an accelerometer and a compass providing acceleration and orientation data that is input to an extended Kalman filter (EKF) to determine a filtered TPL in order to match the TPL determined based on the LQI more closely to a theoretical model of the TPL under the assumption that stronger TPL readings correlate to distance and orientation of mobile device relative to the loudspeaker, which is then used to determine the distance and to adjust the TPL.

SUMMARY

It is an object of the present disclosure to avoid at least one of the above mentioned disadvantages and to provide a hearing device and/or a method of operating the hearing device allowing to provide position data of the housing with respect to a reference point in a more reliable way, in particular such that data errors and/or inaccuracies can be decreased and/or temporal delays during data generation can be reduced. It is another object to allow a substantially continuous generation of position data in subsequent periods of a rather short time, in particular within a predetermined time constraint. It is a further object to provide the position data in a way allowing to account for undesired side effects provoked by movements of a user wearing the hearing device, in particular to provide a correction of those side effects in the position data. It is yet another object to allow the generation of the position data with respect to a reference point at least temporarily moving relative to the earth's reference frame. It is another object to apply the position data for an improved hearing device operation, in particular regarding applications involving a remote source at the reference point such as a sound source and/or a radio wave source.
At least one of these objects can be achieved by a hearing device comprising the features of patent claim 1 and/or in a method of operating a hearing device comprising the features of patent claim 14. Advantageous embodiments of the invention are defined by the dependent claims and the following description.
The present disclosure proposes a hearing device comprising a housing configured to be worn at an ear of a user. The hearing device further comprises a displacement detector mechanically coupled with the housing. The displacement detector is configured to provide displacement data indicative of a rotational displacement and/or a translational displacement of the housing. The hearing device further comprises a processing unit communicatively coupled with the displacement detector. The processing unit is configured to collect the displacement data in subsequent periods. The processing unit is also configured to generate position data based on the collected displacement data. The position data is indicative of an angular orientation and/or a spatial location of the housing with respect to a reference point. The processing unit is also configured to obtain a reliability measure of the position data. The reliability measure is indicative of a reliability of said position data after the subsequent periods. The processing unit is also configured to adjust the position data based on said reliability measure. The processing unit is also configured to continuously generate the position data at a first frequency and to continuously obtain the reliability measure at a second frequency, wherein the second frequency is smaller than the first frequency. Thus, the position data can be provided in a reliable way based on the collected displacement data. This may be exploited to generate the position data relative to the reference point rather quick and/or up-to-date based on the collected displacement data. In particular, a rather time consuming verification of the generated position data at every time of position data generation, such as an additional position measurement relative to the reference point, can thus be avoided, wherein the adjustment of the position data based on the reliability measure can assure the desired degree of reliability. In this way, current position data relative to the reference point can be updated at rather high speed and with the desired degree of reliability. This opens up new possibilities for various hearing device operations relying on a fast position data generation relative to the reference point.
Independently, the present disclosure proposes a method of operating a hearing device comprising a housing configured to be worn at an ear of a user. The method comprises providing displacement data indicative of a rotational displacement and/or a translational displacement of the housing. The method further comprises collecting the displacement data in subsequent periods. The method further comprises generating position data based on the collected displacement data. The position data is indicative of an angular orientation and/or a spatial location of the housing with respect to a reference point. The method further comprises obtaining a reliability measure of said position data. The reliability measure is indicative of a reliability of the position data after the subsequent periods. The method further comprises adjusting the position data based on said reliability measure.
Independently, the present disclosure also proposes a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause a hearing device to perform operations of this method.
Subsequently, additional features of some implementations of the hearing device and/or the method of operating a hearing device are described. Each of those features can be provided solely or in combination with at least another feature. The features can be correspondingly provided in some implementations of the hearing device and/or of the method of operating the hearing device and/or the computer-readable medium.
In some implementations, the reliability measure is obtained based on auxiliary position data. The auxiliary position data can be provided independently from the position data.
In some implementations, the hearing device comprises a sound detector configured to provide audio data to the processing unit. The audio data can be indicative of an ambient sound. The ambient sound can be defined as a sound in an environment of the housing, in particular an environment of the user. In particular, the ambient sound can include sound emitted at the reference point. The sound can be emitted by a sound source localized at the reference point. The reference point may thus be provided by a position of a sound source, for instance a sound source moving relative to the earth's reference frame and/or a sound source having a fixed position in the earth's reference frame. The auxiliary position data may be generated from the sound emitted at the reference point. The auxiliary position data may be generated based on the audio data.
The sound detector may comprise a plurality of spatially arranged microphones each configured to provide audio data to the processing unit. The audio data provided by each microphone can be indicative of the ambient sound. It may be that a difference between the audio data provided by at least two of said spatially arranged microphones is determined. The auxiliary position data can be generated based on the difference. The difference may comprise a difference in phase and/or a difference in signal level. The audio data between which the difference is determined may be selected such that the difference is indicative of a propagation direction of at least part of the ambient sound emitted at the reference point.
In some implementations, a signal to noise ratio is determined in the audio data. The reliability measure can comprise the signal to noise ratio.
It may be that a presence of a sound emitted from a sound source is determined from the audio data. The auxiliary position data can be determined such that the auxiliary position data is indicative of an angular orientation and/or a spatial location of the housing with respect to a position of said sound source. Thus, the reference point can be associated with the position of the sound source. The presence of a sound emitted from a sound source may be determined by evaluating the audio data with respect to a directionality of at least part of the ambient sound. The directionality may be defined as a direction along which said part of the ambient sound propagates to the sound detector. The directionality may be determined by evaluating audio data provided by a plurality of spatially arranged microphones.
In some implementations, audio data is obtained from a remote sound detector. The remote sound detector can be provided at a position remote from the housing. The audio data on which the output signal is based can comprise the audio data provided by the remote sound detector. The hearing device can comprise the remote sound detector. The remote sound detector can be provided at the reference point. The hearing device can comprise a signal receiver communicatively coupled with the processing unit. The signal receiver can be configured to receive the audio data from the remote sound detector transmitted by radio waves. The auxiliary position data can be generated based on the received radio waves. The reference point may be selected such that it is fixed with respect to a position at which the remote microphone is provided.
In some implementations, the hearing device comprises a signal receiver configured to receive radio waves. The radio waves can be emitted from a radio source. The radio source may be located at the reference point. The auxiliary position data can be generated based on the radio waves. The radio waves can include audio data. The audio data can be indicative of a sound detected at the reference point. The audio data can be provided by a remote sound detector.
It also may be that a presence of radio waves emitted from the radio source is determined from the radio waves. The auxiliary position data can be determined such that the auxiliary position data is indicative of an angular orientation and/or a spatial location of the housing with respect to a position of said radio source. Thus, the reference point can be associated with the position of the radio source. The presence of radio waves emitted from a radio source may be determined by evaluating the radio waves with respect to a directionality of at least part of the radio waves. The directionality may be defined as a direction along which said radio waves propagate to the signal receiver.
The signal receiver may comprise a plurality of spatially arranged receiver units each configured to receive the radio waves. A difference between the radio waves received by at least two of said spatially arranged receiver units may be determined. The auxiliary position data can be generated based on the difference. The difference may comprise a difference in phase and/or a difference in signal level.
In some implementations, the auxiliary position data is generated at the reference point. The auxiliary position data may be transmitted from the reference point to the processing unit via radio waves to obtain the reliability measure.
In some implementations, the reliability measure is obtained based on an algorithm. The algorithm may be performed by the processing unit. The algorithm may be applied to the position data and/or the displacement data, in particular the displacement data collected in the subsequent periods and/or the position data generated from the displacement data. A correction value may be determined from the applied algorithm. The adjusting the position data can be based on the correction value. For instance, the position data can be adjusted to the correction value and/or to a value depending on the correction value in a predetermined relation such as a functional dependency. The algorithm may be based on a model of an expected movement behavior of the user and/or a model describing an expected position of a sound source with respect to the housing and/or a model describing an expected deviation of the generated position data from a desired value of the position data. For instance, the algorithm can include a functional algorithm and/or a numerical algorithm and/or a statistical algorithm and/or an optimization algorithm and/or a filter algorithm and/or a classification algorithm and/or a machine learning algorithm, in particular a machine learning algorithm for training a classifier of the displacement data and/or position data.
The algorithm may comprise determining an intermediate value between a value extracted from the position data and a predefined value. The correction value can be set to the intermediate value. The predefined value can be indicative of a predefined angular orientation and/or a predefined spatial location of the housing with respect to the reference point. In this way, the correction value can be obtained by a rather low computing effort and/or rather short computational time. The predefined value may be selected such that it corresponds to an expected user behavior. For instance, the predefined value may be selected such that it corresponds to a position value of the angular orientation and/or a spatial location of the housing indicative of a viewing direction of the user wearing the housing. For instance, the predefined value can be provided as a position value of the angular orientation and/or a spatial location of the housing corresponding to a preferred listening direction of the user relative to the reference point.
The setting of the correction value to the intermediate value can be iteratively applied. In particular, the obtaining the reliability measure of the position data can be repeated at a frequency, for instance after constant and/or irregular time intervals. The setting of the correction value to the intermediate value can then be applied in a respective iteration step which is carried out at least once each time when the reliability measure is obtained. Thus, the number of iteration steps may increase at least once during obtaining the reliability measure. For instance, when a temporal variation of the position data generated from the collected position data is rather small, the correction factor may iteratively approach the predefined value while increasing the number of iteration steps. In this way, the correction value can converge to the expected user behavior.
The algorithm may comprise multiplying a value of the generated position data with a correction factor. The correction factor can be selected such that the value of the position data approaches the predefined value, in particular such that the value of the position data converges to the predefined value after a plurality of iteration steps in which said multiplying is repeated. The algorithm may comprise adding an offset value to a value of the generated position data. The offset value can be selected such that the predefined value corresponds to the offset value. The algorithm may comprise determining a variation of the position data and/or displacement data over the subsequent periods relative to a threshold. The correction value can be set to the predefined value when the variation exceeds the threshold. The threshold may comprise a minimum duration, wherein the obtaining a reliability measure further comprises determining whether a number of said subsequent periods in which said variation has been determined exceeds the minimum duration. The threshold may comprise a minimum limit, wherein the obtaining the reliability measure further comprises determining whether said variation exceeds the minimum limit.
The algorithm may comprise classifying, based on patterns of significance of displacement data and/or position data, the collected displacement data and/or the generated position data with respect to a significance level. The significance level can be indicative of a probability that the collected displacement data and/or the generated position data is significant for said angular orientation and/or spatial location of the housing with respect to the reference point. For instance, the patterns of significance can comprise a sequence of displacement data and/or position data as a function of time, in particular a sequence indicative of a trajectory. The sequence can be subject to uncertainty. In some implementations, the significance level can comprise a measure of the uncertainty. For instance, the significance level may indicate a likelihood that the sequence can be assigned to the collected displacement data and/or the generated position data. The likelihood may be determined by a predictive model of a machine learning algorithm. In some implementations, the significance level can comprise a probability that the collected displacement data and/or the generated position data assigned to the sequence can be classified as being significant for the angular orientation and/or spatial location of the housing with respect to the reference point. The probability may be determined by a predictive model of a machine learning algorithm. The correction value can be determined depending on the significance level. A record of the collected displacement data and/or generated position data may be maintained over time. The record may comprise said sequence of displacement data and/or position data. The patterns of significance may be determined from the record. In particular, the patterns of significance may be determined based on at least one feature reoccurring in said collected displacement data and/or generated position data over time. The patterns of significance may also be determined based on a correlation between said auxiliary position data and said generated position data and/or collected displacement data.
The algorithm may comprise a predictive model provided by a trained machine learning algorithm. The collected displacement data and/or the generated position data can be input into a trained machine learning algorithm, which determines the significance level. The machine learning algorithm can be configured to learn displacement data and/or position data corresponding to specific movement situations of the user. For instance, the machine learning algorithm may learn data from said sequence of displacement data and/or position data by using a statistical method. The statistical method may comprise, for instance, an Expectation Maximization (EM) algorithm and/or a Hidden Markov Model (HMM). The collected displacement data and/or the generated position data can also be input into at least two different trained machine learning algorithms, each of which determines a respective probability that the collected displacement data and/or the generated position data is significant for said angular orientation and/or spatial location of the housing with respect to the reference point. The significance level can be determined from the probabilities determined from the at least two machine learning algorithms.
It may be that at least one of the above described algorithms and/or a combination of the above described algorithms is applied to determine the correction value. It also may be that at least one of the above described algorithms and/or a combination of the above described algorithms is applied independently and/or in conjunction with said generating of auxiliary position data to obtain the reliability measure.
In some implementations, the hearing device comprises an output transducer configured to stimulate the user's hearing by outputting an output signal. The output signal may be provided based on a processing of audio data. It may be that a directionality of the output signal is provided. In particular, the directionality can be provided by amplifying a part of the audio data corresponding to a desired direction relative to another part of the audio data deviating from the desired direction. The desired direction may be determined based on the position data. The directionality of the output signal may be provided in a manner to create, when stimulating the user's hearing, a hearing perception of a sound coming from the desired direction. The desired direction may be selected such that it points to the reference point. The audio data on which the output signal is based may comprise the audio data provided by said sound detector.
The providing the directionality of the output signal may comprise beamforming based on the processing of the audio data. A property of the beamforming may be controlled based on the position data, in particular the generated position data and/or the adjusted position data. The controlling the property of the beamforming may comprise steering a directionality of the beamforming and/or adjusting a beam size, in particular a beam width, of a beam provided by the beamforming.
In some implementations, a beam width of the beam provided by the beamforming is enlarged when said position data is indicative of a variation of the angular orientation and/or a spatial location of the housing with respect to the reference point over time, in particular as compared to position data generated at an earlier time. The beam width may be defined as an angular range that is covered by the beam provided by the beamforming. In some implementations, the beam width is enlarged to a fixed value of the beam width. The fixed value may be predetermined. In some implementations, the beam width is changed to a value of the beam width depending on said variation of the angular orientation and/or a spatial location over time. For instance, the beam width may be enlarged to a larger value when a larger variation of the angular orientation and/or a spatial location of the housing over time is determined as compared to a smaller value when a smaller variation is determined.
Correspondingly, a beam width of the beam can be reduced when no variation of the position data over time is determined, for instance no variation at least for a predetermined time interval. In particular, a beam width of the beam can be reduced when the position data is indicative of a constant angular orientation and/or a spatial location of the housing with respect to the reference point over time, in particular as compared to position data generated at an earlier time. For example, the beam width may be reduced when the position data is determined to be constant for at least a predetermined time interval. In some implementations, the reference point is adjusted from an earlier reference position to a later reference position. For instance, the reference position may be defined by coordinates of the reference point in a predefined reference frame, such as the earth's reference frame. The coordinates of the later reference position may thus differ from the coordinates of the earlier reference position. Adjusting the reference point from the earlier reference position to the later reference position can comprise adjusting said position data indicative of an angular orientation and/or a spatial location of the housing by a difference between the earlier reference position and the later reference position of the reference point.
In some implementations, the adjusted reference point at the later reference position is selected such that the reference point is comprised in an angular range covered by the reduced beam width. In this way, the adjusted reference point at the later reference position may be provided as a spatial position at which a sound source is located. For instance, the reference point at the earlier reference position may correspond to a position of a first sound source, and the adjusted reference point at the later reference position may correspond to a position of a second sound source. Such a functionality may be advantageously employed to track multiple sources located at different reference points, for instance to account for a user behavior in which the user's attention changes from the first source to the second source.
In some implementations, the housing is a first housing, the hearing device comprising a second housing, the first housing and the second housing configured to be worn at two ears of the user in a binaural configuration. The displacement detector can be a first displacement detector, the hearing device comprising a second displacement detector mechanically coupled with the second housing and communicatively coupled with the processing unit. The sound detector can be a first sound detector, the hearing device comprising a second sound detector mechanically coupled with the second housing and communicatively coupled with the processing unit. Said spatially arranged microphones may be mechanically coupled with at least one of the first housing and the second housing.
The position data is continuously generated at a first frequency and the reliability measure is continuously generated at a second frequency. The second frequency is smaller than the first frequency. In particular, the second frequency can be selected such that the position data is generated multiple times before the reliability measure is obtained. In particular, the first frequency can be selected such that the position data is generated each time after collecting said displacement data for a predetermined number of said subsequent periods. The first frequency can be indicative of a repetition rate in which the reliability measure is generated and the second frequency can be indicative of a repetition rate in which the reliability measure is obtained. The first frequency and/or the second frequency can be constant and/or vary with time. For instance, the generating the reliability measure and/or the obtaining the reliability measure may be repeated in irregular time intervals such that the first frequency and/or the second frequency may vary.
In some implementations, the displacement detector comprises an inertial sensor. The inertial sensor may be provided by an accelerometer. In some implementations, the processing unit is configured to transmit the position data to an auxiliary device. The auxiliary device may be configured to further process the position data and/or to store the position data in a memory. The auxiliary device may comprise a display. The auxiliary device may be configured to graphically reproduce the position data on the display. The auxiliary device may be a communication device, in particular a smartphone, tablet and/or the like.
The displacement data may be indicative of a rotational displacement and/or a translational displacement of the housing relative to a reference frame of the earth and/or any other reference frame. In some implementations, said generating position data comprises integrating data acquired from said collected displacement data over said subsequent periods.
In some implementations, the obtaining the reliability measure comprises determining a variation of the position data and/or displacement data over said subsequent periods relative to a threshold. The threshold may comprise a minimum duration, wherein the obtaining a reliability measure further comprises determining whether a number of said subsequent periods in which said variation has been determined exceeds the minimum duration. The threshold may comprise a minimum limit, wherein the obtaining the reliability measure further comprises determining whether said variation exceeds the minimum limit.
The subsequent periods may comprise a first number of periods and a second number of periods. The generating position data can comprise generating first position data based on said displacement data collected within the first number of periods, and generating second position data based on said displacement data collected within the second number of periods. The number of the subsequent periods may be continuously increased with time, wherein the first number of periods corresponds to a number of the subsequent periods at an earlier time, and the second number of periods corresponds to a number of said subsequent periods at a later time. The obtaining the reliability measure may comprise determining an intermediate value between said first position data and said second position data. The adjusting the position data may comprise setting the position data to the intermediate value. The number of the subsequent periods may comprise a current number of periods corresponding to a number of said subsequent periods at a current time. The generating position data can comprise generating current position data based on the displacement data collected within the current number of periods. The obtaining the reliability measure can comprise replacing the first position data with the second position data; and replacing the second position data with the current position data.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. The drawings illustrate various embodiments and are a part of the specification. The illustrated embodiments are merely examples and do not limit the scope of the disclosure. Throughout the drawings, identical or similar reference numbers designate identical or similar elements. In the drawings:

Figs. 1-4: schematically illustrate exemplary hearing devices including a processing unit, an output transducer, and a displacement detector, in accordance with some embodiments of the present disclosure;
Figs. 5-13: illustrate exemplary methods of providing position data in a hearing device, in accordance with some embodiments of the present disclosure;
Fig. 14: schematically illustrates a processing module for performing an algorithm to obtain a reliability measure for position data in a hearing device, in accordance with some embodiments of the present disclosure;
Fig. 15: schematically illustrates a displacement detector, in accordance with some embodiments of the present disclosure;
Figs. 16A - D: schematically illustrate operations of a hearing device for providing position data in different movement situations, in accordance with some embodiments of the present disclosure;
Figs. 17A, B: schematically illustrate operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure;
Figs. 18A, B: schematically illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure;
Figs. 19A - C: schematically illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure; and
Figs. 20A, B: schematically illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring to FIG. 1, a hearing device 100 according to some embodiments of the present disclosure is illustrated. As shown, hearing device 100 includes a processing unit 102 communicatively coupled to a sound detector 106, a displacement detector 108, and an output transducer 110. The hearing device components 102, 106, 108, 110 are included in a housing unit 111. In some implementations, housing unit 111 can be provided as a single housing configured to be worn at an ear of a user, for instance at a wearing position behind the ear and/or at least partially inserted into the ear canal. In some implementations, housing unit 111 can comprise two separate housings. For instance, at least one of hearing device components 102, 106, 108, 110 can be included in a first housing of housing unit 111 configured to be worn behind the ear and other hearing device components 102, 106, 108, 110 can be included in a second housing of housing unit 111 configured to be at least partially inserted into the ear canal. In some implementations, at least one of hearing device components 102, 106, 108, 110 is provided externally from housing unit 111. Hearing device 100 may include additional or alternative components as may serve a particular implementation.
Hearing device 100 may be implemented by any type of hearing device configured to enable or enhance hearing of a user wearing hearing device 100. For example, hearing device 100 may be implemented by a hearing aid configured to provide an amplified version of audio content to a user, an earphone, a cochlear implant system configured to provide electrical stimulation representative of audio content to a user, a sound processor included in a bimodal hearing system configured to provide both amplification and electrical stimulation representative of audio content to a user, or any other suitable hearing prosthesis.
Sound detector 106 may be implemented by any suitable audio detection device, such as a microphone or a plurality of microphones, and is configured to detect a sound presented to a user of hearing device 100. The sound can comprise ambient sound such as audio content (e.g., music, speech, noise, etc.) generated by one or more sound sources included in an environment of the user. The sound can also include audio content generated by a voice of the user during an own voice activity, such as a speech by the user. Sound detector 106 is configured to output an audio data comprising information about the sound detected from the environment of the user. Sound detector 106 may be included in or communicatively coupled to hearing device 100 in any suitable manner. Output transducer 110 may be implemented by any suitable audio output device, for instance a loudspeaker of a hearing device or an output electrode of a cochlear implant system, configured to output an output signal to stimulate the user's hearing.
Displacement detector 108 may be implemented by any suitable sensor configured to provide displacement data indicative of a rotational displacement and/or a translational displacement. In particular, displacement detector 108 can comprise at least one inertial sensor. The inertial sensor can include a motion sensor, for instance an accelerometer, and/or a rotation sensor, for instance a gyroscope and/or an accelerometer. Alternatively or additionally, displacement detector 108 can comprise an optical detector such as a camera. For instance, the optical detector can be employed as a motion sensor and/or a rotation sensor by generating optical detection data over time and evaluating variations of the optical detection data. Alternatively or additionally, displacement detector 108 can comprise a sound detector such as a microphone or a plurality of microphones. For instance, the sound detector can be employed as a motion sensor and/or a rotation sensor by generating audio data over time and evaluating variations of the audio data. Displacement detector 108 can be configured to provide the displacement data over time in subsequent periods. Displacement detector 108 is mechanically coupled with housing unit 111 such that it remains in a fixed position relative to at least part of housing unit 111 upon a rotational and/or translational displacement of this part. Thus, displacement data provided by displacement detector 108 is indicative of a rotational displacement and/or a translational displacement of housing unit 111.
FIG. 1 further illustrates a reference point 202 at a position remote from housing unit 111. To illustrate, reference point may be defined by spatial coordinates in an abstract coordinate system constituting a reference frame 200. In particular, reference frame 200 may be expressed in rectangular (Cartesian) coordinates comprising an x-axis, a y-axis, and a z-axis. Reference point 202 can then be defined by a fixed position with respect to reference frame 200. For instance, during a movement of reference point 202 at a constant and/or accelerated speed relative to another reference frame, reference point 202 remains in its fixed position in reference frame 200 such that reference frame 200 moves at the same speed with respect to the other reference frame. In some examples, reference frame 200 may be provided as the earth's reference frame such that reference point 202 remains at a fixed position with respect to the earth's surface. In some examples, reference frame 200 may be provided as a reference frame of a signal source, in particular a sound source such as a conversation partner or a loudspeaker, moving at least temporarily relative to the earth's reference frame such that reference point 202 moves accordingly with respect to the earth's surface, in particular with regard to at least one of the x-axis, the y-axis, and the z-axis. In some examples, reference frame 200 may be provided as a reference frame of a part of hearing device 100 remote from housing unit 111, for instance a sound detector positioned at reference point 202, which can be moved independently from housing unit 111.
Processing unit 102 can be configured to access displacement data provided by displacement detector 108 and/or audio data provided by sound detector 106. In this way, processing unit 102 can be operative to collect the displacement data over time in subsequent periods. Processing unit 102 can also be operative to generate position data based on the collected displacement data. The position data can be indicative of an angular orientation and/or a spatial location of housing unit 111 with respect to reference point 202. Processing unit 102 can further be operative to obtain a reliability measure of the position data. The reliability measure can be indicative of a reliability of the position data after the subsequent periods in which the displacement data has been collected. Processing unit 102 can also be operative to adjust the position data based on the reliability measure. These and other operations that may be performed by processing unit 102 are described in more detail in the description that follows.
References to operations performed by hearing device 100 may be understood to be performed by processing unit 102 of hearing device 100. To this end, processing unit 102 may comprise a single processor or a plurality of processors performing different tasks. For instance, processing unit 102 may comprise a first processor operative to collect the displacement data and a second processor communicatively coupled to the first processor. The second processor can then be operative to generate the position data based on the displacement data collected by the first processor and/or to obtain the reliability measure and/or to adjust the position data.
In some implementations, hearing device 100 further comprises a memory. The memory may be implemented by any suitable type of storage medium and may be configured to maintain (e.g., store) data generated, accessed, or otherwise used by processing unit 102. For example, the memory may maintain data representative of a sound processing program that specifies how processing unit 102 processes audio data (e.g., audio data detected by sound detector 106) to present audio content to a user. The memory may also maintain data representative of a program encoding a method of providing position data in hearing device 100, in particular a program encoding instructions that can be executed by processing unit 102 to perform the collecting of the displacement data and/or the generating of the position data and/or the obtaining of the reliability measure and/or the adjusting of the position data. The memory may also maintain data representative of the collected displacement data and/or the generated position data and/or the obtained reliability measure and/or the adjusted position data. The memory may also maintain data representative of an algorithm that can be executed by processing unit 102 to obtain the reliability measure. The Memory may also maintain data representative of settings for a sound processing program.
In some implementations, hearing device 100 further comprises an auxiliary device. The auxiliary device may be a smartphone and/or also may comprise a displacement detector 108, which is configured to provide displacement data. Processing unit 102 may be implemented in the auxiliary device and/or the auxiliary device may comprise an additional processing unit. Subsequently described methods and/or algorithms may be executed by processing unit 102 implemented in housing unit 111 and/or by processing unit 102 implemented in the auxiliary device. A memory may be implemented in the auxiliary device and/or the auxiliary device may comprise an additional memory.
FIG. 2 illustrates exemplary implementations of hearing device 100 as a receiver-in-the-canal (RIC) hearing aid 300, in accordance with some embodiments of the present disclosure. RIC hearing aid 300 comprises a behind-the-ear (BTE) part 301 configured to be worn at an ear at a wearing position behind the ear. Hearing aid 300 further comprises an in-the-ear (ITE) part 302 configured to be worn at the ear at a wearing position at least partially inside an ear canal of the ear. Housing unit 111 is implemented by a first housing 311 of BTE part 301 and a second housing 312 of ITE part 302. First housing 311 accommodates processing unit 102, sound detector 106, and displacement detector 108. In the example, sound detector 106 is provided by a plurality of spatially arranged microphones 306, 307. Microphones 306, 307 can be included in a microphone array. Microphones 306, 307 can be configured to provide audio data to processing unit 102. The audio data can be indicative of an ambient sound. The ambient sound can include sound emitted at reference point 202. Furthermore, a battery 309 is enclosed by first housing 311. Output transducer 110 is provided as a receiver accommodated in second housing 312 of ITE part 302. BTE part 301 and ITE part 302 are interconnected by a cable 316. Receiver 110 is operationally coupled to processor 102 via cable 316 and a cable connector 315 provided at second housing 312 of BTE part 301. A wireless coupling between processor 102 and receiver 310 is also conceivable.
In some implementations, processing unit 102 is configured to generate position data by employing the audio data. The position data based on the audio data can be provided auxiliary to the position data based on the collected displacement data. In particular, processing unit 102 can be configured to determine a difference between the audio data provided by microphones 306, 307 and to generate the auxiliary position data based on the difference. The difference may comprise a difference in phase and/or a difference in signal level in the audio data provided by at least two of microphones 306, 307. To illustrate, sound emitted by a sound source located at reference point 202 can be detected by each of microphones 306, 307. A position of housing 311, in particular an angular orientation and/or a spatial location, relative to reference point 202 can then be determined from a difference in the audio data provided by microphones 306, 307. The auxiliary position data can thus be obtained independently from the position data which is based on the collected displacement data. In some implementations, the reliability measure obtained by processing unit 102 for the position data generated from the displacement data can thus be based on the auxiliary position data.
In some implementations, processing unit 102 is configured to provide an output signal to output transducer 110 based on the audio data provided by microphones 306, 307. Processing unit 102 can be configured to provide a directionality of the output signal. The directionality may be provided during processing of the audio data by amplifying a part of the audio data which corresponds to a desired direction, for instance audio data provided by some of microphones 306, 307, relative to another part of the audio data deviating from the desired direction, for instance audio data provided by other microphones 306, 307. Processing unit 102 can be configured to determine the desired direction based on the position data. In some implementations, processing unit 102 is operative to provide beamforming. The directionality of the output signal can comprise beamforming based on the audio data provided by microphones 306, 307. The processing unit can further be configured to control a property of the beamforming based on the position data. The property of the beamforming can comprise steering a directionality of the beamforming and/or adjusting a beam size, in particular a beam width, of the beamforming. To illustrate, the property of the beamforming may be adjusted depending on a momentary position of housing 311 relative to reference point 202.
In some implementations, processing unit 102 is operative to provide the directionality of the output signal in a manner to create, when stimulating the user's hearing, a hearing perception of a sound coming from the desired direction. This can be exploited, for instance, to create an augmented reality for the user by adding perceptual auditory information to the output signal corresponding to the sound from the desired direction. This can also be exploited, for instance, to provide a directionality of a streamed audio signal with respect to a streaming source. Providing directionality of a streamed audio signal per se, as described in international patent application publication No. WO 2016/116160 A1 is known in the art. To illustrate, a remote sound detector may be provided at reference point 202. The remote sound detector may be connected to a transmission unit configured to transmit audio data of the remote sound detector as a radio frequency signal to processing unit 102 which then processes the audio data in a way to provide an angular localization impression of the output signal to the user. The angular localization impression can correspond to an estimated azimuthal angular location of the transmission unit and/or the remote sound detector at reference point 202.
FIG. 3 illustrates a hearing device 400 in accordance with some embodiments of the present disclosure. Hearing device 400 is a binaural hearing device comprising a first hearing device unit 401 configured to be worn at a first ear of the user and a second hearing device unit 403 configured to be worn at a second ear of the user. First hearing device unit 401 comprises components 102, 106, 108, 110, as described above, included in housing unit 111 forming a first housing unit configured to be worn at the first ear. Second hearing device unit 403 comprises corresponding components including a second processing unit 402 communicatively coupled to a second sound detector 406, a second displacement detector 408, and a second output transducer 110. Components 402, 406, 408, 410 are included in a second housing unit 411 configured to be worn at the second ear. First processing unit 102 and second processing unit 402 are communicatively coupled via a binaural link 410. In this way, audio data provided by first sound detector 106 and second sound detector 406 and/or displacement data provided by first displacement detector 108 and second displacement detector 408 can be shared between processing units 102, 402. The position data based on the collected displacement data and/or the reliability measure of the position data and/or the adjusted position data may be provided by both or by one of processing units 102, 402.
Second displacement data provided by second displacement detector 408 can be used in conjunction with the first displacement data provided by first displacement detector 108 to improve the reliability of the generated position data. Second audio data provided by second sound detector 406 can be used in conjunction with the first audio data provided by first sound detector 106 to provide auxiliary position data, as described above. To illustrate, first sound detector 106 and second sound detector 406 may each comprise at least one microphone which are spatially arranged with respect to each other. The auxiliary position data can then be generated based on a difference determined between the audio data provided by the microphones. Audio data provided by sound detectors 106, 406 can also be processed to provide a directionality of the output signal, as described above. In this way, binaural beamforming and/or a hearing perception of a sound coming from a desired direction may be implemented in an analogous way.
FIG. 4 illustrates a hearing device 500 in accordance with some embodiments of the present disclosure. Hearing device 500 further comprises a remote sound detector 506 provided at reference point 202. Remote sound detector 506 is communicatively coupled to a signal transmitter unit 507 configured to transmit audio data from remote sound detector 506 by radio waves, in particular as a radio frequency signal, to a signal receiver unit 508 communicatively coupled to processing unit 102. Signal receiver unit 508 is implemented in housing unit 111. Thus, the remote audio data provided by remote sound detector 506 at reference point 202 can be employed by processing unit 102 in conjunction with the audio data provided by sound detector 106 at the position of housing unit 111. Auxiliary position data can thus be generated based on a difference determined between the audio data and the remote audio data, as described above and below. The audio data and remote audio data can also be processed to provide a directionality of the output signal, as described above, in particular as further disclosed in international patent application publication No. WO 2016/116160 A1 . In this way, beamforming and/or a hearing perception of a sound coming from a desired direction may be implemented. To illustrate, remote sound detector 506 may be provided as a microphone carried by a conversation partner such that reference point 202 moves with the movements of the conversation partner, in particular relative to housing 111. The directionality of the output signal can then be provided such that it points from a momentary position of housing 111 in the direction of the conversation partner at reference point 202.
In some implementations, hearing device 500 comprises first unit 401 and second unit 403 of hearing device 400 illustrated in FIG. 3, wherein signal receiver unit 508 is implemented in each unit 401, 402. In this way, signal receiver units 508 can be spatially distributed at the different wearing positions of housings 111, 411 of both units 401, 402. Radio waves received by signal receiver units 508 at the different spatial positions can thus be evaluated with respect to a difference, for instance a phase difference and/or a difference in signal level, in particular by a received signal strength indicator (RSSI). This can be exploited to generate auxiliary position data, as further described below.
FIG. 5 illustrates a method of operating a hearing device for providing position data according to some embodiments of the present disclosure. As shown, displacement data is provided in operation 602. The displacement data is indicative of a rotational displacement and/or a translational displacement of housing 111, 311, 312, 411. In operation 603, the displacement data is collected. Operations 602 and 603 can be continuously repeated such that the displacement data is collected in subsequent periods over time. In operation 604, the position data is generated based on the collected displacement data. The position data is indicative of an angular orientation and/or a spatial location of housing 111, 311, 312, 411 with respect to reference point 202. In some implementations, collecting the displacement data in operation 603 and/or generating the position data in operation 604 can comprise integrating the displacement data and/or data acquired from the displacement data over said subsequent periods. For instance, the position data may be calculated as a sum of displacements collected over a time in operations 602, 603, which time corresponds to a sum of periods in which operations 602, 603 have been subsequently performed.
In operation 605, a reliability measure of the position data is obtained. The reliability measure is indicative of a reliability of the position data after said subsequent periods. In some implementations, as indicated in FIG. 5 by a solid arrow leading to operation 605, operation 605 can be performed independently from operations 602, 603, 604. In particular, the reliability measure can be obtained independently from the position data obtained in operation 605. In some implementations, as indicated in FIG. 5 by a dashed arrow leading to operation 605, the reliability measure can be obtained based on the position data obtained in operation 605. In some implementations, the reliability measure can be obtained based on a combination of the position data obtained in operation 605, as indicated by the dashed arrow, and other data independent from the position data, as indicated by the solid arrow leading to operation 605. In some implementations, operations 602, 603, 604, 605 can be integrated in a Kalman filter. In particular, operations 602, 603, 604 can be implemented as a prediction step of the Kalman filter. A measurement update step of the Kalman filter can include operation 605. In some implementations, obtaining the reliability measure in operation 605 can also be based on an algorithm, such as a machine learning algorithm, as further described below. In a subsequent operation 606, the position data generated in operation 604 is adjusted based on the reliability measure obtained in operation 605. The adjustment can be based on both the position data generated in operation 604 and the reliability measure obtained in operation 605, for instance based on a comparison and/or correlation between the position data and the reliability measure. Alternatively, the adjustment can be solely based on the reliability measure obtained in operation 605, for instance by replacing the generated position data with new position data deduced from the reliability measure. After adjusting the position data, the method may be continuously repeated, wherein the position data generated in operation 604 is based on the position data that has been previously adjusted in operation 606.
A number of subsequent periods in which the displacement data is collected in operation 603 can be continuously increased with time. Adjusting the position data in operation 606 can be performed when a number of said subsequent periods is adequate, for instance when the period number exceeds a minimum duration. For instance, the minimum duration can be set such that it corresponds to at least two of said subsequent periods. The minimum duration can also be set such that it corresponds to an arbitrary time, in particular an arbitrary number of said subsequent periods, which is required for obtaining the reliability measure in operation 605. Thus, obtaining the reliability measure in operation 605 on which the adjusting of the position data is based in operation 606 can be performed less frequent than generating the position data in operation 604. This can be exploited for generating the position data in operation 604 rather fast and to adjust the position data, e.g. for inaccuracies and/or errors occurring in operation 604, in operation 606, which may be slower, in particular due to the time required for obtaining the reliability measure in operation 605, at a later stage. In this way, the position data may be provided at a rather high frequency and with a satisfactory accuracy. According to the invention, the position data is continuously generated at a first frequency in operation 604 and the reliability measure is continuously obtained at a second frequency in operation 606, wherein the second frequency is smaller than the first frequency.
FIG. 6 illustrates some embodiments of the method for providing position data in which the adjusting of the position data in operation 606 can be implemented at a smaller frequency than the obtaining of the reliability measure in operation 605. Before adjusting the position data in operation 606, an operation 611 is performed in which it is determined whether the number of said subsequent periods is adequate to perform operation 606. For instance, the number of said subsequent periods can be determined to be adequate when the period number exceeds a minimum duration and/or when the reliability measure has been found to be obtained in operation 605. When the period number has been determined to be adequate, operation 606 of adjusting the position data can be performed. In the contrary case, operations 602, 603, 604 can be repeated until the period number is determined as adequate.
FIG. 7 illustrates some further embodiments of the method for providing position data. Before generating the position data in operation 604, an operation 612 is performed in which it is determined whether the number of said subsequent periods is adequate to perform operation 604. In the contrary case, operations 602, 603 can be repeated until the period number is determined as adequate. Thus, generating the position data in operation 604 can be delayed until the displacement data has been collected for an adequate number of periods. In this way, the reliability of the generated position data may be further enhanced. The adequate period number in operation 612 can be selected as a first period number smaller than a second period number that is implemented as the adequate period number in operation 611. Thus, the position data can be generated at a frequency in operation 604 that is larger than a frequency in which the reliability measure is obtained in operation 605.
FIG. 8 illustrates a method of obtaining a reliability measure of the position data according to some embodiments of the present disclosure. The method may be implemented in the place of operation 605 according to any of the previously described methods. In operation 621, an ambient sound originating from the environment of the user is detected and corresponding audio data is provided. The ambient sound can include sound emitted at the reference point such that the audio data is indicative of the sound emitted at the reference point. In some implementations, as illustrated in operation 622, the sound is detected at the housing of the hearing device, for instance by sound detector 106, 306, 307, 406 implemented in housing 111, 311, 312 411. In operation 623, auxiliary position data is generated based on the audio data. The auxiliary position data can thus be indicative of an angular orientation and/or a spatial location of the housing with respect to reference point 202. The auxiliary position data can thus be provided independently from the position data generated in operation 604 based on the displacement data collected in operation 603. The reliability measure of the position data obtained in operation 605 can then comprise or consist of the auxiliary position data. In particular, spatially arranged microphones 306, 307 and/or spatially arranged sound detectors 106, 406 can be employed to provide respective audio data indicative of the detected sound at each spatial position. The auxiliary position data can then be generated in operation 623 based on a difference determined between the audio data, as described above, for instance a difference in phase and/or a difference in signal level in the audio data. In particular, the audio data between which the difference is determined can be selected in operation 623 such that the difference is indicative of a propagation direction of at least part of said ambient sound which is emitted at the reference point.
In some implementations, as illustrated in operation 624, the ambient sound is detected at the reference point, for instance by remote sound detector 506 provided at reference point 202. The auxiliary position data generated in operation 623 can then be based on the audio data detected at the reference point in operation 624. For instance, the audio data detected at the reference point can be transmitted as a radio frequency signal to the housing of the hearing device, in particular such that the audio data can be received at two spatially separated points at the housing and/or at two housings of a binaural hearing device configured to be worn at different ears of the user. The auxiliary position data generated in operation 623 can then be based on a difference of the radio waves received at the differing spatial points, for instance a phase difference and/or a difference in signal level, in particular by RSSI measurements. Such an operational principle is known per se, as disclosed in international patent application publication No. WO 2016/116160 A1 . In some implementations, operation 622 and operation 624 as described above can be combined in order to generate the auxiliary position data in operation 623. Thus, the reliability of the auxiliary position data may be further enhanced. In some implementations, a method as further described below in conjunction with FIG. 10 can be correspondingly applied to generate the auxiliary position data.
FIG. 9 illustrates a method of determining a relative position with respect to a sound source, in particular to detect a presence of such a sound source in an environment of the user. The method starts with operation 621 of detecting ambient sound, as described above. In particular, the ambient sound may be detected at the housing according to operation 622 and/or at the reference point according to operation 624 and/or anywhere else in an environment close or far away from the user. In operation 632, a presence of a sound emitted from a sound source is determined in the detected ambient sound. The sound source may be localized at any position in the environment of the user and/or moving and/or accelerating in the environment. For instance, the sound source can be a conversation partner of the user, a loudspeaker and/or the like. Determining the presence of the sound source can comprise determining a directionality of the ambient sound or at least of a component of the ambient sound. In particular, audio data indicative of the ambient sound can be evaluated with respect to a directionality of at least part of the ambient sound. The directionality can be defined as a direction in which a part of the ambient sound, in particular a predominant part, propagates to a sound detector for detecting the ambient sound. For instance, a plurality of spatially arranged microphones may be employed as a sound detector, as described above. The directionality of the ambient sound can then be determined based on a difference determined between the audio data of at least two of the spatially arranged microphones, such as a difference in phase and/or a difference in sound level. The directional part of the ambient sound can then be associated with a sound source in the environment. A remaining part of the ambient sound may be regarded as background noise and/or other environmental sound.
After determining the presence of a sound source, position data of the housing with respect to the sound source is determined in operation 633. The position data can thus be indicative of an angular orientation and/or a spatial location of the housing with respect to a position of said sound source. Thus, reference point 202 can be associated with the position of the sound source. The determining of the position data with respect to the sound source in operation 633 can be performed analogously to the determining of the auxiliary position data as described above, for instance as described in conjunction with operations 622, 623, 624. In this way, a sound source can be localized in the environment at the reference point by associating the reference point with the position of the sound source. Operations 632, 633 can also be performed to provide the position data of the housing with respect to the sound source as the auxiliary position data, for instance in the place of operation 623 described above. Operations 621, 632, 633 can also be performed to obtain the reliability measure, for instance in the place of operation 605 described above. The method comprising operations 621, 632, 633 can also be performed to provide initial position data for any of the methods described in conjunction with FIGS. 5 - 7. In particular, the position data with respect to the sound source determined in operation 633 can be employed as an original position data based on which further position data with evolving time can be generated in operation 604. In this way, an original position relative to the reference point may be defined as the position relative to the sound source and the position changes evolving with time can then be determined based on the collected displacement data.
FIG. 10 illustrates a method of determining a relative position with respect to a radio source, in particular to detect a presence of the radio source in an environment of the user. In operation 635, radio waves are received. The radio waves may be emitted from a remote source provided at reference point 202 and detected at the housing of the hearing device, for instance by a signal receiver such as signal receiver unit 508. In operation 636, spatial characteristics of the received radio waves are determined. To this end, the signal receiver may comprise a plurality of spatially arranged signal receiver units 508. For instance, a respective signal receiver unit 508 may be implemented in each housing 111, 411 of binaural hearing device 400. The position data determined in operation 633 can then be based on a difference of the radio waves received at the differing spatial positions, for instance a phase difference and/or a difference in signal level, in particular by RSSI measurements. The position data can thus be indicative of an angular orientation and/or a spatial location of the housing with respect to a position of the radio source. In particular, the position data determined in operation 633 can be provided as the auxiliary position data generated in operation 623.
Operations 635, 636 can also be performed in conjunction with operations 632, 633 such that the position data determined in operation 633 can be based on both the detected ambient sound and the received radio waves. For instance, a sound detected by remote sound detector 506 may be transmitted as an audio signal encoded in radio waves between transmitter 507 and receiver unit 508. In addition, the sound may be detected as ambient sound by sound detector 106. Thus, operations 635, 636 can be performed based on the radio waves received by receiver unit 508, and operations 632, 633 can be performed based on the ambient sound detected by sound detector 106, which may be employed in operation 633 to determine the position data with an enhanced reliability.
FIG. 11 illustrates another method of obtaining a reliability measure of the position data according to some embodiments of the present disclosure. The method may be implemented in the place of operation 605 according to any of the previously described methods. The method may also be implemented in operation 605 in conjunction with any of the methods illustrated in FIGS. 8 - 10 in order to obtain the reliability measure. In operation 641, an algorithm is applied to the position data and/or the displacement data. In operation 642, a correction value is determined from the applied algorithm. The correction value can be indicative of a value and/or an amount for correcting the generated position data. Thus, the adjusting the position data in operation 606 can be based on the correction value. In some implementations, the algorithm can be based on a model of an expected movement behavior of the user and/or a model describing an expected position of a sound source with respect to the housing and/or a model describing an expected deviation of the generated position data from a desired value of the position data. For instance, the algorithm can include a functional algorithm and/or a numerical algorithm and/or a statistical algorithm and/or an optimization algorithm and/or a filter algorithm and/or a classification algorithm and/or a machine learning algorithm, in particular a machine learning algorithm for training a classifier of the displacement data and/or position data.
It also is possible that a user can interfere in the method as described with respect to FIGS. 5 to 11. For example, the hearing device may comprise a user interface by which the user can select a suitable procedure of obtaining the reliability measure in operation 605. In particular, the user may select between a procedure in which auxiliary position data is generated, in particular according to operation 623, and/or one or more different algorithms applied in operation 641. For example, the user may select a movement scenario indicative of a specific movement situation of the user. A procedure of obtaining the reliability measure in operation 605 can then be selected depending on the movement scenario. To illustrate, the movement scenario may be indicative of the user facing another person in the specific situation and/or of a plurality of persons surrounding the user and/or the user being involved in a movement activity such as walking or sitting. The movement scenario may be encoded with a number from a set of numbers. The movement scenario may also be automatically determined by the hearing device, for instance with a machine learning algorithm, as further described below. An example for a movement scenario may be the user sitting in a car next to a conversation partner. In a car, the user may not have the chance to look in the direction of the conversation partner, which means that the interpretation of the collected displacement data and/or the generated position data has to be different compared to the movement scenario involving a conversation partner facing the user. As further examples, a situation in which the user is sitting on a table with one or more conversation partners and/or with an audience during a public speech and/or when watching a movie may be different movement scenarios. Another movement scenario may be a situation in which an audio signal is streamed from a remote source, for instance from a remote sound detector at the reference point.
It is further conceivable that a suitable procedure of obtaining the reliability measure in operation 605, in particular suitable auxiliary position data generated in operation 623 and/or a suitable algorithm applied in operation 641, is selected based on a speaking activity of the user. For instance, own voice detection may be applied to determine the speaking activity of the user. During the speaking activity, it may be assumed that the user faces a conversation partner and/or another reference point to which he addresses his speech. The selected algorithm may thus take into account a position of the reference point facing the user's mouth.
In some implementations, the algorithm can be based on a recursive state estimator and/or Bayes filter, such as the Kalman filter, where the reliability measure can be represented as multivariate Gaussian distributions. The prediction step can be computed as projection of the current state estimate (including the current position data), given by the predicted mean X̂_t+1 = A * Xt at the discrete time t and error covariance Ê_t+1 = A * Et * A^T + R. The matrix A describes how the current state at time t affects the future state at time t+1 and the covariance matrix R describes the uncertainty introduced by such state transition. The measurement update step can be computed as correction of the state estimate, given by the updated mean Xt+i = Xt+i + K * (Zt+i - H * Xt+i) and error covariance Et+i = (I - K * H) * Ê_t+1, with identity matrix I. The matrix H describes the relationship between the state and the output of the system and Z_t+1 are the measurements at time t+1, which may comprise the auxiliary position data for example. K is the so-called Kalman gain, K = Ê_{t | 1} * H^T * (H * Ê_{t | 1} * H^T + Q)^-1, where the covariance matrix Q describes the uncertainty of the measurements. Alternatively, also a non-linear version of a Kalman filter, such as the Extended Kalman filter (EKF) or the Unscented Kalman filter (UKF), which apply linearization about the current state estimate of mean and error covariance, may be used.
As a specific example, the user may select a movement scenario corresponding to a conversation with a conversation partner. Such a movement scenario may also be automatically determined, for instance by a machine learning algorithm and/or by detecting speech of the user with the conversation partner. The reference point then corresponds to a position of the conversation partner. The user's position data with respect to the conversation partner may be initially determined by the method illustrated in FIG. 9 and/or in FIG. 10. Subsequently, the method illustrated in FIGS. 4 to 7 may be performed to continuously provide updated position data in operation 606. The procedure of obtaining the reliability measure in operation 605 may then be selected based on the movement scenario. For instance, auxiliary position data can be provided in operation 623 based on detected speech of the conversation partner and/or a suitable algorithm may be applied in operation 641. Some embodiments of algorithms which may be applied in operation 641 are subsequently described.
For instance, the algorithm can comprise determining an intermediate value between a value extracted from the position data and a predefined value. The predefined value can be indicative of a predefined angular orientation and/or a predefined spatial location of the housing with respect to the reference point. For instance, the predefined value may be selected as a fixed position value relative to the reference point. The correction value can then be set in operation 642 to the intermediate value such that it approaches the predefined value, for instance after a certain duration and/or a certain number of iterations. In this way, variations of the position data occurring only at a smaller timescale such that they at least partially cancel out at a larger timescale can be attributed with a lower significance for the position data. A corresponding algorithm may be applied as a low-pass filter in order to filter out fast changes of the position data.
To illustrate, the algorithm applied in operation 641 may yield a correction value CP of a value P of the position data generated in operation 604 by calculating a product of P with a correction factor C. Such an algorithm may be defined by the equation CP = C * P. The correction factor C can be selected as a value larger than zero and smaller than one, for instance as a constant factor. In this way, the predefined value selected as the predefined angular orientation and/or the predefined spatial location relative to the reference point is provided as zero. Such an algorithm may implement a user movement behavior model which is based on the assumption that the user, on the long term, generally aligns his head with respect to the reference point. For instance, when the reference point corresponds to the position of a conversation partner, the user may tend to align his head such that, at least on the average over a longer time scale, he faces the conversation partner. The respective housing position with respect to the conversation partner during this head alignment of the user can then correspond to said predefined value in the user movement behavior model. Repeated application of the algorithm can iteratively drive the generated position data, and thus the position estimate, to the predefined value, in particular after a large enough number of iteration steps in which the algorithm is repeated. In this way, rather fast position changes of the housing, such as changes caused by fast rotational head movements of the user, can be compensated at first by the correction factor C, but after a while the position data, such as for instance an angle estimate of the housing relative to the reference point, is forced back to the predefined value. The iteration may be carried out at least once every time in which operation 604 is performed in order to obtain the reliability measure as the correction value CR. This may correspond to a repeated application of the algorithm after a certain time interval, for instance a time interval corresponding to a sum of said subsequent periods. As a specific example, the correction factor C may be selected to be 0.9 such that the correction value CP of position data value P would be given as CP = 0.9 * P. The algorithm may be iteratively applied once every 0.5 seconds such that short-term user movements and/or other position variations occurring at a smaller timescale would be compensated by the correction factor to be driven against the predefined value zero.
As another example, the algorithm applied in operation 641 may yield a correction value CP of a value P of the position data generated in operation 604 by calculating a product of P with a correction factor C, and then adding an offset value O to the value of CP. Thus, the algorithm may be defined by the equation CP = C * P and afterwards adding an offset value O to the value of CP. Operation 605 may be continuously repeated, wherein the algorithm in operation 641 is applied each time to the position data P generated in operation 604. Thus, the value of CP may converge to zero after a large enough number of repetitions of operation 641 in a case in which the position data P generated in operation 604 does not change, as described above, which may indicate that the user wearing the housing does not move. After adding the offset O to CP, the generated position data thus may converge to the offset O. In this way, the predefined value selected as the predefined angular orientation and/or the predefined spatial location relative to the reference point can be provided by the value of offset O. The correction factor C can again be selected as a value larger than zero and smaller than one, and the offset O can be selected as an arbitrary position value.
Such an algorithm may implement a user movement behavior model which is based on the assumption that the user, on the long term, generally has a preferred listening direction with respect to the reference point. For instance, when the reference point corresponds to the position of a conversation partner sitting next to user, for instance in a car, or to the position of a person talking to many people, for instance during a lecture or speech, the user may tend to align his head to a direction pointing away from the reference point by a rather fixed value, at least on the average over a longer time scale, such that he faces another position of interest, for instance a front window in the car or a projection screen during the lecture. The respective housing position with respect to the conversation partner during this head alignment of the user can then correspond to said predefined value in the user movement behavior model. Thus, the offset O may be provided such that it corresponds to a preferred listening direction of the user relative to the reference point. Applications requiring the relative housing position with respect to the user's preferred listening direction, such as for instance beamforming, can thus rely on the position data adjusted by the correction value CP, in particular the value of CP and the added offset O, in operation 606. For instance, the beamforming direction can be iteratively readjusted with respect to the preferred direction by a correction value determined by the correction factor C such that, after a while, the beamforming is driven back to the preferred direction.
The correction factor C and/or the offset O can be selected as a constant value and/or as a value changing over time, in particular as a regularly updated value. For instance, the correction factor C and/or the offset value O may be automatically provided by a setting and/or a program of the hearing device and/or a connected remote device, such as a portable device like a smartphone, and/or manually adjustable by the user by selecting the correction factor C and/or the offset O through a user interface of the hearing device and/or by means of a connected remote device, for example by an App on a smartphone.
In some implementations, the algorithm applied in operation 641 can be restricted to be applied depending on a criterion which the generated position data and/or the collected displacement data must meet. In particular, the criterion can comprise a threshold for the position data and/or the displacement data. The algorithm can thus comprise determining a variation of the position data and/or the displacement data over said subsequent periods, as illustrated in operation 643 in FIG. 11, and to evaluate the variation of the position data and/or the displacement data relative to the threshold, as illustrated in operation 644 in FIG. 11. In some implementations, the threshold comprises a minimum duration. The criterion can then comprise a time, in particular a number of said subsequent periods, in which said variation has been determined, wherein the time exceeds the minimum duration. In this way, movements occurring only at a rather short time scale smaller than the minimum duration may be compensated by the algorithm and/or disregarded in the position data, for instance by choosing previously generated position data as the correction value. Movements occurring at a time scale larger than the minimum duration may be accounted for by a correction value representing substantially unaltered position data with respect to the generated position data and/or by a different procedure implemented in the algorithm, for instance by an algorithm providing said predefined value as the correction value. In some implementations, the threshold comprises a minimum limit. The criterion can then comprise variations determined in the position data and/or the displacement data, in particular with respect to previously generated position data and/or displacement data and/or with respect to a fixed value, exceeding the minimum limit. In this way, movements causing only rather small position changes and/or displacements of the housing may be compensated by the algorithm and/or disregarded in the position data, for instance by choosing previously generated position data as the correction value. Movements exceeding the minimum limit of position changes and/or displacements may be accounted for by a correction value representing substantially unaltered position data with respect to the generated position data and/or by a different procedure implemented in the algorithm, for instance by an algorithm providing said predefined value as the correction value. In some implementations, the threshold comprises a minimum duration and a minimum limit, as described above.
To illustrate, the algorithm may implement a user movement behavior model which is based on the assumption that the user, when the user is turning his head at an angle exceeding a threshold angle, is focusing on a new reference point. For instance, when a first reference point corresponds to a position of a first conversation partner and a second reference point corresponds to a position of a second conversation partner, turning the user's head over an angle larger than the threshold angle can indicate that the user changes his listening attention from the first conversation partner to the second conversation partner by aligning his position from the first conversation partner to the second conversation partner. As a specific example, head movements exceeding a threshold of +/- 20° may serve as an indication of an alignment to another reference point. In such a situation, when the variation of position data and/or displacement data exceeds the threshold, it could be undesirable that the position data is corrected by the algorithm in the above described way, in particular when the algorithm is based on assumptions of a user movement behavior model regarding the first reference point. For instance, it could be undesirable to iteratively drive the generated position data to the predefined value in the above described way since it may require a certain delay for readjusting to the second reference point. Such a delay, for instance when beamforming is applied based on the generated position data, could be perceived as too long by the user. In some implementations, the algorithm may comprise a procedure setting the correction value to the predefined value, when the variation of the position data and/or displacement data is determined to exceed the threshold. In this way, the delay for readjusting to the second reference point may be reduced and/or avoided. For instance, a procedure of the algorithm including correction factor C and/or offset O, as described above, could be applied only to a limited range of variations in position and/or displacements, e.g. for head movements defined as being below said threshold, and if the variations in position and/or displacements are even larger, e.g. when the user turns his head even more, another procedure of the algorithm may provide a different correction value, for instance a correction value which is set back to said predefined value immediately.
FIG. 12 illustrates a method of determining a significance level of the collected displacement data and/or the generated position data for said angular orientation and/or spatial location of the housing relative to the reference point. The method may be implemented as an algorithm applied to the generated position data and/or the collected displacement data, in particular in the place of operation 641 illustrated in FIG. 11. The algorithm may comprise a predictive model provided by a machine learning algorithm, as further illustrated below. In operation 651, patterns of significance of displacement data and/or position data are provided. Each pattern of significance may comprise a sequence of displacement data and/or position data as a function of time. The sequence can be associated with a significance level indicating a probability for the displacement data and/or position data being significant for an angular orientation and/or spatial location of the housing with respect to the reference point. In this way, various displacement data and/or position data may be classified by the patterns of significance with respect to a significance level corresponding to the respective probability. As described below, these probabilities may be determined with a machine learning algorithm.
In operation 652, the displacement data collected in operation 603 and/or the position data generated in operation 604 is classified based on the patterns of significance provided in operation 651 with respect to the significance level. To this end, the algorithm applied in operation 641 can comprise a classifier that maps the collected displacement data and/or the generated position data to a significance level associated with at least one of said patterns of significance having a required degree of similarity with the collected displacement data and/or the generated position data. The degree of similarity may be evaluated by an uncertainty measure associated with a respective pattern of significance. For instance, the significance level associated with the respective pattern of significance can also be indicative for the uncertainty measure. In this way, a significance level of the collected displacement data and/or the generated position data can be determined in operation 653. The attributed significance level can thus be indicative of a probability that the collected displacement data and/or the generated position data is significant for the angular orientation and/or spatial location of the housing with respect to the reference point. The significance level can then be used in operation 642 to determine the correction value for adjusting the position data depending on the significance level.
To illustrate, the displacement data collected in operation 603 and/or the position data generated in operation 604 may be classified in operation 652 as having a rather small significance level. This may indicate a small probability that the collected displacement data and/or the generated position data is significant for the angular orientation and/or spatial location of the housing with respect to the reference point. In particular, the collected displacement data and/or the generated position data may be evaluated with respect to a sequence of displacement data and/or position data as a function of time. The sequence can be associated with a respective significance level. The significance level can comprise an uncertainty measure that the collected displacement data and/or the generated position data can be assigned to the sequence. Such a temporal sequence can be indicative, for instance, of a trajectory of the housing caused by a user movement. The significance level can further comprise a probability that the collected displacement data and/or the generated position data assigned to the sequence can be classified as being significant for the angular orientation and/or spatial location of the housing with respect to the reference point.
For instance, the user may exert quite irregular and rather fast rotational and translational movements of his head and body during daily usage of a hearing device being unrelated to a specific reference point, which therefore may have a low significance level. Those user movements may comprise a number of individual short-term movement actions, such as shaking the head and/or temporarily turning the body, which can at least partially cancel out or balance each other on a more long-term average. The corresponding significance level of the trajectory caused by the irregular user movement can thus be assigned to a low probability that the collected displacement data and/or the generated position data is significant for the angular orientation and/or spatial location of the housing with respect to the reference point. The collected displacement data and/or the generated position data may thus be classified as being not significant for the angular orientation and/or spatial location of the housing with respect to the reference point. The correction value can then be determined in operation 642 such that the generated position data is disregarded and/or accordingly corrected in operation 606 of adjusting the position data. For example, the correction value can then be determined such that the adjusted position data corresponds to previously generated position data for which a larger significance level has been determined before in operation 653. As another example, the correction value can be determined as an intermediate value between a value extracted from the position data and a predefined value and/or as a predefined value, as described above in conjunction with FIG. 11. As a further example, the correction value can then be determined such that the adjusted position data corresponds to a fixed value that is attributed to the pattern of significance of the determined significance level.
In an opposite situation, the displacement data collected in operation 603 and/or the position data generated in operation 604 may be classified in operation 652 as having a rather large significance level corresponding to a large probability of the collected displacement data and/or the generated position data being significant for the angular orientation and/or spatial location of the housing with respect to the reference point. The correction value can then be determined in operation 642 such that the generated position data is applied substantially unchanged in operation 606 as the adjusted position data.
FIG. 13 illustrates a method of determining patterns of significance of displacement data and/or position data. The method may be implemented in the method illustrated in FIG. 12 in order to provide the pattern of significance in operation 651. In operation 661, displacement data and/or position data is provided. In some implementations, a record of said collected displacement data and/or generated position data is maintained over time in operation 661. For instance, operation 661 can comprise building a history record over time the user is wearing the hearing device. The record may be maintained by a processing unit and/or stored in and/or accessed from a memory of the hearing device. The record can comprise a sequence of displacement data and/or position data as a function of time, in particular a sequence indicative of a trajectory.
In operation 662, patterns of significance are determined from the displacement data and/or position data provided in operation 661, in particular from the recorded displacement data and/or position data. For instance, a temporal sequence of displacement data and/or position data can be extracted from the recorded displacement data and/or position data as a pattern of significance. Operation 662 can comprise classifying the recorded displacement data and/or position data, for instance based on previously determined patterns of significance, as described above in conjunction with operation 652, and/or previously recorded displacement data and/or position data. Operation 662 can also comprise determining a significance level of the recorded displacement data and/or the generated position data, in particular as described above in conjunction with operation 653. Operation 662 can also comprise evaluating the recorded displacement data and/or position data with respect to at least one feature reoccurring in said collected displacement data and/or generated position data over time. In this way, a similarity can be determined occurring in the recorded displacement data and/or position data over time. The similarity can be determined based on a similarity measure. For instance, a distance measure between the displacement data and/or position data recorded at different times, such as relative distances of different data points in a Minkowski metric, and/or conceptual clustering may be employed as a similarity measure. The similarity can also be determined based on a likelihood that the collected displacement data and/or generated position data of a certain time corresponds to the collected displacement data and/or generated position data of a previous time, for instance based on a model of the user's movements, as further illustrated below. In this way, the pattern of significance can be based on the similarity measure and/or likelihood.
To illustrate, it may be expected that the reference point frequently changes during usage of the hearing device over the total recording time such that a feature significant for an angular orientation and/or a spatial location of the housing with respect to a specific reference point would be assumed to change accordingly, at least when different time ranges of the recording time are compared. The recorded displacement data and/or position data may comprise a feature frequently reoccurring over the total recording time. Such a feature may thus be assumed not to be particularly significant for an angular orientation and/or a spatial location of the housing with respect to a specific reference point and/or of a changing orientation and/or location with respect to the reference point. In consequence, a respective pattern of significance based on this feature may be attributed with a rather low significance level. The recorded displacement data and/or position data may also comprise a feature only reoccurring over a limited time span of the total recording time. Such a feature may be assumed to be rather significant for an angular orientation and/or a spatial location of the housing with respect to a specific reference point and/or of a changing orientation and/or location with respect to the reference point. In consequence, a respective pattern of significance based on this feature may be attributed with a rather high significance level.
As a specific example, rather small variations of the displacement data and/or position data and/or variations following a certain movement pattern, such as movements related to certain gestures such as nodding or shaking the head or chewing, may frequently occur in the recorded displacement data and/or position data over the total recording time such that a respective pattern of significance may be attributed with a rather low significance level. Rather large variations and/or abrupt variations followed by smaller variations may indicate the user focusing a new reference point such that a respective pattern of significance may be attributed with a rather high significance level. In this way, the patterns of significance may be determined based on a repeating occurrence, in particular a frequency and/or a time range of occurrence, of at least one feature being common in the recorded displacement data and/or position data over time.
In some implementations, as illustrated in FIG. 13 by operations 664 and 665, the patterns of significance can also be based on a correlation between auxiliary position data and the displacement data and/or position data provided in operation 661. In operation 664, the auxiliary position data is provided. For instance, the auxiliary position data can be provided in the above described way by detecting ambient sound and generating the auxiliary position data from the ambient sound corresponding to operations 621 and 623 described in conjunction with FIG. 8 and/or corresponding to operations 621, 632, and 633 described in conjunction with FIG. 9 and/or corresponding to operations 635, 636, and 633 described in conjunction with FIG. 10. Operation 664 can comprise maintaining a record of the auxiliary position data over time, as described above in conjunction with operation 661. Alternatively, the auxiliary position data may be only evaluated at a specific time associated with a corresponding time of the displacement data and/or position data provided in operation 661. In operation 665, a correlation between the auxiliary position data and the recorded displacement data and/or position data is determined. In operation 662, at least one pattern of significance is determined based on the correlation.
To illustrate, when a rather high correlation of the auxiliary position data with the recorded displacement data and/or position data has been determined in operation 665, a respective pattern of significance may be based on at least one feature of the recorded displacement data and/or position data and may be attributed with a rather high significance level. Conversely, when a rather low correlation of the auxiliary position data with the recorded displacement data and/or position data has been determined in operation 665, a respective pattern of significance may be based on at least one feature of the recorded displacement data and/or position data and may be attributed with a rather low significance level.
As a specific example, the user may have a characteristic movement behavior in situations involving a conversation partner talking to the user. This characteristic movement behavior can be different from user movements in daily situations not involving a conversation partner. The auxiliary position data can indicate those situations involving a conversation partner. For instance, auxiliary position data determined from a directionality of ambient sound, as described above in conjunction with operations 621, 632, and 633, may be employed to recognize the presence of a conversation partner and also the relative position of the conversation partner at the reference point. In such a situation, a high correlation of the auxiliary position data with the recorded displacement data and/or position data can be determined in operation 665 when at least one feature of the auxiliary position data with the recorded displacement data and/or position data are determined to be similar and/or coincide. The respective pattern of significance determined in operation 662 based on this feature can then be attributed with a rather high significance level. In the same situation, a low correlation of the auxiliary position data with the recorded displacement data and/or position data can be determined in operation 665 when at least one feature of the auxiliary position data with the recorded displacement data and/or position data are not determined to be similar and/or do not coincide. The respective pattern of significance determined in operation 662 based on this feature can then be attributed with a rather low significance level. The patterns of significance determined in such a manner can thus be employed, in particular in operations 651, 652, 653 illustrated in FIG. 12, to determine a significance level of the displacement data collected in operation 603 and/or the position data generated in operation 604 in future situations involving a conversation partner talking to the user, even without a provision of auxiliary position data corresponding to operation 664.
In some implementations, a movement scenario, as described above, can be determined in operation 662 to determine the patterns of significance. The movement scenario may be determined with a machine learning algorithm from the collected displacement data and/or the generated position data and optionally the auxiliary position data mentioned above. The patterns of significance may be determined additionally based on the movement scenario. In some implementations, also a speaking activity of the user can be determined in operation 662 to determine the patterns of significance. For instance, own voice detection may be applied to determine the speaking activity of the user. During the speaking activity, it may be assumed that the user faces a conversation partner and/or another reference point to which he addresses his speech. The respective pattern of significance determined in operation 662 can then be attributed with a rather high significance level.
The patterns of significance may be provided as any data structure suitable to assign a significance level to the position data. For instance, the patterns of significance may be provided as a matrix having at least a first column containing values of the significance level, and at least a second column containing values of the position data associated with the significance level. In some implementations, the matrix may comprise N+1 columns, the first column containing values of the significance level and the remaining N columns each containing values of the position data associated with the significance level. In particular, position data representative for a certain amount of time can be encoded in the remaining N columns indicating how the position may change over time in order to correspond to the significance level. For example, two subsequent columns in the matrix may represent position data representative of consecutive times. This way, each row of the matrix may represent a different trajectory J = {P_t1, ..., P_tN}, describing a sequence of position data P as function of time t = {ti, ..., t_N}. In this case, the matrix may serve as a trajectory database. It also may be that the value of position data in each column is representative of a variation of the position data over time. For instance, the values may be provided as a difference of two values of the position data at different times. A record of the position data generated in operation 604 over time and/or the displacement data collected in operation 603 over time may then be evaluated as being similar or not being similar to the values of position data contained in the matrix of the significance patterns in order to determine the associated significance level in operation 653. Such a similarity may be determined by defining a similarity measure, such as a distance measure, and/or a probability distribution of the position data contained in the matrix of the significance patterns, such as a Gaussian distribution, and by applying the similarity measure and/or probability distribution to the recorded position data. To illustrate, position data values in the matrix associated with a rather low significance level may encode typical movement patterns of the user that are not related to movements of targeting a reference point, in particular a sound source location, but for instance any other frequently occurring movement gestures. In contrast, position data values in the matrix associated with a rather high significance level may encode typical movements of the user that are related to movements of aligning to a reference point, for instance when envisaging a source located at the reference point.
For instance, the rows of the matrix may represent uncertain trajectories. An uncertain trajectory D̂ = {P̂_t1, ..., P̂_tN} may be provided by multivariate Gaussian distributions, where each probability P̂_t ~ N(Mt, St) is distributed normally with mean vector Mt and covariance matrix St at time t = {ti, ..., t_N}. Mean and covariance may be included in the trajectory database. The patterns of significance may rely on uncertain trajectories, which are described by mean and covariance values for example, indicative of specific movement situations of the user. A similarity may be computed as the likelihood of a trajectory J = {P_t1, ..., P_tN}, under the movement situation provided by D̂, given by a probability of J given $\hat{\hat{D}}$
$p (J | \hat{D}) = \prod_{i = 1}^{N} p (P_{t_{i}} | {\hat{P}}_{t_{i}}) .$
J may be a concrete trajectory based on a record of collected displacement data and/or generated position data, and D̂ may be an uncertain trajectory from the data base.
A machine learning algorithm can be configured to learn the specific movement situations of the user, for instance in accordance with the method illustrated in Fig. 13. The machine learning algorithm can thus provide a predictive model for a significance of the generated position data and/or the collected displacement data. In some implementations, the machine learning algorithm can learn the data for the trajectory data base, for example using a statistical method such as an Expectation Maximization (EM) algorithm. EM is an iterative method that first calculates the expected values of the likelihood over a set of trajectories given a model of the user's movements, and second finds a new updated model with the maximum expected value of the likelihood under this expectations. To illustrate, a set of trajectories Y = {J₁, ..., J_L1} can be a training set consisting of a number of L₁ trajectories J that may be obtained from a record of collected displacement data and/or generated position data, which serve as training data. The model of the user's movements B can represent behaviors and/or specific movement situations of the user, which may be stored in the trajectory data base and can be represented by a set of L₂ uncertain trajectories B = {Di, ..., D̂_L2} for example. Starting from some initial model B₀, the expected values of the likelihood p(Y, V | B) of trajectories Y can then be computed from the log likelihood, which is given for a fixed covariance S in 3 dimensions by $\begin{array}{l} \ln p (Y,V | B) = \\ \sum_{k = 1}^{L_{1}} ({N*L}_{2} * \ln \frac{1}{\sqrt{{(2 π)}^{3} * \det (S)}} - \frac{1}{2} * \sum_{i = 1}^{N} \sum_{l = 1}^{L_{2}} V_{kl} * {(P_{t_{i}}^{k} - M_{t_{i}}^{1})}^{T} S^{- 1} (P_{t_{i}}^{k} - M_{t_{i}}^{1})), \end{array}$
where V_kl are correspondence variables, i.e. binary variables in {0, 1} with $\sum_{l = 1}^{L_{2}} V_{kl} = 1$
. In some implementations, the user's movement can also be learnt through a Hidden Markov Model (HMM) and the trajectories may be classified by HMM classifiers.
The predictive model of the user's movements provided by the machine learning algorithm may be employed to predict trajectories J associated with the displacement data collected in operation 603 and/or the position data generated in operation 604. The prediction may involve an uncertainty. The uncertainty may be determined by the machine learning algorithm during obtaining the reliability measure in operation 605. In particular, the machine learning algorithm may determine the significance level of the collected displacement data and/or the generated position data such that the significance level is indicative for the uncertainty. The model of the user's movements provided by the machine learning algorithm can also be used to predict a significance of the collected displacement data and/or the generated position data with respect to the reference point. Such a significance may be provided as a probability that the collected displacement data and/or the generated position data is relevant for a change of position of the housing with respect to the reference point, or that it is irrelevant for the change of the housing position with respect to the reference point. For instance, trajectories related to short term user movements and/or repetitive movement habits of the user may be learned by the algorithm to be irrelevant for the change of the housing position with respect to the reference point. Thus, the significance level of the associated collected displacement data and/or the generated position data may be determined to be rather small. For instance, trajectories related to a rather long term alignment of the user relative to a rather stationary direction may be learned by the algorithm to be relevant for the change of the housing position with respect to the reference point. Thus, the significance level of the associated collected displacement data and/or the generated position data may be determined to be rather large.
In some implementations, the significance level values in the matrix may be determined in operation 662 based on the auxiliary position data provided in operation 664. For instance, a large value of the significance level may be included in the patterns of significance when a good agreement between the position data and the auxiliary position data has been determined in operation 665. The significance level values produced in the matrix may also contain information about the correction value. For instance, a significance level of zero may be used to encode a high significance level such that no correction value may be applied during the adjusting of the position data in operation 606. A significance level having a certain value different from zero may be used to encode a lower significance level. At the same time, those non-zero values may be employed as the correction value that is applied during the adjusting of the position data in operation 606 and/or as a correction factor for multiplying the generated position data in order to provide the correction value during the adjusting of the position data.
FIG. 14 schematically illustrates a functional design of a processing module 701 which may perform operations 651, 652, 653 of the method illustrated in FIG. 12 and/or operations 661, 662 and/or operations 664, 665 of the method illustrated in FIG. 13. Processing module 701 may be operated by processing unit 102. In particular, processing module 701 may comprise an information acquisition module 702, one or more machine learning algorithm modules 703, 704 and a decision module 705. Information acquisition module 702 may maintain a record of said collected displacement data and/or generated position data over time and/or collect the auxiliary position data and may transform it, such that it may be input into the one or more machine learning algorithm modules 703, 704. A machine learning algorithm module 703, 704 may be used for determining probabilities based on patterns of significance of the displacement data and/or position data. Instead of only one or two machine learning algorithm modules 703, 704, a larger number of machine learning algorithm modules 703, 704 connected in parallel may be used to compute the probabilities. The decision module 705 in the end determines and/or outputs the significance level of the displacement data and/or position data for the angular orientation and/or a spatial location of the housing with respect to the reference point, which can then be used in operation 642 for determining the correction value. For example, the decision module may be based on a decision tree algorithm.
The collected displacement data and/or generated position data, which may be pre-processed by the module 702, may be input into one or more different trained machine learning algorithms 703, 704, each of which determines probabilities based on said patterns of significance of the displacement data and/or position data. The significance level for the angular orientation and/or a spatial location with respect to the reference point then may be determined from said patterns of significance, as determined from the at least two machine learning algorithms 703, 704. The machine learning algorithms 703, 704 may be trained offline, i.e. before the method illustrated in FIG. 12 and/or the method illustrated in FIG. 13 is performed. The position data and/or displacement data may be recorded in real life situations in diverse scenarios. Those data and the known resulted angular orientation and/or spatial location of the housing with respect to a reference point and/or the known resulted significance level may be input into a classification algorithm to train offline the machine learning algorithm 703, 704. The machine learning algorithm 703, 704 may be a (deep) neural network, a convolutional neural network, an algorithm based on Multivariate analysis of variance (Manova), a support vector machine (SVM), a Hidden Markov Model (HMM) or any other machine learning algorithm or pattern recognition algorithm.
FIG. 15 illustrates a displacement detector 801 in accordance with some embodiments of the present disclosure. Displacement detector 801 may be implemented in the place of displacement detector 108 in any of hearing devices 100, 300, 400, 500 illustrated in FIGS. 1 to 4. Displacement detector 801 can be provided by an inertial sensor 808, such as an accelerometer, which is configured to detect rotational and/or a translational displacements with respect to one, two, or three distinct spatial directions. In the illustrated example, displacement detector 801 is configured to detect the displacements of a three dimensional coordinate frame 802, as illustrated by an x'-axis, a y'-axis, and a z'-axis, relative to the earth's reference frame. When implemented in a hearing device housing worn at a user's ear, displacement detector 801 is thus configured to provide respective displacement data indicative of a rotational displacement and/or a translational displacement of coordinate frame 802, corresponding to a reference frame of the housing. For instance, when the user rotates his head, the displacement data may indicate a rotational displacement of the housing frame 802. When the user walks, the displacement data may indicate a translational displacement of the housing frame 802.
FIG. 15 further illustrates reference point 202 defined by a fixed position in reference frame 200, as described above in conjunction with FIG. 1. A position of the hearing device housing relative to reference point 202 can be described by an angular orientation and/or a spatial location of housing frame 802 with respect to reference point 202. For instance, as illustrated in FIG. 15, an angular orientation of the housing with respect to reference point 202 may be defined by angular position data 803, such as a set of angles α, β, γ defined between each axis of housing frame 802 and a vector 804 extending between an origin of housing frame 802 and reference point 202. Correspondingly, a spatial location of the housing with respect to reference point 202 may be defined by spatial position data such as a length of vector 804. It is understood that the position of the housing with respect to reference point 202 may be parametrized in many other different ways, for instance by expressing vector 804 in Cartesian coordinates, cylindrical coordinates and/or spherical coordinates, wherein housing frame 802 may be at a fixed position relative to the housing, and/or by assigning a predefined value, e.g. zero, to the angular orientation and/or spatial position of vector 804, wherein housing frame 802 may vary relative to the housing position when the housing is displaced relative to reference point 202.
Rotational and/or translational displacements of the housing thus change the angular orientation and/or a spatial location of the housing with respect to reference point 202. Knowing the momentary position of the housing with respect to reference point 202, however, is important for many applications of the hearing device, some of which are further described below. Collecting the displacement data of displacement detector 801 in subsequent periods, as performed in operation 603, and generating position data from the collected displacement data, as performed in operation 604, can be used to determine the momentary position of the housing with respect to reference point 202. For instance, the displacement data may be integrated over time to provide the position data. The position data generated in such a manner can be prone to inaccuracies and errors, which may add up and/or increase with time. One error source may be numerical errors during collecting and/or integrating the displacement data. Another error source may be external influences, such as unrecognized displacements of reference point 202 relative to the housing and/or rapid movement activities of the user which may be undesirable to be included in the position data, for instance because they may disturb applications of the position data relying on a rather constant user movement behavior and/or only slow relative position changes of reference point 202. As an example, a directionality of a beamformer included in the hearing device may be advantageously based on position data in which rather short-term displacements and/or re-alignments of the user with respect to reference point 202 are attenuated and/or eliminated since continuous readjustments of the beamforming may result in an unpleasant hearing perception. Those error sources can be mitigated by obtaining the reliability measure in operation 605 and adjusting the position data based on the reliability measure in operation 606, as described above.
FIGS. 16A to 16D illustrate operations of a hearing device for providing position data in movement situations that can occur when a user 901 is wearing a hearing device, in accordance with some embodiments of the present disclosure. In the examples, housing units 111,411 of hearing device 400 illustrated in FIG. 3 are worn by user 901 at his ears. A housing frame 903, corresponding to housing frame 802 described above, is defined such that its origin is positioned at a center of the head of user 901. Housing frame 903 is illustrated in a simplified manner by an x'-axis and a y'-axis. Correspondingly, a reference frame 900 corresponding to reference frame 200 of reference point 202 is depicted in a simplified manner by an x-axis and a y-axis, wherein the reference point corresponds to the position of a source 902 in reference frame 900. Source 902 may be a sound source such as a conversation partner or a loudspeaker. Source 902 may also be source of a radio signal transmitted to the hearing device, for instance a streaming source, more particularly a remote sound detector configured to transmit a detected audio signal via radio waves.
FIG. 16A illustrates an initial situation in which user 901 concentrates his attention to source position 902, in particular such that he faces the source. An initial position of housings 111, 411 with respect to source position 902 corresponding to the reference point may be determined based on a signal from the source received by the hearing device, for instance according to the method described in conjunction with FIG. 9 and/or FIG. 10, and/or by an algorithm based on a model, for instance a model of an expected movement behavior of the user as described in conjunction with the method illustrated in FIG. 11, and/or by a machine learning algorithm, for instance as described in conjunction with FIGS. 12 - 14. FIG. 16B illustrates a later situation in which the source has changed its position 902 relative to the earth's reference frame. For instance, the source may be a conversation partner or a person wearing a remote sound detector walking around. User 901 continues to concentrate his attention to source 902, in particular such that he faces source 902. Thereby, user 901 turns his head by an angle α. The altered housing position relative to the earth's reference frame is illustrated in FIG. 16B by a rotated housing frame 904 comprising an x"-axis and a y"-axis rotated with respect to initial housing frame 902 by the angle α. The angular orientation of housings 111, 411 with respect to source position 902, however, remains substantially unchanged. FIG. 16C illustrates a different later situation in which the source has changed its position 902 relative to the earth's reference frame, but the user does not follow the source movement. Instead, user 901 continues to look in the same direction as in the initial situation depicted in FIG. 16A. As a result, the angular orientation of housings 111, 411 with respect to source position 902 has changed by the angle α. FIG. 16D illustrates another different later situation in which the source remains in the initial position of FIG. 15A but the user turns his head by the angle α. In consequence, the angular orientation of housings 111, 411 with respect to source position 902 has also changed by the angle α.
Typically, real life situations are characterized by a superposition of the various movement situations idealized by FIGS. 16A to 16D. In particular, user 901 may not always pay attention to source position 902 when the source is moving, as shown in FIG. 16C, for instance when he is looking at a different point of interest such as a projection screen and/or involved in another activity such as writing. Moreover, user 901 may actively turn away from the source position 902, as shown in FIG. 16D, for instance during a longer conversation to avoid starring at the conversation partner. Those movement activities, however, make it rather difficult to provide reliable position data based on the collected displacement data, at least over a longer time in which the displacement data is collected. The reliability of the position data can be improved, however, by adjusting the position data in operation 606 based on the reliability measure obtained in operation 605, as described above.
FIGS. 17A and 17B illustrate operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure. A source 911 is provided at reference point 202. In the exemplary movement situation, user 901 wearing the hearing device is looking in a direction pointing away from the source. The processing unit of the hearing device is configured to process audio data based on a signal emitted from source 911 and to provide the processed audio signal as an output signal to the output transducer. During the signal processing, a directionality of the output signal is provided by amplifying a part of the audio data corresponding to a desired direction relative to another part of the audio data deviating from the desired direction. The directionality of the output signal, as perceived by user 901, is exemplified in FIGS. 17A and 17B by a conical beam 912 tapering towards the position of user 901. The desired direction of the processed audio data can be based on the position data of the housing relative to reference point 202. In the examples shown in FIGS. 17A and 17B, the directionality of beam 912 is selected such that it points from housing 111,411 toward source 911 at reference point 202. Thus, when user 901 moves with respect to reference point 202 at which source 911 is located from an initial position, as illustrated in FIG. 17A, to a later position, as illustrated in FIG. 17B, for instance by a rotational displacement, the directionality of beam 912 in the desired direction toward source 911 can be maintained. In this way, the user can perceive the audio output signal as a sound coming from the position of source 911, irrespective of his relative position to source 922.
For example, source 911 may be provided as a sound source in an environment of user, such as a conversation partner or a loudspeaker. A sound emitted from sound source 911 can be detected, for instance, by sound detector 106, 306, 307, 606 provided in housing 111, 411. The sound detector can thus provide audio data to the processing unit based on the detected ambient sound including sound emitted from sound source 911, which then can be processed by the processing unit in the above described way to provide the output signal. In addition, the audio data may be processed as described in conjunction with any of the methods illustrated in FIGS. 8 - 10 to provide auxiliary position data. For instance, the sound detector may comprise said plurality of spatially arranged microphones 306, 307 each providing audio data based on the detected ambient sound. From the audio data, the part of the audio data corresponding to the desired direction relative to the other part of the audio data deviating from the desired direction can be determined. The auxiliary position data can be also determined from the audio data. In this way, beamforming can be performed based on the audio data. A property of the beamforming can be controlled based on the position data generated in operation 604 and/or on the position data adjusted in operation 606. In this way, an improved signal to noise ratio of the output signal can be obtained. In the example illustrated in FIGS. 17A and 17B, the controlled property of the beamforming is a directionality of beam 912 toward the position of sound source 911. Alternatively or additionally, the controlled property of the beamforming may comprise a size of beam 912, such as a beam width, depending on the position data. As a specific example, FIGS. 17A and 17B may illustrate a situation in a car, wherein source 911 is a conversation partner sitting next to user 901.
As another example, source 911 may be provided as a radio source emitting radio waves received at housing 111, 411. For instance, source 911 may comprise remote sound detector 506 and radio transmitter 507 of hearing device 500, as depicted in FIG.4, wherein radio transmitter 507 is configured to transmit the sound detected by remote sound detector 506 at reference point 202 via radio waves. The radio waves can be received by a signal receiver communicatively coupled with the processing unit at housing 111, 411. For instance, the signal receiver may comprise a plurality of spatially arranged receiving units 508 each configured to receive the radio waves. For instance, a respective receiving unit 508 may be included in each housing 111, 411. The part of the audio data corresponding to the desired direction relative to the other part of the audio data deviating from the desired direction can then be determined from the radio waves received at different spatial positions, for instance by determining a phase difference and/or a difference in signal level, as determined by RSSI measurements. In this regard, an operational principle as disclosed in international patent application publication No. WO 2016/116160 A1 , which is included be reference, may be employed. The auxiliary position data can be also determined from the radio waves received at the different spatial positions. Alternatively or additionally, a plurality of spatially arranged microphones 306, 307 may be included in housing 111, 411 to detect ambient sound. The detected ambient sound can include the sound that is detected by remote sound detector 506. The part of the audio data corresponding to the desired direction relative to the other part of the audio data deviating from the desired direction and/or the auxiliary position data can then be determined from the audio data provided by microphones 306, 307, in the above described way. As a specific example, FIGS. 17A and 17B may illustrate a situation in which remote sound detector 506 is a microphone worn by a conversation partner of user 901.
FIGS. 18A and 18B illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure. The directionality of the output signal, as perceived by user 901, is selected such that beam 912 points in a different direction than the position of reference point 202 at which source 911 is located. The directionality of beam 912 relative to the position of source 911, however, is maintained when the user moves from an initial position, as illustrated in FIG. 18A, to a later position, as illustrated in FIG. 18B, for instance by a rotational displacement. In this way, the directionality of the output signal, as perceived by user 901, can be provided in a manner to create a hearing perception of a sound coming from a direction different than the source location. Such a sound perception produced by beam 912 can be employed, for instance, to provide an augmented reality to user 901. The augmented reality can thus add an interactive sound experience to a real world environment.
FIGS. 19A - 19C illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure. As depicted in FIG. 19A, source 911 is a first source positioned at reference point 202. A second source 913 is provided at a different spatial position. When the user moves from an initial position, as illustrated in FIG. 19A, to a later position, as illustrated in FIG. 19B, for instance by a rotational displacement, the property of the beamforming is controlled based on the position data such that beam 912 is transformed to a beam 914 comprising a larger beam width as compared to beam 912. The beam width of the beam provided by the beamforming is thus enlarged when the position data is indicative of a variation of the angular orientation and/or a spatial location of housing 111, 411 with respect to reference point 202 over time. In this way, beam 914 can be optimized such that it encompasses a signal emitted by first source 911 and second source 913. For instance, sources 911, 913 may be different conversation partners of user 901. The ambient sound detected by the hearing device at the position of housing 111, 411 can thus be processed as audio data indicative of a sound emitted by both sources 911, 913. The directionality of the output signal can thus be adapted with regard to the position of both sources 911, 913. The beam width may be enlarged to a fixed value of the beam width and/or changed to a value depending on the variation of the angular orientation and/or a spatial location over time.
After a while, for instance after a predetermined time interval, in which user 901 does not change his position with respect to reference point 202 at which first source 911 is located, as illustrated in FIG. 19C, beam 914 is transformed back to a beam 912 comprising the smaller beam width. Thus, the beam width is reduced when the position data is indicative of a constant angular orientation and/or a spatial location of housing 111,411 with respect to reference point 202 over time. Subsequently, reference point 202 is adjusted from a first reference position at which first source 911 is located to a second reference position at which second source 913 is located. Adjusting reference point 202 to the later reference position can comprise adjusting the position data indicative of an angular orientation and/or a spatial location of housing 111, 411 corresponding to the spatial difference between the earlier reference position and the later reference position of reference point 202. For instance, the method described above in conjunction with FIG. 9 and/or FIG. 10 may be employed to determine the position of second source 913 as reference point 202. The position data with respect to the adjusted reference point 202 produces a directionality of beam 912 such that it points from housing 111,411 toward second source 913. In particular, the situation depicted in FIG. 19C in which the position data does not change for a while may indicate that user 901 now focuses on a new source 913. The readjustment of reference point 202 and the resulting readjustment of the directionality of beam 912, in particular from the situation depicted in FIG. 19A to the situation depicted in FIG. 19C, can be used to account for this user behavior.
FIGS. 20A and 20B illustrate further operations of a hearing device employing position data, in accordance with some embodiments of the present disclosure. The position data generated in operation 604 and/or on the position data adjusted in operation 606 is transferred to an auxiliary device 921. For instance, auxiliary device 921 can be a smartphone or tablet operated by user 901. The transferred position data can then be employed in a program executed by auxiliary device 921 and/or stored in a memory of auxiliary device 921. For example, as illustrated in FIGS. 20A and 20B, the position of reference point 202 may be graphically reproduced as a point 922 on a map displayed by auxiliary device 921. The displayed map may depend on a viewing direction and/or a spatial position of user 901 with respect to the earth's reference frame. Thus, when the user moves from an initial position, as illustrated in FIG. 20A, to a later position, as illustrated in FIG. 20B, for instance by a rotational displacement, the position of point 922 is reproduced on a different position on the map corresponding to the transmitted position data.
While the principles of the disclosure have been described above in connection with specific devices and methods, it is to be clearly understood that this description is made only by way of example and not as limitation on the scope of the invention. The above described preferred embodiments are intended to illustrate the principles of the invention, but not to limit the scope of the invention. The scope of the present invention that is solely defined by the claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

A hearing device comprising
- a housing (111, 311, 411) configured to be worn at an ear of a user;

- a displacement detector (108, 408, 801) mechanically coupled with the housing (111, 311, 411), the displacement detector configured to provide displacement data indicative of a rotational displacement and/or a translational displacement of the housing (111, 311, 312, 411); and

- a processing unit (102, 402) communicatively coupled with the displacement detector (108, 408, 801), the processing unit configured to collect said displacement data in subsequent periods and to generate position data based on said collected displacement data, the position data indicative of an angular orientation and/or a spatial location of the housing (111, 311, 312, 411) with respect to a reference point (202, 902),
characterized in that the processing unit (102, 402) is configured to obtain a reliability measure of said position data, the reliability measure indicative of a reliability of said position data after said subsequent periods, and to adjust said position data based on said reliability measure, wherein the processing unit (102, 402) is configured to continuously generate the position data at a first frequency and to continuously obtain the reliability measure at a second frequency, wherein the second frequency is smaller than the first frequency.
The device according to claim 1, characterized in that the processing unit (102, 402) is configured to obtain the reliability measure based on auxiliary position data, the auxiliary position data provided independently from said position data, wherein the hearing device comprises a sound detector (106, 306, 307, 406, 506) configured to provide audio data to the processing unit (102, 402), the audio data indicative of an ambient sound, wherein the processing unit (102, 402) is configured to generate the auxiliary position data based on the audio data.
The device according to claim 2, characterized in that processing unit (102, 402) is configured to determine a presence of a sound emitted from a sound source (911, 913) in said audio data and to generate the auxiliary position data such that the auxiliary position data is indicative of an angular orientation and/or a spatial location of the housing (111, 311, 312, 411) with respect to a position of said sound source (911, 913).
The device according to claim 2 or 3, characterized in that the sound detector (106, 306, 307, 406, 506) comprises a plurality of spatially arranged microphones (306, 307) each configured to provide audio data to the processing unit (102, 402), the processing unit configured to determine a difference between the audio data provided by at least two of said spatially arranged microphones (306, 307) and to generate the auxiliary position data based on the difference.
The device according to any of the preceding claims, characterized in that the processing unit (102, 402) is configured to obtain the reliability measure based on auxiliary position data, the auxiliary position data provided independently from said position data, wherein the hearing device comprises a signal receiver (508) configured to receive radio waves emitted from a radio source (507), wherein the processing unit (102, 402) is configured to generate the auxiliary position data based on the received radio waves such that the auxiliary position data is indicative of an angular orientation and/or a spatial location of the housing (111, 311, 312, 411) with respect to a position of said radio source (507).
The device according to any of the preceding claims, characterized in that the processing unit (102, 402) is configured, while obtaining the reliability measure, to apply an algorithm to the position data and/or the displacement data and to determine a correction value from the applied algorithm, wherein the adjusting the position data is based on the correction value, wherein the algorithm comprises determining an intermediate value between a value extracted from the position data and a predefined value, the predefined value indicative of a predefined angular orientation and/or a predefined spatial location of the housing (111, 311, 312, 411) with respect to the reference point (202, 902), wherein the correction value is set to the intermediate value.
The device according to any of the preceding claims, characterized in that the processing unit (102, 402) is configured, while obtaining the reliability measure, to apply an algorithm to the position data and/or the displacement data and to determine a correction value from the applied algorithm, wherein the adjusting the position data is based on the correction value, wherein the algorithm comprises classifying, based on patterns of significance of displacement data and/or position data, the collected displacement data and/or the generated position data with respect to a significance level, the significance level indicative of a probability that the collected displacement data and/or the generated position data is significant for said angular orientation and/or spatial location of the housing (111, 311, 312, 411) with respect to the reference point (202, 902), and to determine the correction value depending on the significance level.
The device according claim 7, characterized in that the processing unit is configured to maintain a record of said collected displacement data and/or generated position data over time and to determine said patterns of significance from said record.
The device according any of claims 2 to 5 and claim 7 or 8, characterized in that the processing unit is configured to determine said patterns of significance based on a correlation between said auxiliary position data and said collected displacement data and/or generated position data.
The device according to any of the preceding claims, characterized in that the processing unit is configured to provide an output signal based on audio data processed by the processing unit (102, 402), the hearing device comprising an output transducer (110, 410) configured to output the output signal to stimulate the user's hearing, wherein the processing unit (102, 402) is configured to provide a directionality of the output signal by amplifying a part of said audio data corresponding to a desired direction relative to another part of said audio data deviating from the desired direction, wherein the processing unit (102, 402) is configured to determine the desired direction based on said position data.
The device according to claim 10, characterized in that said processing unit is configured, while providing said directionality of the output signal, to provide beamforming of the output signal, and to control a beam width of the beamforming based on said position data such that the beam width is enlarged when said position data is indicative of a variation of the angular orientation and/or the spatial location of the housing with respect to the reference point over time.
The device according to claim 10 or 11, characterized in that the processing unit is configured to obtain audio data from a remote sound detector (506) provided at a position remote from the housing (111, 311, 312, 411), wherein the audio data on which the output signal is based comprises the audio data provided by the remote sound detector (506).
The device according to claim 12, characterized in that the hearing device comprises a signal receiver (508) communicatively coupled with the processing unit (102, 402), the signal receiver (508) configured to receive said audio data from the remote sound detector (506) transmitted by radio waves.
A method of operating a hearing device comprising a housing (111, 311, 312, 411) configured to be worn at an ear of a user and a displacement detector (108, 408, 801) mechanically coupled with the housing (111, 311, 411), the displacement detector configured to provide displacement data indicative of a rotational displacement and/or a translational displacement of the housing (111, 311, 312, 411), the method comprising
- providing said displacement data;

- collecting said displacement data in subsequent periods;

- generating position data based on said collected displacement data, the position data indicative of an angular orientation and/or a spatial location of the housing (111, 311, 312, 411) with respect to a reference point,
characterized by
- obtaining a reliability measure of said position data, the reliability measure indicative of a reliability of the position data after said subsequent periods; and

- adjusting the position data based on said reliability measure, wherein the position data is continuously generated at a first frequency and the reliability measure is continuously obtained at a second frequency, wherein the second frequency is smaller than the first frequency.
A computer-readable medium storing instructions that, when executed by a processor, cause a hearing device to perform operations of the method according to claim 14.