EP4611398A1

EP4611398A1 - System and method for providing hearing assistance

Info

Publication number: EP4611398A1
Application number: EP24160026.1A
Authority: EP
Inventors: Georg Dickmann
Original assignee: Sonova AG
Current assignee: Sonova Holding AG
Priority date: 2024-02-27
Filing date: 2024-02-27
Publication date: 2025-09-03

Abstract

There is provided a hearing assistance system comprising: an audio broadcast device (14) configured to receive an audio source signal as a non-acoustic first audio signal (20) from an audio source device (12), add audio processing metadata (38) to the first audio signal; and broadcast, via a wireless interface (40) of the audio broadcast device, the first audio signal together with the audio processing metadata, as an Auracast broadcast stream; (42); a microphone arrangement (28) configured to receive the audio source signal as an acoustic stream (18) and capture a second audio signal (30) from the acoustic stream; a correlation unit (32) configured to determine a correlation of the first audio signal and the second audio signal; a metadata configuration unit (36) configured to configure the audio processing metadata based on the determined correlation; and an audio listening device (16) for rendering the first audio signal to the user, including a wireless interface (44) configured to receive the Auracast broadcast stream from the audio broadcast device, a signal processing unit (46) configured to process the received first audio signal according to the received audio processing metadata, and an output transducer (50) configured to stimulate the user's hearing according to the processed received first audio signal (64).

Description

The invention relates to a system and a method for providing hearing assistance to a user, wherein an audio source signal from an audio source device, such as a TV set, is provided both as a non-acoustic audio signal and as an acoustic stream and wherein an audio broadcast device transmits the audio signal from the audio source device as a wireless Auracast broadcast stream to an audio listening device which stimulates the user's hearing according to the audio signal received from the audio broadcast device. The audio listening device may be a hearing device to be worn at ear level, such as a hearing aid, a headset or headphones.
WO 2017/088909 A1 relates to a hearing aid which receives an wireless audio signal from an audio source device and captures an ambient audio signal by a the hearing aid microphone, wherein the wireless audio signal and the ambient audio signal are mixed for reproducing a mixed audio signal to the hearing aid user, and wherein the mixing weights are determined based on the sound level differences of the wireless audio signal and the ambient audio signal.
US 2011/0142268 A1 relates to a wireless TV set to be used with a hearing aid, wherein the hearing aid receives both an acoustic stream provided by a loudspeaker of the TV set and, directly or via a mobile relay device, a wireless stream of the audio signal. A delay between the acoustic stream and the audio signal in the wireless stream is determined in the hearing aid or the relay device, respectively, by comparing the received signals, and the corresponding delay information is transmitted wirelessly to the TV set which then applies the respective delay compensation on the loudspeaker input to delay the acoustic stream.
It is an object of the invention to provide for a hearing assistance system wherein an audio source signal is provided both as a wireless stream and as an acoustic stream to the user of an audio listening device and wherein audio quality is to be optimized in a user-friendly manner, without the need for significantly increased resources in the listening device. It is a further object to provide for a corresponding method.
According to the invention, these objects are achieved by a system as defined in claim 1, an audio broadcast device as defined in claim 12 and a method as defined in claim 23, respectively.
The invention is beneficial in that, by adding audio processing metadata configured in conformity with a determined correlation of the first audio signal and the second audio signal to the first audio signal and broadcasting the first audio signal together with the audio processing metadata as an Auracast broadcast stream, the audio listening device which receives the Auracast broadcast stream can automatically adjust the audio signal processing to various usage situations, so as to optimize quality of the rendered audio signal, in particular with regard to timing and level, without a need for user interaction.
Preferred embodiments of the invention are defined in the dependent claims.
One aspect of the invention relates to a system as defined in claim 1.
According to one example, the correlation unit may be configured to determine whether the first audio signal is present as a component in the second audio signal and, if the first audio signal is determined to be present as a component in the second audio signal, to determine a delay between the first audio signal and the first audio signal component in the second audio signal and a level of the first audio signal component in the second audio signal. In particular, the audio processing metadata may be configured to adjust a delay of the rendering of the first audio signal by the audio listening device. Further, the audio processing metadata may be configured to minimize the delay of the rendering of the first audio signal by the audio listening device when the first audio signal is determined to be present as a component in the second audio signal, so as to minimize the time difference between the user's perception of the rendered first audio signal and the user's perception of the acoustic stream or the rendering of an audio signal captured from the acoustic stream.
For example, the audio processing metadata may be configured to adjust a level of the rendering, by the audio listening device, of the received first audio signal and/or an ambient audio signal captured by the audio listening device. In particular, the audio processing metadata may be configured to reduce the level of the rendering of the ambient audio signal captured by the audio listening device, when the first audio signal is determined to be present as a component in the second audio signal, so as to attenuate the rendering of the ambient audio signal relative to the rendering of the received first audio signal. Further, the audio processing metadata may be configured to adjust a level of the rendering of the received first audio signal in such a manner that the loudness perception of the rendered first audio signal by the user approximates the loudness perception of the acoustic stream as perceived by a normal-hearing person not wearing an audio listening device, when the first audio signal is determined to be present as a component in the second audio signal.
According to one example, the audio broadcast device may be configured to switch between different wireless schedules based on whether or not the first audio signal has been determined to be present as a component in the second audio signal. In particular, the audio broadcast device may be is configured to select a first wireless schedule when the first audio signal has been determined to be present as a component in the second audio signal and a second wireless schedule when the first audio signal has been determined to be not present as a component in the second audio signal, wherein the first wireless schedule may have a lower transmission robustness and a lower transport latency than the second wireless schedule. In particular, the audio broadcast device may be configured to select a third wireless schedule when the first audio signal 20) has been determined to be present as a component in the second audio signal and a delay of the second audio signal relative to the first audio signal has been determined to exceed a predetermined threshold value, wherein the first wireless schedule may have a lower transmission robustness and a lower transport latency than the third wireless schedule.
According to one example, the audio listening device may be a hearing device. In particular, the audio listening device may be a hearing aid or an auditory prothesis, such as a cochlear implant.
According to one example, the audio listening device may comprise an ambient microphone arrangement for capturing an ambient audio signal from ambient sound and the audio signal processing unit may comprise adjustable scaling units for scaling the received first audio signal and the captured ambient audio signal, respectively, based on the received audio processing metadata, a delay unit for adjusting a delay of the received first audio signal based on the received audio processing metadata and a mixing unit for combining the scaled delayed first audio signal and the scaled ambient audio signal prior to supplying the combined signal to the output transducer.
According to one example, the correlation unit and the metadata configuration unit may be integrated within the audio broadcast device. In particular, the microphone arrangement may be integrated within the audio broadcast device or may be connected to the audio broadcast device via a wired connection or a wireless link.
According to one example, the metadata configuration unit may be integrated within the audio broadcast device and the correlation unit may be integrated within an external device configured to receive the Auracast broadcast stream for using the received first audio signal in the correlation unit and to transmit data representative of the determined correlation to the audio broadcast device. In particular, the microphone arrangement may be integrated within the external device.
According to one example, the external device may be formed by the audio listening device.
Another aspect of the invention relates to an audio broadcast device as defined in claim 12.
Still another aspect of the invention relates to a method of providing hearing assistance to a user as defined in claim 13.
According to one example, the audio source signal may be selected from a plurality of audio source devices by an input selection unit. In particular, the acoustic stream corresponding to the audio source signal may be generated via a loudspeaker arrangement by selectively providing the audio source signal, via the input selection unit, to the loudspeaker arrangement.
According to one example, the audio source device may be a TV set, a media player or a microphone device, such as a conference microphone, for capturing a speaker's voice.
According to one example, the microphone arrangement may be located at a position of the user of the audio listening device. According to an alternative example, the microphone arrangement may comprise a plurality of spaced apart microphones placed at multiple listener positions.
Hereinafter, examples of the invention will be illustrated by reference to the attached drawing, wherein:
Fig. 1 is a schematic illustration of a first example of a system according to the invention.
A "hearing device" as used hereinafter is any ear level device suitable for reproducing sound by stimulating a user's hearing, such as an electroacoustic hearing aid, a bone conduction hearing aid, an active hearing protection device, a hearing prosthesis device such as a cochlear implant, a wireless headset, an earbud, an earplug, an earphone, etc.
The expression "the first audio signal is present as a component in the second audio signal" as used hereinafter means that there is a certain degree of correlation between the first audio signal and the second audio signal. This is the case when a signal corresponding to the first audio signal is contained as part of the second audio signal.
Auracast is a new technology providing audio broadcasting via Bluetooth Low Energy and may be used in various public locations, including schools.
An audio broadcast device providing an Auracast broadcast stream, as used hereinafter, is a device implementing the Public Broadcast Source (PBP) role as defined in the Bluetooth Public Broadcast Profile specification and therefore acting as an Auracast audio source. It is noted that the Auracast broadcast stream, in addition to audio data, may include associated metadata which may be carried by associated periodic and extended advertisements.
"Metadata" as used hereinafter is meant to refer to any data associated with the broadcast audio stream other than the actual audio content and hence even may apply to data elements which the Bluetooth specification relating to LE Audio Broadcast (PBP, Basic Audio Profile (BAP)), which specifies data structures to be transmitted in parallel to an audio broadcast, does not label "metadata". For example, "metadata" as used hereinafter also includes a field named 'presentation delay' in the so-called "Basic Audio Announcements" defined in the Basic Audio Profile specification, which field is not labeled "metadata in" the Bluetooth specification.
An audio listening device for receiving an Auracast broadcast stream, as used hereinafter, is a device implementing the Public Broadcast Sink (PBS) role as defined in the Bluetooth Public Broadcast Profile specification and therefore acting as an audio receiver device for receiving an Auracast broadcast stream from an Auracast broadcast device. For example, the Auracast audio listening device may be implemented as a hearing device, such as a hearing aid, which is worn by student in a classroom.
Fig. 1 is a schematic illustration of an example of a system for providing hearing assistance to a user, comprising an audio broadcast device 14 and an audio listening device 16 to be worn by the user.
An audio source signal originating from one of a plurality of audio source devices 12 is provided both as an acoustic stream 18 and as a non-acoustic first audio signal 20. The audio source devices 12 may include, for example, a TV set 12-1, a microphone device 12-2 for capturing a speaker's voice, such as a conference microphone, and a media player 12-3.
An input selection unit 22 may be provided for selecting, at a time, one of the audio source devices for providing the audio source signal. The respectively selected first audio signal 20 is supplied to an audio input 24 of the audio broadcast device 14.
Further, the respectively selected first audio signal 20 may also be supplied to a loudspeaker arrangement 26, such as a PA (public address) system, for generating an acoustic stream 18 corresponding to the first audio signal 20.
Alternatively, an audio source device 12 may include its own loudspeaker arrangement for generating the acoustic stream 18 corresponding to the first audio signal 20 (in case the audio source device is a microphone device 12-2, the acoustic stream is given by the voice of the speaker using the microphone device 12-2).
The audio broadcast device 14 further comprises - or is connected to - a microphone arrangement 28 which receives the acoustic stream 18 and captures a second audio signal 30 from the acoustic stream 18 (the microphone arrangement 28 may be considered as a kind of an "ambient microphone" and the second audio signal may be considered as a kind of an "ambient audio signal").
The first audio signal 20 and the second audio signal 30 are supplied to a correlation unit 32 of the audio broadcast device 14 for determining a correlation of the first audio signal 20 and the second audio signal 30.
The output 34 of the correlation unit 32 is provided to a metadata configuration unit 36 of the audio broadcast device 14 for configuring audio processing metadata based on the correlation determined by the correlation unit 32.
The first audio signal 20 and the audio processing metadata 38 are supplied to a Bluetooth Low Energy (BTLE) interface 40 of the audio broadcast device 14 which combines the first audio signal 20 and the audio processing metadata 38 so as to generate and broadcast an Auracast broadcast stream 42 which includes an audio stream and associated metadata carried by associated periodic and extended advertisements.
The audio listening device 16 comprises a BTLE interface 44 for receiving the Auracast broadcast stream 42 from the audio broadcast device 14, a signal processing unit 46 for processing the received first audio signal 20 according to the received audio processing metadata 38 and an output transducer 50 for stimulating the hearing of the user of the audio listening device 16 according to the processed received first audio signal 48 generated by the signal processing unit 46.
The audio listening device 16 may include - or be connected to - a microphone arrangement 52 (hereinafter referred to as "ambient microphone arrangement") for capturing an audio signal 54 from ambient sound (hereinafter referred to as "ambient audio signal") which is supplied - in parallel to the received first audio signal 20 - to the signal processing unit 46 wherein it is processed so as to be rendered to the user by the output transducer 50 as part of the processed received first audio signal 48.
In the example shown in Fig. 1, the audio signal processing unit 46 comprises an adjustable scaling unit 56 for scaling the received first audio signal 20 and an adjustable scaling unit 58 for scaling the captured ambient audio signal 54, respectively, based on the received audio processing metadata 38, a delay unit 60 for adjusting a delay of the scaled received first audio signal 62 based on the received audio processing metadata and a mixing unit 64 for combining the delayed scaled received first audio signal 66 and the scaled ambient audio signal 68.
For example, the audio listening device 16 may be a hearing device to be worn at ear level, in particular a hearing instrument, such as a hearing aid or cochlear implant, a headset, an earphone or a headphone.
The correlation unit 32 is configured to determine whether the first audio signal 20 is present as a component in the captured second audio signal 30, i.e. whether a signal similar to the first audio signal 20 can be found in the second audio signal 30. If so, the correlation unit 32 determines a delay between the first audio signal 20 and the first audio signal component in the second audio signal 30 and a level of the first audio signal component in the second audio signal 30.
Such delay information is then transported in the audio processing metadata 38 so as to adjust, via the delay unit 60, a delay of the rendering of the first audio signal by the audio listening device 16 based on the delay determined by the correlation unit 32.
For example, thereby the delay of the rendering of the fist audio signal by the audio listening device may be minimized so as to minimize the time difference between the perception of the rendered wirelessly received first audio signal (indicated at 20 in Fig. 1) by the user and the perception of the ambient audio signal, i.e. the rendering of the audio signal (indicated at 54 in Fig. 1) captured from the acoustic stream 18. This helps to avoid echo or comb filter effects.
The delay unit 60 can also be utilized for achieving or improving lip-synchronization of the rendered received first audio signal with regard to a video presentation associated with the respective audio source device 12 providing the first audio signal 20, such as the TV-set 12-1.
To this end, the audio broadcast device 14 may be configured to determine whether there is a video presentation associated with an audio source device providing the audio source signal, and, if so, to configure the audio processing metadata 38 such that the rendering of the first audio signal is delayed in a manner so as to achieve or improve lip-synchronization of the rendered first audio signal with regard to the video presentation. The required information may be obtained by the audio broadcast device 14 via a specific input or user control through which the presence of video material and, if available, the relative timing of the audio input and the video material can be configured.
For example, in case that the audio source 12-1 is a TV device that provides the first audio signal 20 as output such that immediate rendering by the hearing device 16 results in good lip-synchronization, then a metadata element 'render as soon as possible' applied to the delay unit 60 will result in best lip synchronization. On the other hand, in case that the TV device 12-1 provides the first audio signal 20 as output that is early relative to the video, some (unknown) moderate delay may result in best lip synchronization. Here, lip-synchronization can be achieved, for example, by the user manually configuring the delay metadata via the metadata configuration unit 36 in the audio broadcast device 14 based on knowledge of the relative audio/video delay settings of the TV device 12-1. Alternatively, some specific automated configuration interface between the TV device 12-1 and the audio broadcast device 14 may be foreseen.
In addition to delay adjustment the audio processing metadata 38 can be used to adjust a level of the rendering of an audio signal by the audio listening device 16. In the example of Fig. 1 such level adjustment occurs via corresponding control of the scaling units 56 and 58, respectively, by audio processing metadata 38.
For example, the level of the rendering of an ambient audio signal 54 captured by the microphone arrangement 52 of the audio listening device 16 may be reduced when the first audio signal 20 is determined to be present as a component in the second audio signal 30, so as to attenuate the rendering of the ambient audio signal 54 relative to the rendering of the received first audio signal 20. Thereby ambient noise can be reduced, since the streamed first audio signal 20 in this case is a noise-reduced variant of the ambient audio signal 54.
Reducing the level of the ambient audio signal 54 to a minimum may be particularly helpful whenever the first and second audio signals are correlated but synchronicity of rendering cannot be achieved, thereby avoiding comb filter effects or echo.
If both first and second audio signal can be aligned thanks to appropriate adjustment of the delay unit 60, and if the noise level in the ambient signal is low, the level of the ambient audio signal 54 may be higher for better naturalness and ambient awareness.
In particular, when the first audio signal is determined to be present as a component in the second audio signal, the level of the rendering of the first audio signal 20 may be adjusted in such a manner that the loudness perception of the rendered first audio signal by the user approximates the loudness perception of the acoustic stream 18 as perceived by a normal-hearing person not wearing the audio listening device 16.
For example, the level / intensity of the rendering of the first audio signal 20 may be adjusted so as to be proportional to the determined level of the first audio signal component in the second audio signal 30.
Further, the correlation result obtained by the correlation unit 32 may be used to not only adjust the delay and the level of the rendering of the first audio signal by the audio listening device 16, but to also adjust wireless scheduling parameters.
To this end, the audio broadcast device 14 may switch between different wireless schedules based on whether or not the first audio signal 20 has been determined to be present as a component in the second audio signal 30.
For example, the audio broadcast device 14 may select a first wireless schedule when the first audio signal has been determined to be present as a component in the second audio signal (this may be an indication of a live speaker using a microphone, resulting in a need for low latency) and a second wireless schedule when the first audio signal has been determined to be not present as a component in the second audio signal, wherein the first wireless schedule has a lower transport latency - and a resulting lower transmission robustness - than the second wireless schedule.
Moreover, the audio broadcast device may select a third wireless schedule when the first audio signal has been determined to be present as a component in the second audio signal and a delay of the second audio signal relative to the first audio signal has been determined to exceed a predetermined threshold value, wherein the first wireless schedule has a lower transport latency - and hence a lower transmission robustness - than the third wireless schedule. The third wireless schedule utilizes the fact that, if the ambient audio signal arrives substantially later than the wireless audio signal, there is enough time for a more robust schedule. The selection threshold value should be defined accordingly.
Thereby audio latency can be traded against robustness. i.e., when short latency is not relevant, a transmission schedule can be adopted that leads to more robust reception of the audio signals.
In the example of Fig. 1, the correlation unit 34 and the metadata configuration unit 36 are integrated within the audio broadcast device 14, and also the ambient microphone arrangement 28 may be integrated within the audio broadcast device 14.
However, in alternative embodiments some or all of these components may be implemented externally.
For example, the ambient microphone arrangement 28, instead of being integrated within the audio broadcast device 14, may be connected to the audio broadcast device 14 via a wired connection or a wireless link, thereby acting as a remote microphone. This allows, in principle, to place the ambient microphone arrangement 28 at a position close to the where the audio listening device 16 typically will be located when used by the user (for precisely adjusting the delay and the level of the rending of the streamed first audio signal 20 it is beneficial to pick up the acoustic stream 18 at a position close to where it is picked up by the microphone arrangement 52 of the audio listening device 16). It is noted that the microphone arrangement also may comprise a plurality of spaced apart microphones placed at multiple listener positions, with audio processing metadata being derived to achieve settings that are appropriate for multiple listener positions. Such spaced apart microphones also may receive the Auracast broadcast stream 42 from the audio broadcast device 14 so as to locally perform derivation of optimum metadata from local correlation and transfer these metadata to the audio broadcast device 14.
Further, the correlation unit 32 may be integrated within an external device which is configured to receive, via a suitable interface, the audio broadcast stream 42 and to also receive the second audio signal 30 captured from the acoustic stream 18, for example via a microphone arrangement integrated within or connected to the external device. The external device supplies the received first audio signal 20 and the captured second audio signal 30 to the correlation unit 32 and transmits, via a suitable interface, the output 34 of the correlation unit 32, i.e., data representative of the determined correlation, to the audio broadcast device 14 where the correlation data 34 is supplied to the metadata configuration unit 36.
According to a variant of this example, also the metadata configuration unit 36 may be implemented in the external device. In this case, the external device would transmit the audio processing metadata 38 provided by the metadata configuration unit 36 to the audio broadcast device 14, so that the audio processing metadata 38 can be combined in the Bluetooth LE interface 40 with the first audio signal 20.
For example, the external device may be formed by the audio listening device 16, i.e., by an accordingly modified version of the audio listening device 16 shown in Fig. 1.
It is noted that the variants that perform the correlation of the captured ambient audio signal with streamed first audio signal in a device that receives the streamed first audio signal, suffer from the drawback that the correlation must be done once the Auracast broadcast stream 42 is already transmitted. As a result of the correlation / analysis, the Auracast broadcast stream (i.e., the audio stream and/or associated metadata) may need to be reconfigured with the newly determined configuration/metadata, which may result in a temporary outage of the stream.
The audio processing metadata added to the first audio signal in the Auracast broadcast stream may be used to inform the receiving audio listening devices of the best time of rendering, the intended rendering level and whether or not the streamed first audio signal is also present in the captured ambient audio signal.
In the context of Bluetooth Auracast the time of rendering is represented by the BASE parameter "presentation_delay"(specified in the Bluetooth Basic Audio Profile (BAP)) and/or the presence of the metadata "Broadcast Immediate Rendering Flag" (specified in Bluetooth Assigned Numbers).
Further, in the context of Bluetooth Auracast, information as to whether or not the streamed first audio signal is also present in the captured ambient audio signal, can be associated with a value of 'Live' in the Context Type metadata transported by BASE (see BAP for BASE, see Bluetooth Assigned Numbers for the definition of 'Live').
Regarding the intended rendering level Bluetooth Auracast currently does not define any corresponding metadata but these may be specified later or use proprietary data elements.
In a practical use example, the audio broadcast device 14 may be deployed in church services with audio listening devices 16 worn by participants of the church service.
For example, while the priest is talking via the microphone device 12-2 and the loudspeaker arrangement 26, the correlation would detect that the captured ambient signal 30 and the streamed signal 20 are highly correlated and that the streamed signal 20 needs to be rendered as soon as possible (assuming that the priest's voice is delayed by nothing but the speed of sound in the church). Accordingly, the Auracast audio broadcast device 14 will, for example, insert the "Broadcast Immediate Rendering flag" metadata, set the Context Type value to "Live" and set the value of "presentation delay" to a small value that can be expected to be supported by the majority of receiving audio listening devices, e.g. 20 ms.
During some meditation session, the audio source signal may be switched via the input selection unit 22 to a media player 12-3, playing meditative sound. The meditative sound is then also rendered by the local loudspeakers 26, with the amplifier system adding delay the audio signal. As a result, the Auracast audio broadcast device 14 may omit the 'Broadcast Immediate Rendering flag' and set a value for 'presentation delay' that causes good time match between the captured ambient audio signal 54 and the streamed audio signal 20 at the audio listening devices 16. The Context Type may remain at the value of 'Live' to reflect the fact that the streamed audio signal is also present in the captured ambient audio signal.
Alternatively, the meditative sound may not be rendered by a local loudspeaker system, for example, in order not to interfere with guided tours through the church. In this situation, the Auracast audio broadcast device 14 may switch to a different wireless schedule resulting in higher robustness of the transmission at increased transport latency. At the same time, the value of Context Type may be set to 'media'.
As described above, in wireless hearing assistance systems, it is desirable that, when the streamed audio signal is simultaneously present also as an ambient audio signal, the audio listening device renders the wirelessly received audio signal at the exact same time as the ambient audio signal to avoid echo or comb filter effects and at an intensity level that is proportional to the level of the ambient audio signal; further the audio listening device should attenuate the ambient signal in order to suppress ambient noise.
This can be achieved by an audio broadcast device which propagates, alongside the wireless audio signal, appropriate metadata. Changes to the audio system, like switching from a microphone input to a media input or presence, delay and volume level of a loudspeaker system should ideally be reflected to a corresponding adjustment of the configuration of the audio broadcast device. Automatic adjustment of the metadata can be achieved by determining a correlation between the wireless signal and the ambient audio signal.
When such correlation is found the audio broadcast device may determine the relative timing between both signals and/or the level of the wireless signal within the ambient audio signal. The audio broadcast device then automatically configures the metadata accordingly and, optionally, also may adjust its wireless scheduling parameters to achieve best robustness of wireless transmission at an appropriate audio latency.
Alternatively, the correlation of ambient and transmitted signal can be performed by a device that receives the streamed audio signal which then informs the audio broadcast device about the correlation results or the best transmitter settings and metadata as derived from the correlation.
As a result, audio listening devices that receive the streamed audio signal and associated metadata, can render audio at a time such that ambient and streamed audio have minimum relative time difference, while attenuating the ambient signal, knowing that the streamed signal is a noise-reduced variant of the ambient signal, and at a loudness level that matches the loudness level perceived by a person listening to the ambient signal without resorting to a hearing device
A method for operating an audio broadcast device 14 may comprise the following steps: receiving an audio signal over an audio input; receiving an ambient signal over an ambient microphone input; correlating the audio signal and the ambient signal, deriving metadata; and transmitting the audio signal and the metadata as an Auracast broadcast stream. The audio listening devices receiving the stream then may adjust the presentation delay for the received audio signal based on the metadata and also may adjust the level of ambient signal presented by the audio listening device to the user's ear depending on metadata information describing whether the streamed audio signal is acoustically present in the ambient signal.

Claims

A system for providing hearing assistance to a user, comprising:
an audio broadcast device (14) configured to
receive an audio source signal as a non-acoustic first audio signal (20) from an audio source device (12),

add audio processing metadata (38) to the first audio signal; and

broadcast, via a wireless interface (40) of the audio broadcast device, the first audio signal together with the audio processing metadata, as an Auracast broadcast stream; (42)

a microphone arrangement (28) configured to receive the audio source signal as an acoustic stream (18) and capture a second audio signal (30) from the acoustic stream;

a correlation unit (32) configured to determine a correlation of the first audio signal and the second audio signal;

a metadata configuration unit (36) configured to configure the audio processing metadata based on the determined correlation; and

an audio listening device (16) for rendering the first audio signal to the user, including
a wireless interface (44) configured to receive the Auracast broadcast stream from the audio broadcast device;

a signal processing unit (46) configured to process the received first audio signal according to the received audio processing metadata; and

an output transducer (50) configured to stimulate the user's hearing according to the processed received first audio signal (64).
The system of claim 1, wherein the correlation unit (32) is configured to determine whether the first audio signal (20) is present as a component in the second audio signal (30) and, if the first audio signal is determined to be present as a component in the second audio signal, to determine a delay between the first audio signal and the first audio signal component in the second audio signal and a level of the first audio signal component in the second audio signal.
The system of claim 2, wherein the audio processing metadata (38) is configured to adjust a delay of the rendering of the first audio signal (20) by the audio listening device (16).
The system of claim 3, wherein the audio processing metadata (38) is configured to minimize the delay of the rendering of the first audio signal (20) by the audio listening device (14) when the first audio signal is determined to be present as a component in the second audio signal (30), so as to minimize the time difference between the user's perception of the rendered first audio signal and the user's perception of the acoustic stream or the rendering of an audio signal captured from the acoustic stream.
The system of one of claims 2 to 4, wherein the audio processing metadata (38) is configured to adjust a level of the rendering, by the audio listening device (16), of the received first audio signal (20) and/or an ambient audio signal (54) captured by the audio listening device (16).
The system of claim 5, wherein the audio processing metadata (38) is configured to reduce the level of the rendering of the ambient audio signal (54) captured by the audio listening device (16), when the first audio signal (20) is determined to be present as a component in the second audio signal (30), so as to attenuate the rendering of the ambient audio signal relative to the rendering of the received first audio signal.
The system of one of claims 5 and 6, wherein the audio processing metadata (38) is configured to adjust a level of the rendering of the received first audio signal (20) in such a manner that the loudness perception of the rendered first audio signal by the user approximates the loudness perception of the acoustic stream (18) as perceived by a normal-hearing person not wearing an audio listening device, when the first audio signal is determined to be present as a component in the second audio signal (30).
The system of one of the preceding claims, wherein the audio broadcast device (14) is configured to switch between different wireless schedules based on whether or not
the first audio signal (20) has been determined to be present as a component in the second audio signal (30).
The system of claim 8, wherein the audio broadcast device (14) is configured to select a first wireless schedule when the first audio signal (20) has been determined to be present as a component in the second audio signal (30) and a second wireless schedule when the first audio signal has been determined to be not present as a component in the second audio signal, and wherein the first wireless schedule has a lower transmission robustness and a lower transport latency than the second wireless schedule.
The system of claim 9, wherein the audio broadcast device (14) is configured to select a third wireless schedule when the first audio signal (20) has been determined to be present as a component in the second audio signal (30) and a delay of the second audio signal relative to the first audio signal has been determined to exceed a predetermined threshold value, wherein the first wireless schedule has a lower transmission robustness and a lower transport latency than the third wireless schedule.
The system of one of the preceding claims, wherein the audio listening device (16) is a hearing device, wherein the audio listening device (16) comprises an ambient microphone arrangement (52) for capturing an ambient audio signal (54) from ambient sound (18), and wherein the audio signal processing unit comprises adjustable scaling units (56, 58) for scaling the received first audio signal (20) and the captured ambient audio signal, respectively, based on the received audio processing metadata (38), a delay unit (60) for adjusting a delay of the received first audio signal based on the received audio processing metadata and a mixing unit (64) for combining the scaled delayed first audio signal (66) and the scaled ambient audio signal (68) prior to supplying the combined signal (48) to the output transducer (50).
An audio broadcast device configured to
receive an audio source signal as a non-acoustic first audio signal (20) from an audio source device (12),

receiving a second audio signal (30) captured by a microphone arrangement (28) from an acoustic stream (18) corresponding to audio source signal;

determine a correlation of the first audio signal and the second audio signal;

configure audio processing metadata (38) based on the determined correlation, wherein the audio processing metadata are configured to be used for processing the first audio signal when rendering the first audio signal to the user by an audio listening device (16);

add the audio processing metadata to the first audio signal; and

wirelessly broadcast the first audio signal together with the audio processing metadata as an Auracast broadcast stream (42).
A method of providing hearing assistance to a user, comprising
providing, from an audio source device (12), an audio source signal both as a non-acoustic first audio signal (20) and as an acoustic stream (18);

receiving, by an audio broadcast device (14), the first audio signal from the audio source device;

receiving, by a microphone arrangement (28), the acoustic stream and capturing a second audio signal (30) from the acoustic stream;

determining a correlation of the first audio signal and the second audio signal;

configuring audio processing metadata (38) in conformity with the determined correlation;

adding the audio processing metadata to the first audio signal,

broadcasting, via a wireless interface (40) of the audio broadcast device, the first audio signal together with the audio processing metadata, as an Auracast broadcast stream (42);

receiving, by a wireless interface (44) of an audio listening device 16), the wireless broadcast audio stream containing the first audio signal and the audio processing metadata from the audio transmission device;

processing the received first audio signal according to the received audio processing metadata; and

stimulating, by an output transducer (50) of the audio listening device, the user's hearing according to the processed received first audio signal (48).
The method of claim 13, wherein the audio source signal is selected from a plurality of audio source devices (12-1, 12-2, 12-3) by an input selection unit (22), and wherein the acoustic stream (18) corresponding to the audio source signal (20) is generated via a loudspeaker arrangement (26) by selectively providing the audio source signal, via the input selection unit (22), to the loudspeaker arrangement.
The method of one of claims 13 and 14, wherein the microphone arrangement (28) is located at a position of the user of the audio listening device (16) or wherein the microphone arrangement (28) comprises a plurality of spaced apart microphones placed at multiple listener positions.