EP2755403B1

EP2755403B1 - Hearing assistance device eavesdropping on a bluetooth data stream

Info

Publication number: EP2755403B1
Application number: EP14150843.2A
Authority: EP
Inventors: Jeffrey P. Solum
Original assignee: Starkey Laboratories Inc
Current assignee: Starkey Laboratories Inc
Priority date: 2013-01-10
Filing date: 2014-01-10
Publication date: 2020-10-14
Anticipated expiration: 2034-01-10
Also published as: US10764694B2; US20140192988A1; EP2755403A1; US20200351598A1; US10321244B2; US20190297434A1; US11159896B2

Description

TECHNICAL FIELD

This document relates generally to hearing assistance systems and more particularly to hearing assistance device eavesdropping on a Bluetooth data stream.

BACKGROUND

Modem hearing assistance devices, such as hearing aids, typically include digital electronics to enhance the wearer's listening experience. Hearing aids are electronic instruments worn in or around the ear that compensate for hearing losses by specially amplifying sound. Hearing aids use transducer and electro-mechanical components which are connected via wires to the hearing aid circuitry.
Hearing assistance devices include the capability to receive audio from a variety of sources. For example, a hearing assistance device may receive audio or data from a transmitter or streamer of an assistive listening device (ALD). Data such as configuration parameters and telemetry information can be downloaded and/or uploaded to the instruments for the purpose of programming, control and data logging. Audio information can be digitized, packetized and transferred as digital packets to and from the hearing instruments for the purpose of streaming entertainment or other content.
Accordingly, there is a need in the art for improved systems and methods for eavesdropping on a data stream for hearing assistance devices.
EP2456234A1 discloses a binaural hearing system comprising a left-ear hearing device, a right-ear hearing device and an auxiliary device. The auxiliary device has a connected mode wherein it transmits data messages and a disconnected mode wherein it does not transmit data messages and is adapted to enable the connected mode in dependence on receiving beacon messages, to synchronise its transmission of data messages with received beacon messages and to enable the disconnected mode in dependence on not receiving beacon messages.
US2008146152A1 discloses a system in which a device receives a command to switch communication mode into a close-proximity communication mode. This close-proximity communication mode can be used to support the communication of confidential information. The close-proximity communication mode can be used in conjunction with Bluetooth or Wi-Fi protocols, to securely establish security keys to be used in subsequent communications. For example, public-private key (PPK) cryptography is typically vulnerable to active man-in-the-middle attacks, but is highly resistant to passive eavesdropping attacks.

SUMMARY

Disclosed herein, among other things, are systems and methods for eavesdropping on a data stream for hearing assistance devices. One aspect of the present subject matter includes a hearing assistance system for a wearer comprising: a Bluetooth host device having a transmitter configured to send data including one or more encoded audio streams and a data channel having an advertisement that includes frequency information, frequency hop sequences, information for decoding audio streams, and security keys for decoding audio stream information; one or more Bluetooth slave devices identified by the Bluetooth host device configured to actively participate in a bidirectional connection with the host device to aid the transmitter of the Bluetooth host device in deciding which frequencies to use for frequency hopping and in determining which frequencies are being interfered with and should not be included in a channel map; and one or more hearing assistance devices configured to: receive the advertisement from the data channel; use the advertisement to eavesdrop on the connection between the Bluetooth host device and the one or more slave devices; and receive the data including one or more encoded audio streams by way of the eavesdropping without having to transmit back to the Bluetooth host device.
One aspect of the present subject matter includes a method of operation of a hearing assistance system for a wearer, the hearing assistance system comprising a Bluetooth host device including a transmitter, one or more Bluetooth slave devices identified by the Bluetooth host device each including a Bluetooth receiver, and one or more hearing assistance devices, the method comprising: sending, at the Bluetooth host device, data including one or more encoded audio streams, and a data channel having an advertisement that includes frequency information, frequency hop sequences, information for decoding audio streams, and security keys for decoding audio stream information; actively participating, at the one or more Bluetooth slave devices, in a bidirectional connection with the host device to aid the transmitter of the Bluetooth host device in deciding which frequencies to use for frequency hopping and in determining which frequencies are being interfered with and should not be included in a channel map; receiving, at the one or more hearing assistance devices, the advertisement from the data channel; using, at the one or more hearing assistance devices, the advertisement to eavesdrop on the connection between the Bluetooth host device and the one or more slave devices; and receiving, at the one or more hearing assistance devices, the data including one or more encoded audio streams by way of the eavesdropping without having to transmit back to the Bluetooth host device.
This Summary is an overview of some of the teachings of the present application and not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description and appended claims. The scope of the present invention is defined by the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a system including a hearing assistance device adapted to be worn by a wearer and an external Bluetooth host device, according to the present subject matter.
FIG. 2 illustrates a flow diagram of a method of using a Bluetooth receiver, according to the present subject matter.
FIG. 3 illustrates a diagram of a system in which a host device is physically connected to an antenna, according to the present subject matter.
FIG 4 . Illustrates a diagram of a system in which a slave device is collocated with a host device which shares the same antenna, according to the present subject matter.

DETAILED DESCRIPTION

The following detailed description of the present subject matter refers to subject matter in the accompanying drawings which show, by way of illustration, specific aspects end in which the present subject matter may be practiced. The following detailed description is demonstrative and not to be taken in a limiting sense. The scope of the present subject matter is defined by the appended claims.
The present detailed description will discuss hearing assistance devices using the example of hearing aids. Hearing aids are only one type of hearing assistance device. Other hearing assistance devices include, but are not limited to, those in this document. It is understood that their use in the description is intended to demonstrate the present subject matter, but not in a limited or exclusive or exhaustive sense.
Hearing assistance devices include the capability to receive audio from a variety of sources. For example, a hearing assistance device may receive audio or data from a transmitter or streamer from an external device, such as an assistive listening device (ALD). Data such as configuration parameters and telemetry information can be downloaded and/or uploaded to the instruments for the purpose of programming, control and data logging. Audio information can be digitized, packetized and transferred as digital packets to and from the hearing instruments for the purpose of streaming entertainment, carrying on phone conversations, playing announcements, alarms and reminders. In one aspect, music is streamed from an external device to a hearing assistance device using a wireless transmission. Types of wireless transmissions include, but are not limited to, 802.11 (WIFI), Bluetooth or other means of wireless communication with a hearing instrument.
There is a need in the art for improved systems and methods for obtaining an audio stream for hearing assistance devices. Previous solution included proprietary modes of operation to determine the frequency of operation of an assistive listening device, such as using magnetic inductive receivers to obtain information about the frequency of a narrowband FM signal
Disclosed herein, among other things, are systems and methods for eavesdropping on a data stream for hearing assistance devices. One aspect of the present subject matter includes a hearing assistance system for a wearer including a Bluetooth host device having a transmitter configured to send data including one or more encoded audio streams, and a data channel having an advertisement that includes frequency information, frequency hop sequences, information for decoding audio streams, and security keys for decoding audio stream information. The system also includes one or more Bluetooth slave devices identified by the Bluetooth host device. The Bluetooth slave devices are configured to actively participate in a connection with the host device to aid the host transmitter in deciding which frequencies to use for frequency hopping and in determining which frequencies are being interfered with and should not be included in a channel map.
The present subject matter allows devices to receive a data stream when they are determined to be in close proximity and direction of an audio field, as determined by a host device. An example of this determination can be found in copending, commonly assigned, U.S. Patent Application Serial No. 13/738,775 , entitled "SYSTEM AND METHOD FOR OBTAINING AN AUDIO STREAM BASED ON PROXIMITY AND DIRECTION".
One aspect of the present subject matter provides a system and method for multiple users to receive a data stream for audio without being in a connection with the device that is hosting the information. This subject matter provides a significant deviation to any of the Bluetooth Core Specifications, which require the devices communicating to be in a two-way connection. Bluetooth is a widely used standard that has not yet included multi-cast and broadcast modes of operation. The present subject matter allows devices to receive a data stream without being in a one to one connection with the host device.
In various aspects, the host device can be a standard Bluetooth radio type device using adaptive frequency hopping techniques while allowing other users to participate in receiving the information. The present subject matter allows a standard based approach and a single physical layer in the hearing instrument to receive a broadcast communication over a long range without having to transmit back to a host device.
In previous solutions, if the hearing instrument would need to communicate with the host device over a long range, it would need a large antenna and a much larger energy source than is typically available in a hearing instrument. One example includes multiplex cinema where multiple audio sources may be available.
One aspect of the present subject matter includes methods of obtaining necessary parameters to participate in a Bluetooth audio transmission. Once a hearing aid wearer is identified to be in proximity and moving in the direction of an audio field, a device user (such as a hearing aid wearer) is given the Bluetooth access address, frequency, hop sequence, security keys, cipher codes, etc., necessary to receive the signal being sent from within the area of the audio field. In addition, the user will be given the necessary information to begin hopping in sequence with the host device. Since the devices are worn on each ear, in an aspect, the user also obtains the necessary information to listen to both a right and left channel simultaneously on a right and left worn hearing instrument for the purpose of stereo reception. Any delay between left and right channels is also sent to the device user to aid in the synchronization of the rendering of each channel to allow for synchronized stereo listening. According to various aspects, channel map information is advertised by the master device periodically, which allows devices that have lost synchronization to reacquire the signals.
According to various aspects, in order to facilitate adaptive frequency hopping, the host device uses passive listening between transmissions to determine if the channels being used for hopping should be modified to avoid interference. In another aspect, another Bluetooth transceiver (or transceivers, one for each audio channel) are used and are in communication with the host device to determine through acknowledgements whether the channels being used should be modified due to apparent interference. The device receiving the signals uses the same addresses as the devices used to determine "good" channels for communication within an auditorium, for example. The "master receivers" can be either collocated with the host transceiver device(s) or be remotely located to better simulated devices located throughout the sound area.
In one aspect, the present subject matter uses the same physical layer within the hearing instrument for transmitting and receiving signals wirelessly, such as Bluetooth or Bluetooth low energy. The present subject matter utilizes the proximity sensor profile within Bluetooth low energy. In various aspects, this can be used with a security key (digital rights management) to make the system more robust.
FIG. 1 illustrates a block diagram of a system 100, according to the present subject matter. The illustrated system 100 shows an external Bluetooth device 110 in wireless communication with a hearing assistance device 150. In various aspects, the hearing assistance device 150 includes a first housing 121, an acoustic receiver or speaker 102, positioned in or about the ear canal 130 of a wearer and conductors 123 coupling the receiver 102 to the first housing 121 and the electronics enclosed therein. The electronics enclosed in the first housing 121 includes a microphone 104, hearing assistance electronics 105, a wireless communication receiver 106 and an antenna 107. In various aspects, the hearing assistance electronics 105 includes at least one processor and memory components. The memory components store program instructions for the at least one processor. The program instructions include functions allowing the processor and other components to process audio received by the microphone 104 and transmit processed audio signals to the speaker 102. The speaker or receiver emits the processed audio signal as sound in the user's ear canal. In various aspects, the hearing assistance electronics includes functionality to amplify, filter, limit, condition or a combination thereof, the sounds received using the microphone 104.
In the illustrated embodiment of FIG. 1 , the wireless communications receiver 106 includes a Bluetooth receiver connected to the hearing assistance electronics 105 and the conductors 123 connect the hearing assistance electronics 105 and the speaker 102. In various aspects, the external device 110 includes a Bluetooth streaming audio device such as an ALD. The external device 110 includes an antenna 116 connected to processing electronics 114 that include a transmitter. In various aspects, the external device 110 includes one or more sensors 112 or sensing components connected to the processing electronics 114 to sense proximity and direction of the hearing assistance device 150.
FIG. 2 illustrates a flow diagram of a method of using a Bluetooth receiver, according to the present subject matter. The present subject matter includes a method 200 of using a Bluetooth receiver including, at 202, receiving a data stream from a Bluetooth host device including a transmitter, the data stream including advertisements, frequency hop code sequences and security codes. The method also includes passively listening on potential channels for interference to decide which frequencies can be used for frequency hopping, at 204.
FIG. 3 illustrates an embodiment of the present subject matter in which a host transceiver device 302 is physically connected to an antenna 303. The host device 302 is in two-way communication with slave device 305 which is connected to antenna 304. The communication between host device 302 and slave device 305 is maintained using adaptive frequency hopping techniques to avoid congestion and interference. Hearing assistance devices are worn by wearers 306 and are configured to eavesdrop on the connection between host device 302 and slave device 305. In various aspects, stereo channels can be sent together or as separate radio frequency (RF) streams of information to the devices in sound field 301. A separate advertising channel is used to inform the listeners of the current frequency hopping and timing sequence of the usable RF channels carrying the audio information for both left and right audio channels. The advertising channels contain all necessary information to demodulate and decode the audio information. Such information includes, but is not limited to: a frequency hopping channel map, timing information, spreading codes, security keys and modulation type.
FIG. 4 illustrates an alternate aspect where the slave device 402 is collocated with the host device 407 which shares the same antenna 404. In this aspect, the slave device and host device are connected to antenna 404 through a combiner 403. The signal from the host device to the slave device is attenuated by the isolation of the splitter and through attenuator 405 such that the signal is near the limit of sensitivity for slave device 402. In various aspects, interference from other devices using the spectrum is picked up by antenna 404 so that slave device 402 picks up both the wanted signal from the host device and any interference. As in FIG. 3 , the hearing instrument wearers are eavesdropping on the host transmissions to the slave device 402. The stereo channels can be sent together or as separate RF streams of information to the devices in sound field 401. According to various aspects, a separate advertising channel is used to inform the listeners of the current frequency hopping and timing sequence of the usable RF channels carrying the audio information for both left and right audio channels. The advertising channels can contain all necessary information to demodulate and decode the audio information. Such information includes, but is not limited to: a frequency hopping channel map, timing information, spreading codes, security keys and modulation type.
Various aspects of the present subject matter support wireless communications with a hearing assistance device. In various aspects the wireless communications can include standard or nonstandard communications. Some examples of standard wireless communications include link protocols including, but not limited to, Bluetooth™, IEEE 802.11(wireless LANs), 802.15 (WPANs), 802.16 (WiMAX), cellular protocols including, but not limited to CDMA and GSM, ZigBee, and ultra-wideband (UWB) technologies. Such protocols support radio frequency communications and some support infrared communications. Although the present system is demonstrated as a radio system, it is possible that other forms of wireless communications can be used such as ultrasonic, optical, infrared, and others. It is understood that the standards which can be used include past and present standards. It is also contemplated that future versions of these standards and new future standards may be employed without departing from the scope of the present subject matter.
The wireless communications support a connection from other devices. Such connections include, but are not limited to, one or more mono or stereo connections or digital connections having link protocols including, but not limited to 802.3 (Ethernet), 802.4, 802.5, USB, SPI, PCM, ATM, Fibre-channel, Firewire or 1394, InfiniBand, or a native streaming interface. In various aspects, such connections include all past and present link protocols. It is also contemplated that future versions of these protocols and new future standards may be employed without departing from the scope of the present subject matter.
It is understood that variations in communications protocols, antenna configurations, and combinations of components may be employed without departing from the scope of the present subject matter. Hearing assistance devices typically include an enclosure or housing, a microphone, hearing assistance device electronics including processing electronics, and a speaker or receiver. It is understood that in various aspects the microphone is optional. Antenna configurations may vary and may be included within an enclosure for the electronics or be external to an enclosure for the electronics. Thus, the examples set forth herein are intended to be demonstrative and not a limiting or exhaustive depiction of variations.
It is further understood that any hearing assistance device may be used without departing from the scope and the devices depicted in the figures are intended to demonstrate the subject matter, but not in a limited, exhaustive, or exclusive sense. It is also understood that the present subject matter can be used with a device designed for use in the right ear or the left ear or both ears of the wearer.
It is understood that the hearing aids referenced in this patent application include a processor. The processor may be a digital signal processor (DSP), microprocessor, microcontroller, other digital logic, or combinations thereof. The processing of signals referenced in this application can be performed using the processor. Processing may be done in the digital domain, the analog domain, or combinations thereof. Processing may be done using subband processing techniques. Processing may be done with frequency domain or time domain approaches. Some processing may involve both frequency and time domain aspects. For brevity, in some examples drawings may omit certain blocks that perform frequency synthesis, frequency analysis, analog-to-digital conversion, digital-to-analog conversion, amplification, audio decoding, and certain types of filtering and processing. In various aspects the processor is adapted to perform instructions stored in memory which may or may not be explicitly shown. Various types of memory may be used, including volatile and nonvolatile forms of memory. In various aspects, instructions are performed by the processor to perform a number of signal processing tasks. In such aspects, analog components are in communication with the processor to perform signal tasks, such as microphone reception, or receiver sound embodiments (i.e., in applications where such transducers are used). In various aspects, different realizations of the block diagrams, circuits, and processes set forth herein may occur without departing from the scope of the present subject matter.
The present subject matter is demonstrated for hearing assistance devices, including hearing aids, including but not limited to, behind-the-ear (BTE), in-the-ear (ITE), in-the-canal (ITC), receiver-in-canal (RIC), or completely-in-the-canal (CIC) type hearing aids. It is understood that behind-the-ear type hearing aids may include devices that reside substantially behind the ear or over the ear. Such devices may include hearing aids with receivers associated with the electronics portion of the behind-the-ear device, or hearing aids of the type having receivers in the ear canal of the user, including but not limited to receiver-in-canal (RIC) or receiver-in-the-ear (RITE) designs. The present subject matter can also be used in hearing assistance devices generally, such as cochlear implant type hearing devices and such as deep insertion devices having a transducer, such as a receiver or microphone, whether custom fitted, standard, open fitted or occlusive fitted. It is understood that other hearing assistance devices not expressly stated herein may be used in conjunction with the present subject matter.
This application is intended to cover adaptations or variations of the present subject matter. It is to be understood that the above description is intended to be illustrative, and not restrictive. The scope of the present subject matter should be determined with reference to the appended claims.

Claims

A hearing assistance system for a wearer, comprising:
a Bluetooth host device (302, 407) including a transmitter configured to send data including one or more encoded audio streams and a data channel having an advertisement that includes frequency information, frequency hop sequences, information for decoding audio streams, and security keys for decoding audio stream information;

one or more Bluetooth slave devices (305, 402), identified by the Bluetooth host device, the Bluetooth slave devices (305, 402) including a Bluetooth receiver and configured to actively participate in a bidirectional connection with the host device (302, 407) to aid the transmitter of the Bluetooth host device (302, 407) in deciding which frequencies to use for frequency hopping and in determining which frequencies are being interfered with and should not be included in a channel map; and

one or more hearing assistance devices configured to:
receive the advertisement from the data channel;

use the advertisement to eavesdrop on the connection between the Bluetooth host device (302, 407) and the one or more slave devices (305, 402); and

receive the data including one or more encoded audio streams by way of the eavesdropping without having to transmit back to the Bluetooth host device (302, 407).
The system of claim 1, wherein the one or more Bluetooth slave devices (305, 402) are adapted to receive and decode the audio streams or other data information after receiving the advertised frequency information and other information used to synchronize and decode receptions from the host device (302, 407).
The system of claim 1 or claim 2, wherein at least one of the Bluetooth slave devices (305, 402) is included in a hearing assistance device.
The system of claim 3, wherein the hearing assistance device includes a hearing aid.
The system of claim 3, wherein the hearing assistance device includes a cochlear implant.
A method of operation of a hearing assistance system for a wearer, the hearing assistance system comprising a Bluetooth host device (302, 407) including a transmitter, one or more Bluetooth slave devices (305, 402) identified by the Bluetooth host device each including a Bluetooth receiver, and one or more hearing assistance devices, the method comprising:
sending, at the Bluetooth host device (302, 407), data including one or more encoded audio streams, and a data channel having an advertisement that includes frequency information, frequency hop sequences, information for decoding audio streams, and security keys for decoding audio stream information;

actively participating, at the one or more Bluetooth slave devices (305, 402), in a bidirectional connection with the host device (302, 407) to aid the transmitter of the Bluetooth host device (302, 407) in deciding which frequencies to use for frequency hopping and in determining which frequencies are being interfered with and should not be included in a channel map;

receiving, at the one or more hearing assistance devices, the advertisement from the data channel;

using, at the one or more hearing assistance devices, the advertisement to eavesdrop on the connection between the Bluetooth host device (302, 407) and the one or more slave devices (305, 402); and

receiving, at the one or more hearing assistance devices, the data including one or more encoded audio streams by way of the eavesdropping without having to transmit back to the Bluetooth host device (302, 407).
The method of claim 6, wherein actively participating in the connection with the Bluetooth host device (302, 407) comprises receiving a data stream from the Bluetooth host device (302, 407), including receiving the channel map.
The method of claim 7, wherein receiving the channel map from the Bluetooth host device (302, 407) includes receiving the channel map periodically to reacquire signals if the Bluetooth receiver has lost synchronization with the Bluetooth host device (302, 407).
The method of any of claim 7 through claim 8, wherein receiving a data stream from the Bluetooth host device includes receiving an address to listen.
The method of any of claim 7 through claim 8, wherein receiving a data stream from the Bluetooth host device (302, 407) includes receiving cipher codes.
The method of any of claim 7 through claim 8, wherein receiving a data stream from the Bluetooth host device (302, 407) includes receiving information to begin hopping in sequence with the Bluetooth host device (302, 407).
The method of any of claim 7 through claim 8, wherein receiving a data stream from the Bluetooth host device (302, 407) includes receiving information to listen to both a right and left channel.
The method of claim 12, wherein the right and left channel include data for left and right worn hearing assistance devices.
The method of claim 12 or claim 13, further comprising receiving data indicative of delay between the right and left channels.
The method of claim 14, further comprising using the data indicative of delay to synchronize a rendering of each channel to allow for synchronized stereo listening.