WO2020077389A1 - "a method and system for determining an orientation of a user" - Google Patents

"a method and system for determining an orientation of a user" Download PDF

Info

Publication number
WO2020077389A1
WO2020077389A1 PCT/AU2019/050988 AU2019050988W WO2020077389A1 WO 2020077389 A1 WO2020077389 A1 WO 2020077389A1 AU 2019050988 W AU2019050988 W AU 2019050988W WO 2020077389 A1 WO2020077389 A1 WO 2020077389A1
Authority
WO
WIPO (PCT)
Prior art keywords
transducer
acoustic signal
electronic device
user
time
Prior art date
Application number
PCT/AU2019/050988
Other languages
French (fr)
Inventor
Paul ZRNA
Tong Chen
Original Assignee
Idearlabs Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2018903881A external-priority patent/AU2018903881A0/en
Application filed by Idearlabs Pty Ltd filed Critical Idearlabs Pty Ltd
Publication of WO2020077389A1 publication Critical patent/WO2020077389A1/en
Priority to AU2021101916A priority Critical patent/AU2021101916A4/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/802Systems for determining direction or deviation from predetermined direction
    • G01S3/8027By vectorial composition of signals received by plural, differently-oriented transducers
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S11/00Systems for determining distance or velocity not using reflection or reradiation
    • G01S11/14Systems for determining distance or velocity not using reflection or reradiation using ultrasonic, sonic, or infrasonic waves
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/801Details
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/802Systems for determining direction or deviation from predetermined direction
    • G01S3/808Systems for determining direction or deviation from predetermined direction using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/802Systems for determining direction or deviation from predetermined direction
    • G01S3/808Systems for determining direction or deviation from predetermined direction using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
    • G01S3/8083Systems for determining direction or deviation from predetermined direction using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems determining direction of source
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012Head tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/165Management of the audio stream, e.g. setting of volume, audio stream path
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2400/00Loudspeakers
    • H04R2400/01Transducers used as a loudspeaker to generate sound aswell as a microphone to detect sound

Definitions

  • the present disclosure relates to a method for determining an orientation of an anatomical feature of a user in relation to an electronic device.
  • the anatomical feature may, in some examples, be a head of a user.
  • Knowledge of the orientation of a user’s anatomical feature may be useful to allow a device to focus audible, visual or other information based on the orientation.
  • knowledge of the user’s head orientation may assist in optimising the delivery of audio to the user based on the orientation.
  • a method for determining an orientation of an anatomical feature of a user in relation to an electronic device wherein a wearable device is located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer, and the electronic device comprising a third transducer, the method comprising: transmitting and receiving a first acoustic signal between the first transducer and the third transducer; transmitting and receiving a second acoustic signal between the second transducer and the third transducer; determining a first received time associated with the first acoustic signal; determining a second received time associated with the second acoustic signal; and determining the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.
  • the first received time may indicate a time when the first acoustic signal is received by the third transducer and the second received time may indicate a time when the second acoustic signal is received by the third transducer.
  • determining the orientation of the anatomical feature of the user may be further based on a first time difference and a second time difference.
  • the first time difference may be based on a difference between a first time reference and the first received time, wherein the first time reference indicates a time that the first acoustic signal was transmitted.
  • the second time difference may be based on a difference between a second time reference and the second received time, wherein the second time reference indicates a time that the second acoustic signal was transmitted.
  • the first transducer may transmit the first acoustic signal.
  • the second transducer may transmit the second acoustic signal.
  • the third transducer may comprise two or more microphones.
  • the method may further comprise: receiving the first and second acoustic signal at the two or more microphones; and processing the first and second acoustic signals received at the two or more microphones to determine a first location and a second location associated with the first and second transducers respectively.
  • the at least two microphones may form at least one microphone cluster in the electronic device.
  • the first acoustic signal and the second acoustic signal may comprise an ultrasonic signal.
  • the ultrasonic signal may act as a carrier wave.
  • the first acoustic signal and the second acoustic signal may be acoustic leak signals.
  • the anatomical feature of the user may be a head.
  • the first transducer may be associated with a first ear of the user and the second transducer may be associated with a second ear of the user.
  • the electronic device may be located on the user.
  • the first transducer and the second transducer of the wearable device may be located approximately symmetrical about a central axis associated with the user.
  • the electronic device may be approximately located on the central axis.
  • a system for determining an orientation of an anatomical feature of a user in relation to an electronic device comprising: a wearable device located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer; the electronic device comprising a third transducer; wherein a first acoustic signal is transmitted and received between the first transducer and the third transducer; and wherein a second acoustic signal is transmitted and received between the second transducer and the third transducer; a processor configured to: determine a first received time associated with the first acoustic signal; determine a second received time associated with the second acoustic signal; and determine the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.
  • the third transducer may comprise two or more microphones.
  • the at least two microphones may form at least one microphone cluster in the electronic device.
  • FIG. 1 illustrates a schematic diagram of an example system for determining an orientation of an anatomical feature of a user in relation to an electronic device
  • Fig. 2 illustrates a method of determining an orientation of an anatomical feature of a user in relation to an electronic device
  • FIG. 3 illustrates an example configuration of a wearable device and electronic device
  • Fig. 4 illustrates an anatomical feature of the head of a user turned in a left direction
  • Fig. 5 illustrates an anatomical feature of the head of a user turned in a right direction
  • Fig. 6 illustrates an example of a linear microphone array in an electronic device
  • FIG. 7 illustrates an example of microphone clusters in an electronic device
  • Fig. 8 illustrates an example processing device
  • Fig. 9 illustrates results of simulations
  • Fig. 10 illustrates an example hardware implementation.
  • FIG. 1 illustrates a system 100 for determining an orientation of an anatomical feature of a user 110 in relation to an electronic device 120.
  • the system 100 further comprises a wearable device 130 that is located relative to the anatomical feature of the user 110.
  • the wearable device 130 comprises a first transducer 140 and a second transducer 150.
  • the first transducer 140 and second transducer 150 may be speakers. In other examples the first transducer 140 and second transducer 150 are microphones. The first transducer 140 and second transducer 150 may be transducers capable of acting as both speakers and microphones. In some example the wearable device 130 may comprise headphones.
  • the electronic device 120 comprises a third transducer 160.
  • the third transducer 160 may be a microphone.
  • the third transducer 160 may be a speaker. In other examples the third transducer 160 may be capable of acting as both a speaker and microphone.
  • the electronic device 120 may comprise additional transducers, such as two or more microphones or speakers.
  • a first acoustic signal is transmitted and received between the first transducer 140 and the third transducer 160 (or the additional transducers), and a second acoustic signal is transmitted and received between the second transducer 150 and the third transducer 160 (or additional transducers).
  • the first acoustic signal and second acoustic signal comprise an ultrasonic signal.
  • the system 100 also comprises a processor 170 configured to determine a first received time associated with the first acoustic signal and a second received time associated with the second acoustic signal.
  • the first received time may indicate a time when the first acoustic signal is received by the electronic device 120 and the second received time may indicate a time when the second acoustic signal is received by the electronic device 120.
  • the processor may be provided at the electronic device 120.
  • the method 200 as illustrated by Fig. 2 includes transmitting and receiving 210 a first acoustic signal between the first transducer 140 and the third transducer 160.
  • the method 200 further comprises transmitting and receiving 220 a second acoustic signal between the second transducer 150 and the third transducer 160.
  • the first acoustic signal may be transmitted from the first transducer 140
  • the second acoustic signal may be transmitted from the second transducer 150.
  • the method 200 further comprises determining 230 a first received time associated with the first acoustic signal.
  • the method 200 further comprises
  • the method 200 further comprises determining 250 the orientation of the anatomical feature of the user 110 in relation to the electronic device 120 based on the first and second received times.
  • the method 200 determines an orientation of an anatomical feature of a user 110 in relation to an electronic device 120.
  • a wearable device 130 is located relative to the anatomical feature of the user 110.
  • the wearable device 130 may comprise headphones.
  • the anatomical feature of the user 110 is a head of the user. In this way, the method 200 determines the orientation of the user’s head. More specifically the method 200 determines the orientation of the head in an azimuth direction in relation to the electronic device 120.
  • the first transducer 140 and second transducer 150 of the wearable device 130 may be associated with a first ear of the user and second ear of the user respectively. In one example the first transducer 140 and the second transducer 150 of the wearable device 130 are located approximately symmetrical about a central axis 310 associated with the user 110. This is illustrated in Fig. 3. [0040]
  • the electronic device 120 may be located on the user 110, for example centrally on the body 110. The electronic device 120 may be approximately located on the central axis 310. In one example the electronic device 120 is located in a chest area of the user 110. The electronic device 120 may be located approximately 70 mm down from the base of a neck of the user 110.
  • the electronic device 120 may be located in a range of 70 mm to 300 mm from the base of the neck of the user.
  • the electronic device 120 may be located at other distances from the base of the neck.
  • the electronic device 120 may extend from the user’s body by approximately 50 mm. In other examples the electronic device 120 may extend from the user’s body by more than 50 mm.
  • the electronic device 120 may sit flush against the body. In other examples the electronic device 120 may be located in other places on the body.
  • the electronic device may comprise a hearing aid, mobile phone, audio player, video player, gaming device or radio.
  • the electronic device 120 may be worn on a lanyard around the neck of the user 110. In other examples the electronic device 120 may be attached to clothing of a user.
  • the method 200 includes transmitting and receiving 210 a first acoustic signal between the first transducer 140 and the third transducer 160.
  • the first transducer 140 of the wearable device 130 transmits the first acoustic signal.
  • the third transducer 160 receives the first acoustic signal from the first transducer 140.
  • the third transducer 160 (and/or the additional transducers of the electronic device) transmits the first acoustic signal and the first transducer 140 receives the first acoustic signal.
  • the electronic device 120 may comprise additional transducers, such as two or more microphones. In this way, the first acoustic signal may be received at the two or more microphones of the electronic device 120. In some examples, the at least two microphones of the electronic device 120 may form at least one microphone cluster 120.
  • the method 200 also includes transmitting and receiving 220 a second acoustic signal between the second transducer 150 and the third transducer 160.
  • the second transducer 150 of the wearable device 130 may transmit the second acoustic signal.
  • the third transducer 165 and/or additional transducers of the electronic device 120 receives the second acoustic signal from the second transducer 150.
  • the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal and the second transducer 150 receives the second acoustic signal.
  • the second acoustic signal may be received at the two or more microphones of the electronic device 120.
  • the first acoustic signal and second acoustic signal may be transmitted at the same instant from the first transducer 140 and the second transducer 150 respectively.
  • the first acoustic signal and second acoustic signal may be transmitted at different times from the first transducer 140 and the second transducer 150
  • first acoustic signal and second acoustic signal may be transmitted at the same instant from the electronic device 120.
  • first acoustic signal and second acoustic signal may be transmitted at different times from the electronic device 120.
  • the first acoustic signal and second acoustic signal may comprise an audible acoustic signal, such as music or audio content.
  • the first acoustic signal and second acoustic signal may be acoustic leak (or leakage) signals.
  • the first acoustic signal and second acoustic signal are not purposely emitted, but rather, are leakages from acoustic signals associated with the first transducer 140 and second transducer 150.
  • the first acoustic signal and second acoustic signal may leak out from the first transducer 140 and second transducer 150 respectively and be detected by the third transducer 160.
  • the third transducer 160 and/or additional transducers of the electronic device 120 may detect the first acoustic signal and second acoustic signal as acoustic leak signals.
  • the first acoustic signal and second acoustic signal may comprise a non-audible acoustic signal.
  • the first acoustic signal and second acoustic signal may have a frequency of less than 20 Hz or greater than 20 kHz.
  • the first acoustic signal and second acoustic signal comprise infrasonic signals.
  • the first acoustic signal and second acoustic signal are ultrasonic signals with a frequency greater than 20 kHz.
  • the first acoustic signal and second acoustic signal are frequency modulated signals, and an ultrasonic signal may act as a carrier wave for the frequency modulated signals.
  • the frequency of the carrier wave may be above 132 kHz.
  • the first acoustic signal and/or the second acoustic signal may be a speech signal in the spectrum from 20 Hz to 20 kHz. That is, the first acoustic signal and/or second acoustic signal may be a baseband signal (prior to modulation). In this way, the baseband signal may be a periodic wave such as a sinusoidal wave. In one example, the period of the baseband signal (that is, the first acoustic signal or second acoustic signal prior to modulation) may be greater than twice the time taken for acoustic sound to travel between ears of the user 110. That is, denoting the distance between ears of the user 110 as d and the speed of sound as v, the period T of the baseband signal (before modulation) may be computed as follows: d
  • the first acoustic signal and second acoustic signal are generated by modulating an ultrasound carrier wave by a discrete (baseband) signal.
  • the ultrasound carrier wave is modulated by symbol coded messages.
  • the ultrasound carrier wave is modulated by orthogonal symbols. The frequency for the symbol coded messages may fall between 20.5 kHz and 21 kHz. A minimum of three symbols may be transmitted.
  • An advantage of using an ultrasonic signal as the carrier wave is that ultrasound are not audible to the human ear at frequencies above 20 kHz airborne.
  • the safety guideline for exposure limits on airborne ultrasound according to the Occupational Safety and Health Administration (OSHA) is 105 dB.
  • the exposure limit increases to 115 dB. This exposure limit can be increased if there is no possibility that the ultrasound signal can be coupled with the human body. This means that for an ultrasound signal with a frequency over 132 kHz, the exposure limit is greater than 115 dB.
  • a further advantage of using an ultrasound signal is that ultrasound is more directional than signals within the human audible frequency range. This means that ultrasound signals may have a better defined flight path and spatial filtering techniques may be utilised to minimise multi-path propagation.
  • the electronic device 120 may be located on the user 110 about a central axis 310, with the first transducer 140 and second transducer 150 located approximately symmetrical about the central axis 310. This is illustrated in Fig. 3.
  • the first transducer 140 and second transducer 150 may receive the first acoustic signal and second acoustic signal at approximately the same time.
  • the first acoustic signal and second acoustic signal may require an
  • first transducer 140 and second transducer 150 may receive the first acoustic signal and second acoustic signal within a delay of each other.
  • the electronic device 120 receives the first acoustic signal and second acoustic signal at approximately the same time.
  • the electronic device 120 may receive the first acoustic signal and second acoustic signal within a delay of each other.
  • the time taken for the first acoustic signal and second acoustic signal to be transmitted and received between the transducers 140, 150 and electronic device 120 may not be approximately equal. This means that there may be a delay between the times that the electronic device 120 receives the first acoustic signal and second acoustic signals.
  • the electronic device 120 transmits the first acoustic signal and second acoustic signal there may be a delay between the times that the first transducer 140 and second transducer 150 receive the first acoustic signal and second acoustic signal respectively. This may occur when the orientation of the user’s head changes (for example, when the user turns his head left or right).
  • the distance 420 from the first transducer 140 to the electronic device 120 is different to the distance 430 from the second transducer 150 to the electronic device 120.
  • the distance 420 may be less than the distance 430.
  • the time for the first acoustic signal to transmit and receive between the first transducer 140 and the third transducer 160 (and/or additional transducers of the electronic device 120) may be shorter than the time for the second acoustic signal to transmit and receive between the second transducer 150 and the third transducer 160 (and/or additional transducers of the electronic device 120).
  • FIG. 5 A similar situation is described in Fig. 5, where the head of the user 110 is turned to the right.
  • the distance 530 from the second transducer 150 to the electronic device 120 may be less than the distance 520 from the first transducer 140 to the electronic device 120.
  • the time for the first acoustic signal to transmit and receive between the second transducer 150 and the third transducer 160 (and/or additional transducers of the electronic device 120) may be shorter than the time for the first acoustic signal to transmit and receive between the first transducer 140 and the third transducer 160 (and/or additional transducers of the electronic device 120).
  • the method 200 also includes determining 230 a first received time associated with the first acoustic signal.
  • the first transducer 140 transmits the first acoustic signal and the first received time indicates a time when the first acoustic signal is received by the third transducer 160 (and/or additional transducers) of the electronic device 120.
  • the first transducer 140 transmits the first acoustic signal in accordance with a clock signal.
  • the clock signal may be associated with the first transducer 140, the wearable device 130 or another device.
  • the third transducer 160 and/or additional transducers of the electronic device 120 transmits the first acoustic signal and the first received time indicates a time when the first acoustic signal is received by the first transducer 140.
  • the third transducer 160 and/or additional transducers of the electronic device 120 transmits the first acoustic signal in accordance with a clock signal.
  • the clock signal may be associated with the third transducer 160 and/or additional transducers of the electronic device 120, the electronic device 120 or another device.
  • the method 200 also includes determining 240 a second received time associated with the second acoustic signal.
  • the second transducer 150 transmits the second acoustic signal and the second received time indicates a time when the second acoustic signal is received by the third transducer 160 and/or additional transducers of the electronic device 120.
  • the second transducer 150 transmits the second acoustic signal in accordance with a clock signal.
  • the clock signal may be associated with the second transducer 140, the wearable device 130 or another device.
  • the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal and the second received time indicates a time when the second acoustic signal is received by the second transducer 150.
  • the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal in accordance with a clock signal.
  • the clock signal may be associated with the third transducer 160 and/or additional transducers of the electronic device 120, the electronic device 120 or another device.
  • Determining the orientation of the anatomical feature of the user 250 also includes determining 250 the orientation of the anatomical feature of the user in relation to the electronic device 120 based on the first and second received times.
  • determining 250 the orientation of the anatomical feature of the user 110 is based on the delay in the first acoustic signal and second acoustic signal being received.
  • the first acoustic signal and second acoustic signal may be received by the third transducer 160 (and/or additional transducers of the electronic device 120).
  • the first acoustic signal and second acoustic signal may be received by the first transducer 140 and second transducer 150 respectively.
  • Determining 250 the orientation of the anatomical feature of the user may be based on a first time difference and a second time difference.
  • the first time difference may be based on a difference between a first time reference and the first received time. Examples of the first received time are described above with respect to step 230 of method 200.
  • the first time reference may indicate a time that the first acoustic signal is transmitted by the first transducer 140 or third transducer 160 (or additional transducers of the electronic device 120).
  • the first time reference may indicate a common time for transmission of the first acoustic signal and another acoustic signal, such as the second acoustic signal.
  • the first time reference may indicate another event associated with the first acoustic signal.
  • the second time difference may be based on a difference between a second time reference and the second received time. Examples of the second received time are described above with respect to step 240 of method 200.
  • the second time difference may indicate a time that the second acoustic signal is transmitted.
  • the second time reference may indicate a common time for transmission as described above. In other examples the second time reference may indicate another event.
  • determining 250 the orientation of the anatomical feature of the user 110 comprises determining the orientation of the user’s head in the azimuth direction.
  • determining 250 the orientation of the user’s head may be based on the first time difference and the second time difference. For instance, if the first time difference is greater than the second time difference, determining 250 the orientation may comprise determining that the user’s head is oriented in a left direction as illustrated in Fig. 4. If the first time difference is less than the second time difference, determining 250 the orientation may comprise determining that the user’s head is oriented in a right direction as illustrated in Fig. 5.
  • determining 250 the orientation may comprise determining that the user’s head is oriented approximately on the central axis 310. In other examples the orientation may be determined as on the central axis 310 if the values of the first time difference and second time difference are within a threshold of each other.
  • the electronic device 120 comprises the third transducer.
  • the electronic device 120 may also comprise additional transducers, such as two or more microphones.
  • the two or more microphones may be located in spatially dispersed locations of the electronic device 120.
  • the two or more microphones may form a linear microphone array 600.
  • An example of the linear microphone array 600 is illustrated in Fig. 6.
  • the microphones 610, 620, 630, 640 in the linear microphone array 600 may be spaced to avoid spatial aliasing. This means that the microphones 610, 620, 630,
  • 640 may be spaced so that a distance 660 between each microphone is less than half the wavelength of the first acoustic signal and the second acoustic signal. For instance, if the frequency of the first acoustic signal and the second acoustic signal is 132 kHz the corresponding wavelength will be around 2.5 mm. This means that the microphones 610, 620, 630, 640 in the array 600 may be located with a distance of less than 1.25 mm between each microphone.
  • the microphones 610, 620, 630, 640 in the array 600 are located at a distance greater than half the wavelength of the first acoustic signal and the second acoustic signal.
  • the at least two microphones may form at least one microphone cluster in the electronic device 120. There may be more than one microphone cluster in the electronic device 120. For example there may be two microphones per microphone cluster. The two microphones may be spaced at a distance from each other to avoid spatial aliasing.
  • the microphones 730, 740, 750, 760, 770, 780 of the microphone cluster 710, 720 may be spaced at a distance from each other to avoid spatial aliasing. That is, the microphones 730, 740, 750, 760, 770, 780 are located at a distance 790 less than half the wavelength of the first acoustic signal and the second acoustic signal.
  • each microphone cluster 710, 720 is placed as far apart from each other as allowable by the dimensions of the third transducer 160 and electronic device 120. This is so that the received acoustic signals at the microphones 730, 740, 750, 760, 770, 780 are sampled with less correlation.
  • the method 200 further comprises processing the signals received at the two or more microphones of the electronic device 120 to determine a first location and a second location, where the first location and second location are associated with the first and second transducers respectively.
  • Determining the first location may comprise processing the signals received at the array 600 or clusters 710, 720 to determine the direction of arrival (DOA). This may be based on a time delay of arrival (TDOA) estimation. This may further be based on the delays measured from transmission of the first acoustic signal and second acoustic signal between the electronic device 120 and the transducers 140, 150. In other examples localisation may be based on other acoustic signals transmitted between the electronic device 120 and transducers 140, 150.
  • DOA direction of arrival
  • TDOA time delay of arrival
  • determining the first location comprises performing beamforming on the signals received at the microphones in the array 600 or clusters 710, 210.
  • beamforming may comprise delay and sum beamformer, or minimum variance distortionless response beamformer.
  • MUSIC Multiple Signal Classification
  • ESPRIT Estimation of Signal Parameters via Rotational Invariance Technique
  • DUET Degenerate Unmixing Estimation Technique
  • the system 100 comprises a processor 170.
  • Fig. 8 illustrates an example of a processing device.
  • the processing device includes a processor 810, a memory 820 and an interface device 840 that communicate with each other via a bus 830.
  • the memory 820 stores a computer software program comprising machine-readable instructions 824 and data 822 for implementing the method 200 described above, and the processor 810 performs the instructions from the memory 820 to implement the method 200.
  • the interface device 840 may include a communications module that facilitates communication with a communications network, and in some examples, with the user interface 840 and peripherals such as data store 822. It should be noted that although the processing device may be an independent network element, the processing device may also be part of another network element. Further, some functions performed by the processing device may be distributed between multiple network elements.
  • the model used in developing algorithms is derived using estimated average dimensions. These dimensions are used as guideline for evaluating the model and brings about intuition, whereas the model itself does not rely on these dimension measurements.
  • a rigid body model is assumed with a neck as a stick leaning naturally forward at 15 degrees with the tip of the neck connected with a free rotating vertical rigid stick, which in turn connects with a free rotating horizontal stick with ends denoting ear positions.
  • the width of the head is assumed to be 155 mm, length of the neck 110 mm, vertical displacement from the tip of the neck to the ear canal 30 mm.
  • head is free to rotate sideways by up to 45 degrees pivoting on the tip of the neck in all directions, this is represented by the vertical stick pivoting on top of the neck stick.
  • the head is also allowed to rotate left and right by 80 degrees each side, which is represented by allowing the horizontal ear stick model to rotate.
  • Neck movement is a lot more restricted, allowing 10 degrees backward, 50 degrees frontal, and 45 degrees sideways, with pivot angle allowance interpolated for directions in between.
  • the proposed electronic device 120 is located at 70 mm down from the base of the neck and extends forward by 50 mm. Such a location is used as a general guideline only, the actual location of the microphone plane can be of any orientation as long as its orientation does not change drastically during the life of operation. For example, system can have a number of microphones on the receiving device in an orientation A. If the device was to assume an orientation B after wearer calibration, system needs to be re-calibrated before measurement can be made with reasonable accuracy. For the sake of building a clear model, we assume that horizontally ears sits behind the microphone receiver plane in its home position.
  • Fig. 7 An example configuration which considers this positioning of microphones is illustrated in Fig. 7. The details of the significance of placing microphones at different distances of each other are explained in the implementation section. Some implementations rely only on a minimal number of microphones, while others rely on clusters of microphones which will give better performance in hostile environments. This model assumes two microphones being placed on an arbitrary plane for a minimal setup, with more microphones on more than one given plane will generate less ambiguous results as this gives less room for aliasing. Intuitively, a point in space can be determined by four or more reference points in space.
  • Robotics kinematic methods are used to generate all possible ear locations, by encoding above assumptions in to a set of Denavit-Hartenberg (DH) matrices.
  • DH matrices are four by four matrices taking both rotation and displacement into consideration to generate a new set of basis vectors for the next connected rigid frame.
  • 5 DH matrices are used to find all points of interest, i.e. neck head, left ear, right ear, and arbitrary unit facing direction.
  • the arbitrary unit facing direction is used to generate the corresponding head facing direction in relation to the left and right ear locations.
  • Location simulation can be analysed in two ways, using forward and/or reverse kinematics.
  • Forward kinematic formulation was used to generate all possible ear locations with reasonable resolution.
  • Time differences between sounds emitted from both ears are then calculated at both proposed microphones. It will be seen that left and right ear pieces sound emissions are coded so that sound differences calculated can be either positive or negative depending on the reference ear selected.
  • Fig. 9 illustrates a simulation result 910 with the neck at its natural/home position with head performing both azimuth and altitude rotations.
  • the points on the chart are labelled as (azimuth, altitude) angle pairs.
  • Calibration of the proposed electronic device 120 may occur in the following way.
  • the electronic device 120 may store two pieces of
  • mapping function J ⁇ l ’ 2 ’ ⁇ ⁇ 2 , the mapping function.
  • This mapping function may be stored as a polynomial so that precise calculations may be carried out during run time, or as a look-up table with missing values interpolated at run-time.
  • the head orientation of the user 110 may be calculated as follows:
  • Fig. 10 illustrates a possible hardware implementation 1000 of the proposed electronic device 120.
  • the microphone array 1030 is connected to an audio coder-decoder (CODEC) 1020 which transfers audio signal to a digital signal processor 1010 via multiple I2S buses 1070. In one example there may be three I2S buses.
  • CDEC audio coder-decoder
  • the wearable device 130 may be connected via a physical connection 1040 to the coder-decoder 1020 or a suitable wireless connection through a radio frequency processing unit 1080.
  • the digital signal processor 1010 may be substituted by a micro-controller unit (MCU) that is able to perform calculations used in the proposed system in a synchronous manner.
  • MCU micro-controller unit
  • an additional stereo waveform may be either added on to normally transmitted audio signal and sent out on the same speakers as normal audio, or sent separately on dedicated speakers on the wearable device (such as the first transducer 140 and second transducer 150).
  • the stereo waveform may be designed with guidelines detailed later. With open-fit headphones as the wearable device 130, it is adequate to have the stereo waveform energising the same set of speakers as normally transmitted audio signals. Depending on the distance of the speakers/transducers and the microphone pick-up, sound leakage through the open fit headphones can be adequate for proper functioning of the proposed system. If the headphones of the wearable device 130 are designed as closed fit, i.e.
  • a separate set of speakers may need to be designed on top of the headphones in a position that is exposed and sound output from such a position is able to be picked up by designated microphones of the proposed system.
  • the stereo waveform may be generated directly using the digital signal processor 1010.
  • a combination of the stereo waveform and audio signals may be computed either on the digital signal processor 1010 or using the coder-decoder 1020.
  • the signals of interest must be re-sampled, so that the output of the waveform is not distorted or aliased.
  • Audio communication buses such as PCM or I2S buses, if used, also need to be configured with bit clocks and frame clocks according to re sampled sample rate. If the signal combination is performed on the coder-decoder 1020, the signals of interest are transmitted over separate buses allowing signals with different sampling rates to be transferred in synchronisation.
  • the signals are then combined on the coder-decoder using its analog mixer.
  • the signals to be combined may be re-sampled with the same sampling rate.
  • Addition is then performed on the digital signal processor 1010 before being transmitted to either an on-board or external digital-to-analog converter (DAC), which is then output to the final stage of signal conditioning before energising the speakers.
  • DAC digital-to-analog converter
  • the stereo waveform is generated via the signal generation enabled coder- decoder 1020, and addition of the waveform with processed audio signal is done using the coder-decoder’s internal analog audio multiplexers (MUX).
  • MUX internal analog audio multiplexers
  • a good transmitter design is essential in the working of the proposed system.
  • Key design requirements on the transmitter side includes the ability to provide just enough information so that the receivers (microphones) can differentiate left and right sound sources, yet does not strain the receivers too much that dedicated
  • Shape of the waveform of the acoustic signals is periodically pulsated, with period twice greater than the time taken for sound to travel between two ears.
  • T denote period
  • d denote distance between the ears
  • v denote speed of sound, which in free air approximately equals 340 m/s.
  • the generated waveform can be of any form, but preferably a frequency
  • modulated signal with orthogonal symbols Such signals can be easily differentiated from a naturally occurring sound.
  • the electronic device 120 may comprise at least one microphone.
  • a single microphone with the ability of receiving sound in the lower ultrasonic range is adequate in normal operation of the system. Multiple microphones may be used when improved performance is desired. Multiple microphones allow the system to perform direction of arrival (DOA) estimates as well as beamforming to improve the signal-to-noise ratio (SNR).
  • DOA direction of arrival
  • SNR signal-to-noise ratio
  • Microphone placements for performing these algorithms need to be designed to avoid temporal aliasing, i.e. distance d between the l microphones should be larger than half of the shortest wavelength l , or simply d ⁇ — .
  • beamforming angles are restricted to a given range. For example for a simple two sensor (microphone( array if arriving angles are in between angles (q,p-q) , our
  • restriction can be relaxed to d ⁇ — -— .
  • Received analog signals are first converted into the digital domain by sampling.
  • sampling methods include successive approximation and sigma-delta analog to digital converters (ADC), etc.
  • ADC analog to digital converters
  • the signal can be either oversampled or undersampled, depending on available processing restrictions.
  • undersampling is able to unwrap the higher frequency component without aliasing, and as an extra advantage sampling can be done essentially on a small micro-controller using an on-board ADC module, which bears minimal hardware cost.
  • sampling is carried out at the Nyquist sampling rate and above to avoid aliasing of the wanted signal component.
  • the Nyquist sampling theorem states that sampling should be carried out at a rate that is at least double the frequency component of interest, in another word when sampling at frequency f s the highest frequency component that can be obtained without aliasing is f s/2 .
  • undersampling and oversampling are defined as sampling below and above said Nyquist sampling rate respectively.
  • undersampling is generally avoided unless the system processing cost is constrained, as undersampling will always lose some frequency information, and will generally have a lower SNR due to frequency wrapping.
  • the use of undersampling is used to condition the signal before determining head position in the core algorithm.
  • bandpass sampling which is a subclass of undersampling
  • Bandpass sampling takes advantage of frequency aliasing to sample a frequency region of interest. It is used to reduce the overhead of oversampling in instances where high frequency sampling is not desired (or is difficult to implement), for example on a power constrained embedded system. Bandpass sampling allows the choice of frequencies on which the warp occurs and warped frequencies positioned is arbitrary.
  • the first step in bandpass sampling is to design for an aliasing frequency.
  • This aliasing frequency will be a reference by which all sampled frequencies warp around.
  • One choice for such a frequency warping range is the 500 to 1,000 Hz range with warping occurs at 1000 Hz and headphones emit an ultrasound signal around 20.75kHz.
  • received signals are firstly oversampled and anti-alias filtered at each sensor of the microphone array 1030.
  • two possible examples are proposed for the initial sampling and filtering stage.
  • a multi-cluster microphone array setup is preferred, in a hardware constrained embodiment a system can be made operational with as few as two microphones.
  • each microphone array in the microphone array cluster 1030 is set up to beamform towards the ears, i.e. the source of the emitted waveforms. Beamforming is an effective way to combat multi-path reflections in a reverberant environment. As the proposed electronic device 120 is worn in a relatively stationary position to the source of waveform emission, a simple way to design such a beamformer is to use the delay-sum technique. In this technique the received signal at each microphone is delayed and added to produce the final output.
  • All microphone signals can now be summed with their respective delays to generate a beamformed signal towards azimuth f and elevation Q .
  • the microphones in each array are ideally designed to create two separate beamforming patterns, one for each ear.
  • both beamformers are designed to point towards 67 degrees elevation, but one with an azimuth angle of +90 degrees and the other -90 degrees. Beamforming is effective in reducing multipath effect in a reverberant environment.
  • the more microphones in an array the narrower the main lobe is on the beamformer. With only two microphones, fixed time-delay beamforming is adequate in covering the complete range of head movement. In a more hostile environment a beamformer with more microphones may be required to obtain signal with higher SNR. Under such scenarios, multiple microphones per array are recommended as well as adaptive beamforming. The inclusion of more microphones gives the system ability to beamform on the signal direction. This is possible due to the fact that the signals are predefined and deterministic. In one embodiment, the signal is designed to be interleaving 20.5 kHz and 20.8 kHz sinusoids, resulting in a new periodic signal with a period difference from either of its two components.
  • the signal is deterministic, and when processed carefully gives a good reference signal for the beamformer to beamform onto.
  • signals received from all microphones are firstly filtered through a bandpass filter that enhances signals present in the desired frequency band.
  • H represents a Hermitian transform operator
  • x is a matrix of stacked received signal vectors.
  • y d is the time shifted reference signal found by first extracting the received signal vector in x , say x 0 corresponding to the 1 element in the weight vector (w) , then find the time shifted reference signal that corresponds with the maximum correlation with x 0 .
  • beamscan method is used to determine the direction corresponds to the most prominent signal in given pass-band. Weight vector of beamscan method can be reused directly to beamform onto calculated direction, hence reduces computation cost of the system.
  • the signal again is passed through a bandpass filter.
  • a DOA method is used to estimate direction of the most prominent signal in the filtered frequency band.
  • the estimated direction is then used as an input into the beamformer, which in turn extracts sound in the desired direction.
  • a number of common DOA methods, such as beamscan, MUSIC, and ESPRIT can be used depending on the computation complexity the system can handle.
  • Using a separate DOA and beamforming stage allows more accurate adaptive beamformer methods to be performed, including minimum variance beamformer, linear constant minimum variance beamformer and general sidelobe canceller beamformers. As these beamforming methods require a known direction to beamform onto, it is essential to firstly estimate the DOA. These beamforming methods also perform better than naive reference signal beamforming as more spatial constraints may be defined with later methods.
  • Time difference is then calculated by taking the difference of t s that correspond to the same signal source but at difference receivers.
  • Time difference at the different receivers is then used as model input to generate head position estimates.
  • filter in a simple smoothing filter setup, filter can be formulated as
  • x and y are estimated head direction in (x, y) and a is the damping factor on dc,dg as well as cross damping between them.
  • Direct damping factor is guarded by the reasonable head rotation speed at given angle and cross damping factor is dictated by the physical correlation between x and y in a spherical constraint. All a s can be made adaptive by using physical limits in a head movement model.
  • SNR enhancement technique can also be used at the time estimation stage using a Kalman filter setup if we have prior knowledge on the noise environment.
  • a Kalman filter setup if we have prior knowledge on the noise environment.
  • Timing difference obtained previously is the actual timing difference plus a small amount of noise due to quantization errors and noise induced quantization errors.
  • Noise induced errors in timing are assumed to be Gaussian distributed and the 3dB attenuation time band of auto-correlation main lobe is assumed to sit within 3 deviations of such distribution.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

A method (200) for determining an orientation of an anatomical feature of a user in relation to an electronic device. A wearable device is located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer, and the electronic device comprising a third transducer. The method comprises transmitting and receiving (210) a first acoustic signal between the first transducer and the third transducer. The method further comprises transmitting and receiving (220) a second acoustic signal between the second transducer and the third transducer. The method further comprises determining (230) a first received time associated with the first acoustic signal and determining (240) a second received time associated with the second acoustic signal. The method also comprises determining (250) the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.

Description

"A method and system for determining an orientation of a user" Technical Field
[0001] The present disclosure relates to a method for determining an orientation of an anatomical feature of a user in relation to an electronic device. The anatomical feature may, in some examples, be a head of a user.
Background
[0002] Knowledge of the orientation of a user’s anatomical feature may be useful to allow a device to focus audible, visual or other information based on the orientation.
For example, knowledge of the user’s head orientation may assist in optimising the delivery of audio to the user based on the orientation.
[0003] Throughout this specification the word“comprise”, or variations such as “comprises” or“comprising”, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.
[0004] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each claim of this application.
Summary
[0005] A method for determining an orientation of an anatomical feature of a user in relation to an electronic device, wherein a wearable device is located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer, and the electronic device comprising a third transducer, the method comprising: transmitting and receiving a first acoustic signal between the first transducer and the third transducer; transmitting and receiving a second acoustic signal between the second transducer and the third transducer; determining a first received time associated with the first acoustic signal; determining a second received time associated with the second acoustic signal; and determining the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.
[0006] The first received time may indicate a time when the first acoustic signal is received by the third transducer and the second received time may indicate a time when the second acoustic signal is received by the third transducer.
[0007] In the method, determining the orientation of the anatomical feature of the user may be further based on a first time difference and a second time difference. The first time difference may be based on a difference between a first time reference and the first received time, wherein the first time reference indicates a time that the first acoustic signal was transmitted. The second time difference may be based on a difference between a second time reference and the second received time, wherein the second time reference indicates a time that the second acoustic signal was transmitted.
[0008] The first transducer may transmit the first acoustic signal. The second transducer may transmit the second acoustic signal.
[0009] In the method, the third transducer may comprise two or more microphones. The method may further comprise: receiving the first and second acoustic signal at the two or more microphones; and processing the first and second acoustic signals received at the two or more microphones to determine a first location and a second location associated with the first and second transducers respectively.
[0010] The at least two microphones may form at least one microphone cluster in the electronic device. [0011] The first acoustic signal and the second acoustic signal may comprise an ultrasonic signal. The ultrasonic signal may act as a carrier wave.
[0012] The first acoustic signal and the second acoustic signal may be acoustic leak signals.
[0013] In the method, the anatomical feature of the user may be a head. The first transducer may be associated with a first ear of the user and the second transducer may be associated with a second ear of the user.
[0014] The electronic device may be located on the user.
[0015] The first transducer and the second transducer of the wearable device may be located approximately symmetrical about a central axis associated with the user. The electronic device may be approximately located on the central axis.
[0016] A system for determining an orientation of an anatomical feature of a user in relation to an electronic device, the system comprising: a wearable device located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer; the electronic device comprising a third transducer; wherein a first acoustic signal is transmitted and received between the first transducer and the third transducer; and wherein a second acoustic signal is transmitted and received between the second transducer and the third transducer; a processor configured to: determine a first received time associated with the first acoustic signal; determine a second received time associated with the second acoustic signal; and determine the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.
[0017] In the system, the third transducer may comprise two or more microphones.
[0018] In the system, the at least two microphones may form at least one microphone cluster in the electronic device. Brief Description of Drawings
[0019] Fig. 1 illustrates a schematic diagram of an example system for determining an orientation of an anatomical feature of a user in relation to an electronic device;
[0020] Fig. 2 illustrates a method of determining an orientation of an anatomical feature of a user in relation to an electronic device;
[0021] Fig. 3 illustrates an example configuration of a wearable device and electronic device;
[0022] Fig. 4 illustrates an anatomical feature of the head of a user turned in a left direction;
[0023] Fig. 5 illustrates an anatomical feature of the head of a user turned in a right direction;
[0024] Fig. 6 illustrates an example of a linear microphone array in an electronic device;
[0025] Fig. 7 illustrates an example of microphone clusters in an electronic device; [0026] Fig. 8 illustrates an example processing device;
[0027] Fig. 9 illustrates results of simulations; and
[0028] Fig. 10 illustrates an example hardware implementation.
Description of Embodiments
Overview of the system 100 [0029] Fig. 1 illustrates a system 100 for determining an orientation of an anatomical feature of a user 110 in relation to an electronic device 120. The system 100 further comprises a wearable device 130 that is located relative to the anatomical feature of the user 110. The wearable device 130 comprises a first transducer 140 and a second transducer 150.
[0030] The first transducer 140 and second transducer 150 may be speakers. In other examples the first transducer 140 and second transducer 150 are microphones. The first transducer 140 and second transducer 150 may be transducers capable of acting as both speakers and microphones. In some example the wearable device 130 may comprise headphones.
[0031] The electronic device 120 comprises a third transducer 160. The third transducer 160 may be a microphone. The third transducer 160 may be a speaker. In other examples the third transducer 160 may be capable of acting as both a speaker and microphone.
[0032] In other examples the electronic device 120 may comprise additional transducers, such as two or more microphones or speakers. In the system 100 a first acoustic signal is transmitted and received between the first transducer 140 and the third transducer 160 (or the additional transducers), and a second acoustic signal is transmitted and received between the second transducer 150 and the third transducer 160 (or additional transducers). In some examples the first acoustic signal and second acoustic signal comprise an ultrasonic signal.
[0033] The system 100 also comprises a processor 170 configured to determine a first received time associated with the first acoustic signal and a second received time associated with the second acoustic signal. In one example, the first received time may indicate a time when the first acoustic signal is received by the electronic device 120 and the second received time may indicate a time when the second acoustic signal is received by the electronic device 120. In some examples the processor may be provided at the electronic device 120. [0034] The method 200 as illustrated by Fig. 2 includes transmitting and receiving 210 a first acoustic signal between the first transducer 140 and the third transducer 160. The method 200 further comprises transmitting and receiving 220 a second acoustic signal between the second transducer 150 and the third transducer 160. In some examples the first acoustic signal may be transmitted from the first transducer 140, and the second acoustic signal may be transmitted from the second transducer 150.
[0035] The method 200 further comprises determining 230 a first received time associated with the first acoustic signal. The method 200 further comprises
determining 240 a second received time associated with the second acoustic signal.
[0036] The method 200 further comprises determining 250 the orientation of the anatomical feature of the user 110 in relation to the electronic device 120 based on the first and second received times.
[0037] A detailed example of the method 200 will now be described.
Method 200
[0038] As described above the method 200 determines an orientation of an anatomical feature of a user 110 in relation to an electronic device 120. A wearable device 130 is located relative to the anatomical feature of the user 110. As described above the wearable device 130 may comprise headphones. In some examples the anatomical feature of the user 110 is a head of the user. In this way, the method 200 determines the orientation of the user’s head. More specifically the method 200 determines the orientation of the head in an azimuth direction in relation to the electronic device 120.
[0039] The first transducer 140 and second transducer 150 of the wearable device 130 may be associated with a first ear of the user and second ear of the user respectively. In one example the first transducer 140 and the second transducer 150 of the wearable device 130 are located approximately symmetrical about a central axis 310 associated with the user 110. This is illustrated in Fig. 3. [0040] The electronic device 120 may be located on the user 110, for example centrally on the body 110. The electronic device 120 may be approximately located on the central axis 310. In one example the electronic device 120 is located in a chest area of the user 110. The electronic device 120 may be located approximately 70 mm down from the base of a neck of the user 110. In other examples the electronic device 120 may be located in a range of 70 mm to 300 mm from the base of the neck of the user. The electronic device 120 may be located at other distances from the base of the neck. The electronic device 120 may extend from the user’s body by approximately 50 mm. In other examples the electronic device 120 may extend from the user’s body by more than 50 mm. The electronic device 120 may sit flush against the body. In other examples the electronic device 120 may be located in other places on the body.
[0041] The electronic device may comprise a hearing aid, mobile phone, audio player, video player, gaming device or radio. The electronic device 120 may be worn on a lanyard around the neck of the user 110. In other examples the electronic device 120 may be attached to clothing of a user.
Transmitting and receiving a first acoustic signal 210
[0042] As described above the method 200 includes transmitting and receiving 210 a first acoustic signal between the first transducer 140 and the third transducer 160. In one example the first transducer 140 of the wearable device 130 transmits the first acoustic signal. In this way, the third transducer 160 (and/or the additional transducers of the electronic device 120) receives the first acoustic signal from the first transducer 140.
[0043] In another example, the third transducer 160 (and/or the additional transducers of the electronic device) transmits the first acoustic signal and the first transducer 140 receives the first acoustic signal.
[0044] As described above the electronic device 120 may comprise additional transducers, such as two or more microphones. In this way, the first acoustic signal may be received at the two or more microphones of the electronic device 120. In some examples, the at least two microphones of the electronic device 120 may form at least one microphone cluster 120.
Transmitting and receiving a second acoustic signal 220
[0045] As described above the method 200 also includes transmitting and receiving 220 a second acoustic signal between the second transducer 150 and the third transducer 160. The second transducer 150 of the wearable device 130 may transmit the second acoustic signal. In this way, the third transducer 165 and/or additional transducers of the electronic device 120 receives the second acoustic signal from the second transducer 150.
[0046] In another example the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal and the second transducer 150 receives the second acoustic signal.
[0047] The second acoustic signal may be received at the two or more microphones of the electronic device 120.
[0048] The first acoustic signal and second acoustic signal may be transmitted at the same instant from the first transducer 140 and the second transducer 150 respectively. Alternatively the first acoustic signal and second acoustic signal may be transmitted at different times from the first transducer 140 and the second transducer 150
respectively. In other examples the first acoustic signal and second acoustic signal may be transmitted at the same instant from the electronic device 120. Alternatively the first acoustic signal and second acoustic signal may be transmitted at different times from the electronic device 120.
[0049] The first acoustic signal and second acoustic signal may comprise an audible acoustic signal, such as music or audio content. In one example the first acoustic signal and second acoustic signal may be acoustic leak (or leakage) signals. This means that the first acoustic signal and second acoustic signal are not purposely emitted, but rather, are leakages from acoustic signals associated with the first transducer 140 and second transducer 150. In this way, the first acoustic signal and second acoustic signal may leak out from the first transducer 140 and second transducer 150 respectively and be detected by the third transducer 160. For example the third transducer 160 and/or additional transducers of the electronic device 120 may detect the first acoustic signal and second acoustic signal as acoustic leak signals.
[0050] In other examples the first acoustic signal and second acoustic signal may comprise a non-audible acoustic signal. In this example the first acoustic signal and second acoustic signal may have a frequency of less than 20 Hz or greater than 20 kHz. In one example the first acoustic signal and second acoustic signal comprise infrasonic signals. In another example the first acoustic signal and second acoustic signal are ultrasonic signals with a frequency greater than 20 kHz. In a further example the first acoustic signal and second acoustic signal are frequency modulated signals, and an ultrasonic signal may act as a carrier wave for the frequency modulated signals. The frequency of the carrier wave may be above 132 kHz.
[0051] In some examples the first acoustic signal and/or the second acoustic signal may be a speech signal in the spectrum from 20 Hz to 20 kHz. That is, the first acoustic signal and/or second acoustic signal may be a baseband signal (prior to modulation). In this way, the baseband signal may be a periodic wave such as a sinusoidal wave. In one example, the period of the baseband signal (that is, the first acoustic signal or second acoustic signal prior to modulation) may be greater than twice the time taken for acoustic sound to travel between ears of the user 110. That is, denoting the distance between ears of the user 110 as d and the speed of sound as v, the period T of the baseband signal (before modulation) may be computed as follows: d
T >— x 2
v [0052] In one example digital modulation is used, so that the first acoustic signal and second acoustic signal are generated by modulating an ultrasound carrier wave by a discrete (baseband) signal. In one example the ultrasound carrier wave is modulated by symbol coded messages. In a further example the ultrasound carrier wave is modulated by orthogonal symbols. The frequency for the symbol coded messages may fall between 20.5 kHz and 21 kHz. A minimum of three symbols may be transmitted.
[0053] An advantage of using an ultrasonic signal as the carrier wave is that ultrasound are not audible to the human ear at frequencies above 20 kHz airborne. For sound frequency of 20 kHz, the safety guideline for exposure limits on airborne ultrasound according to the Occupational Safety and Health Administration (OSHA) is 105 dB. For ultrasound signals with a frequency around 100 kHz the exposure limit increases to 115 dB. This exposure limit can be increased if there is no possibility that the ultrasound signal can be coupled with the human body. This means that for an ultrasound signal with a frequency over 132 kHz, the exposure limit is greater than 115 dB.
[0054] A further advantage of using an ultrasound signal is that ultrasound is more directional than signals within the human audible frequency range. This means that ultrasound signals may have a better defined flight path and spatial filtering techniques may be utilised to minimise multi-path propagation.
First acoustic signal and second acoustic signal transmission time
[0055] As described above the electronic device 120 may be located on the user 110 about a central axis 310, with the first transducer 140 and second transducer 150 located approximately symmetrical about the central axis 310. This is illustrated in Fig. 3.
[0056] In this configuration, when the user 110 is facing forward (i.e. facing in a direction of the central axis 310) the distance 320, 330 between the first transducer 140 and second transducer 150 to the electronic device 120 respectively is approximately equal.
[0057] In this way, when the electronic device 120 transmits a first acoustic signal and second acoustic signal to the first transducer 140 and second transducer 150 respectively, the first transducer 140 and second transducer 150 may receive the first acoustic signal and second acoustic signal at approximately the same time. In other words, the first acoustic signal and second acoustic signal may require an
approximately equal amount of time to arrive at the first transducer 140 and second transducer 150 respectively.
[0058] In other examples the first transducer 140 and second transducer 150 may receive the first acoustic signal and second acoustic signal within a delay of each other.
[0059] Similarly, when the first transducer 140 and second transducer 150 transmit the first acoustic signal and second acoustic respectively to the electronic device 120, the electronic device 120 receives the first acoustic signal and second acoustic signal at approximately the same time.
[0060] In other examples the electronic device 120 may receive the first acoustic signal and second acoustic signal within a delay of each other.
[0061] However, when the first transducer 140 and second transducer 150 are not located approximately symmetrical about the central axis 310 the time taken for the first acoustic signal and second acoustic signal to be transmitted and received between the transducers 140, 150 and electronic device 120 may not be approximately equal. This means that there may be a delay between the times that the electronic device 120 receives the first acoustic signal and second acoustic signals. Similarly, when the electronic device 120 transmits the first acoustic signal and second acoustic signal there may be a delay between the times that the first transducer 140 and second transducer 150 receive the first acoustic signal and second acoustic signal respectively. This may occur when the orientation of the user’s head changes (for example, when the user turns his head left or right).
[0062] Referring to Fig. 4, consider the user 110 with the anatomical feature of the head turned in a left direction. In this way, the distance 420 from the first transducer 140 to the electronic device 120 is different to the distance 430 from the second transducer 150 to the electronic device 120. In this example the distance 420 may be less than the distance 430. In this way, the time for the first acoustic signal to transmit and receive between the first transducer 140 and the third transducer 160 (and/or additional transducers of the electronic device 120) may be shorter than the time for the second acoustic signal to transmit and receive between the second transducer 150 and the third transducer 160 (and/or additional transducers of the electronic device 120).
[0063] A similar situation is described in Fig. 5, where the head of the user 110 is turned to the right. In this example the distance 530 from the second transducer 150 to the electronic device 120 may be less than the distance 520 from the first transducer 140 to the electronic device 120. In this way, the time for the first acoustic signal to transmit and receive between the second transducer 150 and the third transducer 160 (and/or additional transducers of the electronic device 120) may be shorter than the time for the first acoustic signal to transmit and receive between the first transducer 140 and the third transducer 160 (and/or additional transducers of the electronic device 120).
Determining a first received time associated with the first acoustic signal 230
[0064] As described above the method 200 also includes determining 230 a first received time associated with the first acoustic signal. In one example, the first transducer 140 transmits the first acoustic signal and the first received time indicates a time when the first acoustic signal is received by the third transducer 160 (and/or additional transducers) of the electronic device 120. In some examples the first transducer 140 transmits the first acoustic signal in accordance with a clock signal. The clock signal may be associated with the first transducer 140, the wearable device 130 or another device.
[0065] In another example, the third transducer 160 and/or additional transducers of the electronic device 120 transmits the first acoustic signal and the first received time indicates a time when the first acoustic signal is received by the first transducer 140. In some examples the third transducer 160 and/or additional transducers of the electronic device 120 transmits the first acoustic signal in accordance with a clock signal. The clock signal may be associated with the third transducer 160 and/or additional transducers of the electronic device 120, the electronic device 120 or another device.
Determining a second received time associated with the second acoustic signal 240
[0066] As described above the method 200 also includes determining 240 a second received time associated with the second acoustic signal. In one example, the second transducer 150 transmits the second acoustic signal and the second received time indicates a time when the second acoustic signal is received by the third transducer 160 and/or additional transducers of the electronic device 120. In some examples the second transducer 150 transmits the second acoustic signal in accordance with a clock signal. The clock signal may be associated with the second transducer 140, the wearable device 130 or another device.
[0067] In another example, the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal and the second received time indicates a time when the second acoustic signal is received by the second transducer 150. In some examples the third transducer 160 and/or additional transducers of the electronic device 120 transmits the second acoustic signal in accordance with a clock signal. The clock signal may be associated with the third transducer 160 and/or additional transducers of the electronic device 120, the electronic device 120 or another device.
Determining the orientation of the anatomical feature of the user 250 [0068] As described above the method also includes determining 250 the orientation of the anatomical feature of the user in relation to the electronic device 120 based on the first and second received times.
[0069] In the present disclosure, determining 250 the orientation of the anatomical feature of the user 110 is based on the delay in the first acoustic signal and second acoustic signal being received. As described above the first acoustic signal and second acoustic signal may be received by the third transducer 160 (and/or additional transducers of the electronic device 120). In other examples the first acoustic signal and second acoustic signal may be received by the first transducer 140 and second transducer 150 respectively.
[0070] Determining 250 the orientation of the anatomical feature of the user may be based on a first time difference and a second time difference. The first time difference may be based on a difference between a first time reference and the first received time. Examples of the first received time are described above with respect to step 230 of method 200.
[0071] In one example the first time reference may indicate a time that the first acoustic signal is transmitted by the first transducer 140 or third transducer 160 (or additional transducers of the electronic device 120). In another example the first time reference may indicate a common time for transmission of the first acoustic signal and another acoustic signal, such as the second acoustic signal. In other examples the first time reference may indicate another event associated with the first acoustic signal.
[0072] In a similar way the second time difference may be based on a difference between a second time reference and the second received time. Examples of the second received time are described above with respect to step 240 of method 200.
[0073] The second time difference may indicate a time that the second acoustic signal is transmitted. In another example the second time reference may indicate a common time for transmission as described above. In other examples the second time reference may indicate another event.
[0074] In one example, determining 250 the orientation of the anatomical feature of the user 110 comprises determining the orientation of the user’s head in the azimuth direction. In this example, determining 250 the orientation of the user’s head may be based on the first time difference and the second time difference. For instance, if the first time difference is greater than the second time difference, determining 250 the orientation may comprise determining that the user’s head is oriented in a left direction as illustrated in Fig. 4. If the first time difference is less than the second time difference, determining 250 the orientation may comprise determining that the user’s head is oriented in a right direction as illustrated in Fig. 5.
[0075] In a similar way, if the first time difference and the second time difference are equal, determining 250 the orientation may comprise determining that the user’s head is oriented approximately on the central axis 310. In other examples the orientation may be determined as on the central axis 310 if the values of the first time difference and second time difference are within a threshold of each other.
Third transducer 160 of the electronic device 120
[0076] As described above the electronic device 120 comprises the third transducer. The electronic device 120 may also comprise additional transducers, such as two or more microphones. The two or more microphones may be located in spatially dispersed locations of the electronic device 120. In one example the two or more microphones may form a linear microphone array 600. An example of the linear microphone array 600 is illustrated in Fig. 6.
[0077] The microphones 610, 620, 630, 640 in the linear microphone array 600 may be spaced to avoid spatial aliasing. This means that the microphones 610, 620, 630,
640 may be spaced so that a distance 660 between each microphone is less than half the wavelength of the first acoustic signal and the second acoustic signal. For instance, if the frequency of the first acoustic signal and the second acoustic signal is 132 kHz the corresponding wavelength will be around 2.5 mm. This means that the microphones 610, 620, 630, 640 in the array 600 may be located with a distance of less than 1.25 mm between each microphone.
[0078] In other examples the microphones 610, 620, 630, 640 in the array 600 are located at a distance greater than half the wavelength of the first acoustic signal and the second acoustic signal.
[0079] As also described above the at least two microphones may form at least one microphone cluster in the electronic device 120. There may be more than one microphone cluster in the electronic device 120. For example there may be two microphones per microphone cluster. The two microphones may be spaced at a distance from each other to avoid spatial aliasing.
[0080] An example of the microphone clusters is illustrated in Fig. 7. The microphones 730, 740, 750, 760, 770, 780 of the microphone cluster 710, 720 may be spaced at a distance from each other to avoid spatial aliasing. That is, the microphones 730, 740, 750, 760, 770, 780 are located at a distance 790 less than half the wavelength of the first acoustic signal and the second acoustic signal. In a further example each microphone cluster 710, 720 is placed as far apart from each other as allowable by the dimensions of the third transducer 160 and electronic device 120. This is so that the received acoustic signals at the microphones 730, 740, 750, 760, 770, 780 are sampled with less correlation.
Microphone array for localisation
[0081] In some examples the method 200 further comprises processing the signals received at the two or more microphones of the electronic device 120 to determine a first location and a second location, where the first location and second location are associated with the first and second transducers respectively. [0082] Determining the first location may comprise processing the signals received at the array 600 or clusters 710, 720 to determine the direction of arrival (DOA). This may be based on a time delay of arrival (TDOA) estimation. This may further be based on the delays measured from transmission of the first acoustic signal and second acoustic signal between the electronic device 120 and the transducers 140, 150. In other examples localisation may be based on other acoustic signals transmitted between the electronic device 120 and transducers 140, 150.
[0083] In another example determining the first location comprises performing beamforming on the signals received at the microphones in the array 600 or clusters 710, 210. Examples of beamforming may comprise delay and sum beamformer, or minimum variance distortionless response beamformer.
[0084] In other examples other methods may be used such as Multiple Signal Classification (MUSIC), Estimation of Signal Parameters via Rotational Invariance Technique (ESPRIT) or Degenerate Unmixing Estimation Technique (DUET).
Processor 170
[0085] As described above the system 100 comprises a processor 170. Fig. 8 illustrates an example of a processing device. The processing device includes a processor 810, a memory 820 and an interface device 840 that communicate with each other via a bus 830. The memory 820 stores a computer software program comprising machine-readable instructions 824 and data 822 for implementing the method 200 described above, and the processor 810 performs the instructions from the memory 820 to implement the method 200.
[0086] The interface device 840 may include a communications module that facilitates communication with a communications network, and in some examples, with the user interface 840 and peripherals such as data store 822. It should be noted that although the processing device may be an independent network element, the processing device may also be part of another network element. Further, some functions performed by the processing device may be distributed between multiple network elements.
Model, simulations and implementation examples
Model
[0087] The model used in developing algorithms is derived using estimated average dimensions. These dimensions are used as guideline for evaluating the model and brings about intuition, whereas the model itself does not rely on these dimension measurements.
[0088] A rigid body model is assumed with a neck as a stick leaning naturally forward at 15 degrees with the tip of the neck connected with a free rotating vertical rigid stick, which in turn connects with a free rotating horizontal stick with ends denoting ear positions. The width of the head is assumed to be 155 mm, length of the neck 110 mm, vertical displacement from the tip of the neck to the ear canal 30 mm. In this proposed model, head is free to rotate sideways by up to 45 degrees pivoting on the tip of the neck in all directions, this is represented by the vertical stick pivoting on top of the neck stick.
[0089] The head is also allowed to rotate left and right by 80 degrees each side, which is represented by allowing the horizontal ear stick model to rotate. Neck movement is a lot more restricted, allowing 10 degrees backward, 50 degrees frontal, and 45 degrees sideways, with pivot angle allowance interpolated for directions in between.
[0090] The proposed electronic device 120 is located at 70 mm down from the base of the neck and extends forward by 50 mm. Such a location is used as a general guideline only, the actual location of the microphone plane can be of any orientation as long as its orientation does not change drastically during the life of operation. For example, system can have a number of microphones on the receiving device in an orientation A. If the device was to assume an orientation B after wearer calibration, system needs to be re-calibrated before measurement can be made with reasonable accuracy. For the sake of building a clear model, we assume that horizontally ears sits behind the microphone receiver plane in its home position.
[0091] The system works on the principle of time of arrival differences in different microphones. Timing differences work best when the microphones are further apart, whereas directional information is best obtained from microphones that are closer together, i.e. less than half wavelength apart. An example configuration which considers this positioning of microphones is illustrated in Fig. 7. The details of the significance of placing microphones at different distances of each other are explained in the implementation section. Some implementations rely only on a minimal number of microphones, while others rely on clusters of microphones which will give better performance in hostile environments. This model assumes two microphones being placed on an arbitrary plane for a minimal setup, with more microphones on more than one given plane will generate less ambiguous results as this gives less room for aliasing. Intuitively, a point in space can be determined by four or more reference points in space.
[0092] In the model, as spatial measurements are unreliable due to multipath effect or echoes, time differences of arrival at given microphones are relied on, assuming signals emitted from both ear pieces following similar pathways. From the simulation results, it is clear that in order to have unambiguous timing difference-head position measurements, at least three microphones are required and be located in such a manner that their position vectors can span a 3-D space. The two microphone locations in the modelling positions are assumed to be located diagonally at the corner of the receiving device, i.e. 180 degree rotational invariant. It is further assumed that sound travels in a homogeneous medium before reaching the receiving device, hence assuming a constant speed. This speed is assumed to be 340 m/s.
[0093] Robotics kinematic methods are used to generate all possible ear locations, by encoding above assumptions in to a set of Denavit-Hartenberg (DH) matrices. DH matrices are four by four matrices taking both rotation and displacement into consideration to generate a new set of basis vectors for the next connected rigid frame. In this model, 5 DH matrices are used to find all points of interest, i.e. neck head, left ear, right ear, and arbitrary unit facing direction. The arbitrary unit facing direction is used to generate the corresponding head facing direction in relation to the left and right ear locations. It is a rigid frame connected to horizontal ears frame on a fixed joint, coinciding with the tip of the vertical head position frame, pointing straight forward when the system is at its home position. Final head facing direction is calculated by subtracting the head position vector from the arbitrary unit facing direction position vector; this gives a unit vector pointing in the same direction as the head is facing.
[0094] Location simulation can be analysed in two ways, using forward and/or reverse kinematics. Forward kinematic formulation was used to generate all possible ear locations with reasonable resolution. Time differences between sounds emitted from both ears are then calculated at both proposed microphones. It will be seen that left and right ear pieces sound emissions are coded so that sound differences calculated can be either positive or negative depending on the reference ear selected.
[0095] It is clear from forward kinematic simulations that there exists a definitive relationship between head movement on a given plane and time differences received at the microphones, denoted by f (HD, TDi D1 , · · ·) , where HD denotes head facing direction, TD denotes time delays at microphone indexed by x . However, due to limitation of microphone locations, which were assumed to be on the same plane, aliasing of measurement on one plane does occur against measurements on other planes, i.e. f(HD | directionl, GZ^,GZl,, · · ·) = /(HO,GZ^,GZl,) .
[0096] Fig. 9 illustrates a simulation result 910 with the neck at its natural/home position with head performing both azimuth and altitude rotations. The points on the chart are labelled as (azimuth, altitude) angle pairs.
[0097] From Fig. 9 it is clear that there is much overlapping between the
measurements at different angles. As discussed above, such aliasing can be reduced by the inclusion of multiple microphones. Yet a two microphone setup does generate a usable set of measurements if the system is calibrated properly. As can be seen from Fig. 9, all vertical curves at different altitudes are similar but with different offsets. A set of time difference pairs are obtained from unskewing such curves by remapping to its PCA vectors. Values on different curves are then averaged to a single curve which in turn is fitted to a polynomial function HD = f(TDv,TD2 , · · ·) . There will be more TD components to the fitting if more than 2 microphones or microphone clusters are used; this now becomes the model of the system.
Calibration of model and system
[0098] Before using the proposed model it is recommended to use the following calibration sequence:
1. Prompt user to rotate head left and right while holding a steady elevation;
2. Prompt user to look up and then down while facing straight ahead;
3. Prompt user to look straight ahead and rock head side to side;
4. Calculate the average vector differences of time delay represented using (left sensor time difference, right sensor time difference) format. This is done separately on samples obtained from step 1 and step 2. As an example, let /¾,/¾, · · · be samples taken from step 1, and vl,v2, · · · be samples taken from step 2. One set of vector differences is found by taking differences between all pairs of ¾, h2 , · · · and difference vectors are all made to face the same direction. And another set of vector differences is found using the same method but on
>v2, · · · .
[0099] The resulting vectors are then normalised. These normalised vectors are approximately the Principal Component Vectors (but may be 180 degrees out of phase, as direction is not certain) in the model. [0100] After obtaining the vectors, the principal axis associated with altitude changes can then be used to map new sample points back on to the 0 altitude curve. After which, HD = /(7 , ,7D2 , · · ·) can be used to obtain the head direction vector desired.
[0101] Calibration of the proposed electronic device 120 may occur in the following way. At implementation, the electronic device 120 may store two pieces of
information:
1. Principal Component Vectors obtained from the calibration process described above; and
2. HD— f (TD TD
J ^ l2’ · · 2 , the mapping function. This mapping function may be stored as a polynomial so that precise calculations may be carried out during run time, or as a look-up table with missing values interpolated at run-time.
At run time, the head orientation of the user 110 may be calculated as follows:
1. Obtaining time difference at both microphones (assuming a dual
microphone setup) measurements post pre-processing;
2. Using stored principal vectors to map given points onto the zero
elevation curve; and
3. Using the stored mapping function
Figure imgf000023_0001
jn
polynomial or table form to generate an estimate of the final head position.
Implementation
[0102] Fig. 10 illustrates a possible hardware implementation 1000 of the proposed electronic device 120. In this implementation the microphone array 1030 is connected to an audio coder-decoder (CODEC) 1020 which transfers audio signal to a digital signal processor 1010 via multiple I2S buses 1070. In one example there may be three I2S buses.
[0103] In this example implementation 1000 the wearable device 130 may be connected via a physical connection 1040 to the coder-decoder 1020 or a suitable wireless connection through a radio frequency processing unit 1080. The digital signal processor 1010 may be substituted by a micro-controller unit (MCU) that is able to perform calculations used in the proposed system in a synchronous manner.
[0104] During operation of the electronic device 120, an additional stereo waveform may be either added on to normally transmitted audio signal and sent out on the same speakers as normal audio, or sent separately on dedicated speakers on the wearable device (such as the first transducer 140 and second transducer 150). The stereo waveform may be designed with guidelines detailed later. With open-fit headphones as the wearable device 130, it is adequate to have the stereo waveform energising the same set of speakers as normally transmitted audio signals. Depending on the distance of the speakers/transducers and the microphone pick-up, sound leakage through the open fit headphones can be adequate for proper functioning of the proposed system. If the headphones of the wearable device 130 are designed as closed fit, i.e. only let out a minimum possible amount of sound leakage through the speakers/transducers, a separate set of speakers may need to be designed on top of the headphones in a position that is exposed and sound output from such a position is able to be picked up by designated microphones of the proposed system.
[0105] In one example, the stereo waveform may be generated directly using the digital signal processor 1010. A combination of the stereo waveform and audio signals may be computed either on the digital signal processor 1010 or using the coder-decoder 1020. When the combination is performed on the digital signal processor 1010 before output, the signals of interest must be re-sampled, so that the output of the waveform is not distorted or aliased. Audio communication buses, such as PCM or I2S buses, if used, also need to be configured with bit clocks and frame clocks according to re sampled sample rate. If the signal combination is performed on the coder-decoder 1020, the signals of interest are transmitted over separate buses allowing signals with different sampling rates to be transferred in synchronisation. The signals are then combined on the coder-decoder using its analog mixer. For a lower cost solution, the signals to be combined may be re-sampled with the same sampling rate. Addition is then performed on the digital signal processor 1010 before being transmitted to either an on-board or external digital-to-analog converter (DAC), which is then output to the final stage of signal conditioning before energising the speakers. In another embodiment, the stereo waveform is generated via the signal generation enabled coder- decoder 1020, and addition of the waveform with processed audio signal is done using the coder-decoder’s internal analog audio multiplexers (MUX).
Transmitter design
[0106] A good transmitter design is essential in the working of the proposed system. Key design requirements on the transmitter side includes the ability to provide just enough information so that the receivers (microphones) can differentiate left and right sound sources, yet does not strain the receivers too much that dedicated
hardware/software are required to decode such information.
[0107] The following rules are recommended:
1. Frequency of the waveform for the carrier wave for modulation should be
outside the human audible range, i.e. 20 Hz to 20 kHz, preferably in the lower ultrasonic ranges for easier signal processing.
2. Shape of the waveform of the acoustic signals (baseband signal) is periodically pulsated, with period twice greater than the time taken for sound to travel between two ears. Let T denote period, d denote distance between the ears, and v denote speed of sound, which in free air approximately equals 340 m/s. We require:
Figure imgf000026_0001
Since T is greater than twice the maximum possible delay, the maximum
T
possible delay is always less than— . Given two sequences of waveforms (i.e.
Figure imgf000026_0002
one from the left ear and one from the right ear) we can always determine the orientation of the sequences is the orientation that always produces minimum delay. Following this constraint greatly simplifies periodicity detection.
3. The generated waveform can be of any form, but preferably a frequency
modulated signal with orthogonal symbols. Such signals can be easily differentiated from a naturally occurring sound.
[0108] In one example (as described above) simple symbol coded message over the carrier wave are recommended. Since low range ultra sound frequencies are preferred as discussed in the previous section, one possible frequency range for such coded message would be the [20.5 kHz, 21 kHz] range. The benefits of using this range are discussed later in the receiver section. At least three symbols are recommended to be transmitted as a symbol count of less than three cannot be guaranteed to be
differentiated when transmission delay time differences are longer than one symbol length. It is good practice to design single symbol transmission times longer than the anticipated delayed time differences.
Receiver (microphone) design
[0109] As described above the electronic device 120 may comprise at least one microphone. A single microphone with the ability of receiving sound in the lower ultrasonic range is adequate in normal operation of the system. Multiple microphones may be used when improved performance is desired. Multiple microphones allow the system to perform direction of arrival (DOA) estimates as well as beamforming to improve the signal-to-noise ratio (SNR). Microphone placements for performing these algorithms need to be designed to avoid temporal aliasing, i.e. distance d between the l microphones should be larger than half of the shortest wavelength l , or simply d <— .
Figure imgf000027_0001
This restriction may be relaxed when the expected angle of arrival and the
beamforming angles are restricted to a given range. For example for a simple two sensor (microphone( array if arriving angles are in between angles (q,p-q) , our
Figure imgf000027_0002
restriction can be relaxed to d <— -— .
2 cos($)
[0110] Received analog signals are first converted into the digital domain by sampling. Commonly used sampling methods include successive approximation and sigma-delta analog to digital converters (ADC), etc. The signal can be either oversampled or undersampled, depending on available processing restrictions. In a processing-restrained system, undersampling is able to unwrap the higher frequency component without aliasing, and as an extra advantage sampling can be done essentially on a small micro-controller using an on-board ADC module, which bears minimal hardware cost.
[0111] The disadvantages of undersampling include loss of accuracy in phase difference measurement between the arriving signals. This means the direction of arrival estimate may become unstable due to frequency wrapping. Oversampling on the other hand is the method of choice when the system signal processing is not as restricted. Oversampling offers greater resolution, where high frequency components are directly available for processing, hence enables algorithms such as direction of arrival (DOA) and beamforming to be more effectively utilized. Details of both embodiments will be discussed in detail below.
[0112] In signal processing sampling is carried out at the Nyquist sampling rate and above to avoid aliasing of the wanted signal component. The Nyquist sampling theorem states that sampling should be carried out at a rate that is at least double the frequency component of interest, in another word when sampling at frequency fs the highest frequency component that can be obtained without aliasing is fs/2 . In comparison, undersampling and oversampling are defined as sampling below and above said Nyquist sampling rate respectively.
Undersampling pre-processing
[0113] Undersampling is generally avoided unless the system processing cost is constrained, as undersampling will always lose some frequency information, and will generally have a lower SNR due to frequency wrapping. In one example the use of undersampling is used to condition the signal before determining head position in the core algorithm.
[0114] In this example, bandpass sampling, which is a subclass of undersampling, is used. Bandpass sampling takes advantage of frequency aliasing to sample a frequency region of interest. It is used to reduce the overhead of oversampling in instances where high frequency sampling is not desired (or is difficult to implement), for example on a power constrained embedded system. Bandpass sampling allows the choice of frequencies on which the warp occurs and warped frequencies positioned is arbitrary.
[0115] The first step in bandpass sampling is to design for an aliasing frequency.
This aliasing frequency will be a reference by which all sampled frequencies warp around. One choice for such a frequency warping range is the 500 to 1,000 Hz range with warping occurs at 1000 Hz and headphones emit an ultrasound signal around 20.75kHz.
[0116] This is a preferred warping frequency in the undersampling example. This may provide the following advantages:
1. Operating in this range only requires a sampling rate of no less than 11,000 samples per second compared to at least 50,000 samples per second, which is a saving on processing power. This eliminates the need for a dedicated high speed ADC and should be easily accomplished by many small micro- controllers. 2. Doing so may also avoid potential constant noise sources such as car and lawn mower noises, which mostly generate noises below 500 Hz. If waveform frequency is not carefully chosen and coincided with a constant noise source during operation, a reduction in SNR will cause a disruption in the head movement estimation routine.
[0117] The disadvantage of having such a low sampling rate is slightly more memory consuming post processing. Once the bandpass signal is obtained, phase difference rather than cycle difference needs to be examined as time delay is less than the period of designed baseband signal. As a result the signal needs to be interpolated to such a sample rate that time differences between samples need to be at most twice double the final resolution.
Oversampling pre-processing
[0118] When adequate hardware is provided for oversampling pre-processing, received signals are firstly oversampled and anti-alias filtered at each sensor of the microphone array 1030. Depending on the hardware budget and required accuracy, two possible examples are proposed for the initial sampling and filtering stage. In applications requiring high accuracy and with no processing constraints, a multi-cluster microphone array setup is preferred, in a hardware constrained embodiment a system can be made operational with as few as two microphones.
[0119] In a multi-cluster microphone array setup, at least two microphone arrays are needed, with at least two microphones in each array. The distances between microphones within an array need to be less than half the wavelength of the highest expected incoming signal frequency. For the preferred frequency operation range, this is just below 1 cm. In this example, each microphone array in the microphone array cluster 1030 is set up to beamform towards the ears, i.e. the source of the emitted waveforms. Beamforming is an effective way to combat multi-path reflections in a reverberant environment. As the proposed electronic device 120 is worn in a relatively stationary position to the source of waveform emission, a simple way to design such a beamformer is to use the delay-sum technique. In this technique the received signal at each microphone is delayed and added to produce the final output.
[0120] Given the location of each microphone represented by its coordinates m = (mx,my,mz ) and the steering vector w = ( I,f,q) with f denoting angle of azimuth and Q denoting angle of elevation, we can calculate the delay needed for the ith microphone as follows:
d
Figure imgf000030_0001
e ay = - speed of sound
[0121] All microphone signals can now be summed with their respective delays to generate a beamformed signal towards azimuth f and elevation Q .
beamformer
Figure imgf000030_0002
[0122] The microphones in each array are ideally designed to create two separate beamforming patterns, one for each ear. In this example, both beamformers are designed to point towards 67 degrees elevation, but one with an azimuth angle of +90 degrees and the other -90 degrees. Beamforming is effective in reducing multipath effect in a reverberant environment.
[0123] The more microphones in an array the narrower the main lobe is on the beamformer. With only two microphones, fixed time-delay beamforming is adequate in covering the complete range of head movement. In a more hostile environment a beamformer with more microphones may be required to obtain signal with higher SNR. Under such scenarios, multiple microphones per array are recommended as well as adaptive beamforming. The inclusion of more microphones gives the system ability to beamform on the signal direction. This is possible due to the fact that the signals are predefined and deterministic. In one embodiment, the signal is designed to be interleaving 20.5 kHz and 20.8 kHz sinusoids, resulting in a new periodic signal with a period difference from either of its two components. Yet, the signal is deterministic, and when processed carefully gives a good reference signal for the beamformer to beamform onto. There are two general ways of carrying out this process, either by doing direct beamforming onto the reference signal after filtering, or firstly finding the direction of arrival of the signal then beamform onto given direction.
[0124] In the former method, two possible implementations are recommended. In implementation one, signals received from all microphones are firstly filtered through a bandpass filter that enhances signals present in the desired frequency band.
Beamforming is then performed with weight vector (w) that minimises E{\ wHx- yd |}
. H represents a Hermitian transform operator x is a matrix of stacked received signal vectors. yd is the time shifted reference signal found by first extracting the received signal vector in x , say x0 corresponding to the 1 element in the weight vector (w) , then find the time shifted reference signal that corresponds with the maximum correlation with x0 . In the second preferred implementation, beamscan method is used to determine the direction corresponds to the most prominent signal in given pass-band. Weight vector of beamscan method can be reused directly to beamform onto calculated direction, hence reduces computation cost of the system.
[0125] In the second approach, the signal again is passed through a bandpass filter. After this a DOA method is used to estimate direction of the most prominent signal in the filtered frequency band. The estimated direction is then used as an input into the beamformer, which in turn extracts sound in the desired direction. A number of common DOA methods, such as beamscan, MUSIC, and ESPRIT can be used depending on the computation complexity the system can handle.
[0126] Using a separate DOA and beamforming stage allows more accurate adaptive beamformer methods to be performed, including minimum variance beamformer, linear constant minimum variance beamformer and general sidelobe canceller beamformers. As these beamforming methods require a known direction to beamform onto, it is essential to firstly estimate the DOA. These beamforming methods also perform better than naive reference signal beamforming as more spatial constraints may be defined with later methods.
Decoding and time difference estimates
[0127] Once signals are received and pre-processed, they are then pushed on to a circular buffer. Buffer size is designed to be larger than symbol length *4. This is adequate given symbol length is assumed to be longer than inter-aural time difference, as discussed in section above. Cross-correlation is then calculated between received signals and time shifted known symbol sequences
Figure imgf000032_0001
and S2(i - r2)
corresponding to left and right ear source encoding. Both 5, and S2 are correlated with all incoming signal streams. Two outcomes are achieved from this”
1. Determining source of given signal stream. By calculating cross-correlation between 5, , S2 and given stream, one cross-correlation would result in a higher max value then the other, hence indicating the correct signal source.
2. Determining time difference. Once signal source is determined, cross- correlation results are reused to find t corresponding to the correct signal source and peak of its cross-correlation spectrum. Time difference is then calculated by taking the difference of t s that correspond to the same signal source but at difference receivers.
[0128] Time difference at the different receivers is then used as model input to generate head position estimates.
SNR enhancement
[0129] There is always noise associated with received signals and also quantisation noise associated with DOA, beamforming and cross-correlation stages. Hence in the output stage, a smoothing filter is recommended. This also makes sense physically, as head movements are smooth in nature. There are two ways to perform such task, one is a simple low pass smoothing filter, which may or may not be adaptive. An SNR improvement technique using adaptive filtering is also discussed.
[0130] In a simple smoothing filter setup, filter can be formulated as
Figure imgf000033_0001
[0131] where x and y are estimated head direction in (x, y) and a is the damping factor on dc,dg as well as cross damping between them. Direct damping factor is guarded by the reasonable head rotation speed at given angle and cross damping factor is dictated by the physical correlation between x and y in a spherical constraint. All a s can be made adaptive by using physical limits in a head movement model.
[0132] SNR enhancement technique can also be used at the time estimation stage using a Kalman filter setup if we have prior knowledge on the noise environment. In designing the Kalman filter, we assume the following:
1. Timing difference obtained previously is the actual timing difference plus a small amount of noise due to quantization errors and noise induced quantization errors.
2. Noise induced errors in timing are assumed to be Gaussian distributed and the 3dB attenuation time band of auto-correlation main lobe is assumed to sit within 3 deviations of such distribution.
[0133] From the above assumptions there is the following:
X = N{mc,sc 2) = N{ timing of x,(^-x autocorr(sL)mainlobe@3dB)2) Y = N(mg , <c2 ) = /V(timing of y , (— x autocorr (sR )mainlobe @ 3dB)2 )
[0134] This gives the final time difference measurement distribution as:
Z = N(m g - mc,sc + sg )
[0135] Hence standard deviation of the observation error for Kalman filtering is:
Figure imgf000034_0001
[0136] As this is a one dimensional Kalman filter, all variables reduce to numerical values. There is:
Xk Xk- 1 Uk
¾ = xk + vk
[0137] Where and z are actual and observed values respectively, u and v are system noise and observation noise respectively. Time update reduces to:
x k = x k
P 1 k = P 1 k- 1
[0138] The measurement update becomes:
Figure imgf000034_0002
Figure imgf000035_0001
[0139] Where xk is the smoothed timing estimation.
[0140] It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure.
[0141] It should be understood that the techniques of the present disclosure might be implemented using a variety of technologies.
[0142] The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

CLAIMS:
1. A method (200) for determining an orientation of an anatomical feature of a user in relation to an electronic device, wherein a wearable device is located relative to the anatomical feature of the user, the wearable device comprising a first transducer and a second transducer, and the electronic device comprising a third transducer, the method comprising: transmitting and receiving (210) a first acoustic signal between the first transducer and the third transducer; transmitting and receiving (220) a second acoustic signal between the second transducer and the third transducer; determining (230) a first received time associated with the first acoustic signal; determining (240) a second received time associated with the second acoustic signal; and determining (250) the orientation of the anatomical feature of the user in relation to the electronic device based on the first and second received times.
2. The method of claim 1 wherein the first received time indicates a time when the first acoustic signal is received by the third transducer and the second received time indicates a time when the second acoustic signal is received by the third transducer.
3. The method of claim 1 or 2, wherein determining the orientation of the anatomical feature of the user is further based on a first time difference and a second time difference, the first time difference being based on a difference between a first time reference and the first received time, wherein the first time reference indicates a time that the first acoustic signal was transmitted; the second time difference being based on a difference between a second time reference and the second received time, wherein the second time reference indicates a time that the second acoustic signal was transmitted.
4. The method of any one of the preceding claims, wherein: the first transducer transmits the first acoustic signal; and the second transducer transmits the second acoustic signal.
5. The method of any one of the preceding claims, wherein the third transducer comprises two or more microphones, the method further comprising: receiving the first and second acoustic signal at the two or more microphones; and processing the first and second acoustic signals received at the two or more microphones to determine a first location and a second location associated with the first and second transducers respectively.
6. The method of claim 5 wherein the at least two microphones form at least one microphone cluster in the electronic device.
7. The method of any one of the preceding claims wherein the first acoustic signal and the second acoustic signal comprise an ultrasonic signal.
8 The method of claim 7, wherein the ultrasonic signal acts as a carrier wave.
9. The method of any one of the preceding claims wherein the first acoustic signal and the second acoustic signals are acoustic leak signals.
10. The method of claim any one of the preceding claims wherein the anatomical feature of the user is a head.
11. The method of any one of the preceding claims wherein the first transducer is associated with a first ear of the user and the second transducer is associated with a second ear of the user.
12 The method of any one of the preceding claims wherein the electronic device is located on the user.
13. The method of any one of the preceding claims wherein the first transducer and the second transducer of the wearable device are located approximately
symmetrical about a central axis associated with the user.
14. The method of claim 12 wherein the electronic device is approximately located on the central axis.
15. A system (100) for determining an orientation of an anatomical feature of a user (110) in relation to an electronic device (120), the system comprising: a wearable device (130) located relative to the anatomical feature of the user (110), the wearable device comprising a first transducer (140) and a second transducer (150); the electronic device (120) comprising a third transducer (160); wherein a first acoustic signal is transmitted and received between the first transducer (140) and the third transducer (160); and wherein a second acoustic signal is transmitted and received between the second transducer (150) and the third transducer (160); a processor (170) configured to: determine a first received time associated with the first acoustic signal; determine a second received time associated with the second acoustic signal; and determine the orientation of the anatomical feature of the user (110) in relation to the electronic device (120) based on the first and second received times.
16. The system of claim 15 wherein the third transducer comprises two or more microphones.
17. The system of claim 16 wherein the at least two microphones form at least one microphone cluster in the electronic device.
PCT/AU2019/050988 2018-10-15 2019-09-13 "a method and system for determining an orientation of a user" WO2020077389A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2021101916A AU2021101916A4 (en) 2018-10-15 2021-04-14 A method and system for determining an orientation of a user

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2018903881A AU2018903881A0 (en) 2018-10-15 A method and system for determining an orientation of a user
AU2018903881 2018-10-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
AU2021101916A Division AU2021101916A4 (en) 2018-10-15 2021-04-14 A method and system for determining an orientation of a user

Publications (1)

Publication Number Publication Date
WO2020077389A1 true WO2020077389A1 (en) 2020-04-23

Family

ID=70282848

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2019/050988 WO2020077389A1 (en) 2018-10-15 2019-09-13 "a method and system for determining an orientation of a user"

Country Status (1)

Country Link
WO (1) WO2020077389A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112383379A (en) * 2020-10-29 2021-02-19 南昌大学 Method and device for calculating node time error of wireless information system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3777305A (en) * 1972-04-10 1973-12-04 Us Navy Ultrasonic angle measurement system
US6535206B1 (en) * 2000-02-18 2003-03-18 Intel Corporation Ultrasonic wireless pen position determination system and method
US20040201857A1 (en) * 2000-01-28 2004-10-14 Intersense, Inc., A Delaware Corporation Self-referenced tracking

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3777305A (en) * 1972-04-10 1973-12-04 Us Navy Ultrasonic angle measurement system
US20040201857A1 (en) * 2000-01-28 2004-10-14 Intersense, Inc., A Delaware Corporation Self-referenced tracking
US6535206B1 (en) * 2000-02-18 2003-03-18 Intel Corporation Ultrasonic wireless pen position determination system and method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
A. R. JIMÉNEZ ET AL.: "Ultrasonic Localization Methods for Accurate Positioning", NEW ACOUSTICS: SELECTED TOPICS II (CSIC, BIBLIOTECA DE CIENCIAS, 29 September 2005 (2005-09-29), pages 147 - 166, XP055702272 *
MATTI KARJALAINEN ET AL.: "HEAD-TRACKING AND SUBJECT POSITIONING USING BINAURAL HEADSET MICROPHONES AND COMMON MODULATION ANCHOR SOURCES", INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, ICASSP 2004, 17 May 2004 (2004-05-17), pages 101 - 104, XP010718415, DOI: 10.1109/ICASSP.2004.1326773 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112383379A (en) * 2020-10-29 2021-02-19 南昌大学 Method and device for calculating node time error of wireless information system
CN112383379B (en) * 2020-10-29 2022-05-20 南昌大学 Method and device for calculating node time error of wireless information system

Similar Documents

Publication Publication Date Title
US10909988B2 (en) Systems and methods for displaying a user interface
JP5814476B2 (en) Microphone positioning apparatus and method based on spatial power density
CN103308889B (en) Passive sound source two-dimensional DOA (direction of arrival) estimation method under complex environment
ES2525839T3 (en) Acquisition of sound by extracting geometric information from arrival direction estimates
US6826284B1 (en) Method and apparatus for passive acoustic source localization for video camera steering applications
KR20210091034A (en) Multiple-source tracking and voice activity detections for planar microphone arrays
CN107211225A (en) Hearing assistant system
Ren et al. A novel multiple sparse source localization using triangular pyramid microphone array
Kraljević et al. Free-field TDOA-AOA sound source localization using three soundfield microphones
Huang et al. Microphone arrays for video camera steering
Saric et al. Acoustic source localization in wireless sensor network
Pourmohammad et al. N-dimensional N-microphone sound source localization
Hoeflinger et al. Passive indoor-localization using echoes of ultrasound signals
Cho et al. Sound source localization for robot auditory systems
WO2020077389A1 (en) &#34;a method and system for determining an orientation of a user&#34;
Huang et al. Spatial localization of sound sources: azimuth and elevation estimation
AU2021101916A4 (en) A method and system for determining an orientation of a user
Zhao et al. A robust real-time sound source localization system for olivia robot
Chen et al. Voicemap: Autonomous mapping of microphone array for voice localization
Aprea et al. Acoustic reconstruction of the geometry of an environment through acquisition of a controlled emission
JP2011188444A (en) Head tracking device and control program
Odom et al. Heading and hydrophone data fusion for towed array shape estimation
US20200169809A1 (en) Wearable beamforming speaker array
Qinqin et al. Study of delay estimation in acoustic source localization based on microphone array
Mosgaard et al. Circular statistics-based low complexity DOA estimation for hearing aid application

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19873361

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19873361

Country of ref document: EP

Kind code of ref document: A1