WO2008119122A1

WO2008119122A1 - An acoustically transparent earphone

Info

Publication number: WO2008119122A1
Application number: PCT/AU2008/000453
Authority: WO
Inventors: Jonathon Wolfe; Andre Van Schaik; Craig Jin; Simon Carlile
Original assignee: Personal Audio Pty Ltd
Priority date: 2007-03-30
Filing date: 2008-03-28
Publication date: 2008-10-09

Abstract

Equipment (10) for the delivery of augmented-reality audio includes an acoustically transparent ear piece (12) receivable within at least one outer ear (20) of a listener, the acoustically transparent ear piece (12) permitting the passage of sounds originating from a real sound source. An acoustic actuator (14) is connected to the acoustically transparent ear piece (12). A signal processor (16) is in communication with the acoustic actuator (14) for delivering virtual spatial audio to the listener through the ear piece (12).

Description

"An acoustically transparent earphone" Cross-Reference to Related Applications

The present application claims priority from Australian Provisional Patent Application No 2007901714 filed on 30 March 2007, the contents of which are incorporated herein by reference.

Field of the Invention

The present invention relates, generally, to an acoustically transparent earphone for the delivery of augmented-reality audio. More particularly, the invention relates to a method for, and a system of, delivering augmented-reality audio for rendering virtual spatial audio concurrently with a real sound environment so that a listener is able clearly to perceive both the real sound source and virtual sound source as localised within the listener's personal auditory space.

Background to the Invention

The Applicant is aware of various techniques for producing virtual spatial audio as well as for delivering electronically produced audio to a listener's ears such that the real sound environment remains audible and accessible, but not necessarily properly spatialised, when the electronically produced audio is played back. Further, use is generally made of closed earphone systems which occlude the ear canal. Reliance is then placed on the use of microphones and speaker systems for detecting real sound sources and transmitting the real sound sources to the listener's ears. A disadvantage of this arrangement is that the listener suffers an occlusion effect that makes sounds generated in the listener's body appear louder. A further disadvantage of this arrangement is that there will be leakage of the external sound through the earpiece and a delay between the detection of the sound by the microphone and the playback via the speaker. The combination of the leakage component and the delayed playback component results in an addition of the sound produced by the real source with a delayed version of itself which, in turn, results in audible distortion. Examples of such closed earphones include circumaural headphones which fit around a listener's outer ear or ear-bud headphones that fit within the ear. These items are referred to as "closed earphones" as they significantly occlude the sound path to the ear and substantially distort the external sound input to the ear. As indicated above, these types of earphones can reduce the quality of an external sound field that is heard by the listener. Yet other techniques for producing virtual spatial audio use "open" earphones. Some of these open earphones provide a low acoustic impedance seal (a partial seal) around the outer ear thereby producing less occlusion for external, real sounds than closed earphones. However, a disadvantage of these techniques is that the low acoustic impedance seal can still distort the acoustic filtering of the outer ear. This distortion disrupts the normal acoustic spatialisation cues associated with outer ear filtering and results in poor acoustic spatialisation of external sound sources.

Summary of the Invention According to a first aspect of the invention, there is provided equipment for the delivery of augmented-reality audio, the equipment including: an acoustically transparent ear piece receivable within at least one outer ear of a listener, the acoustically transparent ear piece permitting the passage of sounds originating from a real sound source; an acoustic actuator connected to the acoustically transparent ear piece; and a signal processor in communication with the acoustic actuator for delivering virtual spatial audio to the listener through the ear piece.

Preferably, to provide authentic virtual spatial audio to the listener, the equipment includes a pair of ear pieces, one for each ear, to effect binaural augmented- reality audio. Further, to ensure that the ear piece is accurately positioned in each ear of the listener, each ear piece may be one of a dedicated right ear piece and a left ear piece, as the case may be.

Each ear piece may be configured so that the ear-filtering function of the listener's outer ear is minimally disrupted. Further, each ear piece may be configured to maximise frequency response and, more particularly, low frequency response of the combination of the acoustic actuator and the ear piece so that low frequency signals can be passed to the listener's ear canal.

Each ear piece may include an anchoring formation for anchoring in position relative to a pinna of the ear. More particularly, the anchoring formation may engage the ear concha in a manner which causes minimal disruption to the filtering function of the pinna. Thus, the anchoring formation may be in the form of a ring-like element of a flexible material which seats in the concha of the listener's ear. A tubular member may project from the anchoring formation to be at least partially received within the ear canal, the tubular member being dimensioned so that it is a loose fit within the ear canal to allow the passage of sounds from a real source to pass through the tubular member into the ear canal. Preferably, the tubular member is integrally formed with the anchoring formation as a one-piece unit of a suitable, resiliency flexible synthetic plastics material.

The tubular member may define an open passageway with the acoustic actuator being associated with the open passageway. Preferably, the acoustic actuator is inserted into an operatively outer end of the open passageway to pass sounds to the tubular member. In another embodiment, the acoustic actuator could be built into the tubular member.

The equipment may include, or be connected to, an auxiliary audio source, the auxiliary audio source being the source of the audio to be delivered to the listener as virtual spatial audio. It will be appreciated that the auxiliary audio source could comprise any suitable device such as, for example, a mobile telephone or other telecommunications device, an MP3 player or other music playing device, or the like.

The signal processor may be operable to process auxiliary audio signals output by the auxiliary audio source to render the audio signals as virtual spatial audio. The signal processor may include a database of head related transfer functions

(HRTFs), the HRTFs being used for rendering the auxiliary audio signal as virtual spatial audio. Thus, the signal processor may be operable to convert the auxiliary audio signal, using the stored HRTFs, into virtual spatial audio for delivery to each ear piece of the equipment. It will be appreciated that, to improve the virtual audio quality, the HRFTs for the left ear and the right ear may differ. With this configuration, the auxiliary audio transmitted to the listener as virtual spatial audio may be rendered for the perception of the listener at some phantom or virtual location in space around the listener.

Further, the signal processor, via inputs from the listener or from some other source, is operable to manipulate various characteristics of the virtual spatial audio. These characteristics may include, but are not limited to, the spatial location or a motion trajectory of an object in extra-personal space, the loudness of the auditory object, its timbre or the ambient acoustic characteristics associated with the virtual environment of an audio object such as, for example, reverberance. If desired, the equipment may include a controller for enabling the listener to control operation of at least the signal processor to adjust the virtual spatial audio delivered to the listener.

Still further, the equipment may include a power supply. The power supply may be a battery-operated power supply. To provide a protective function, the equipment may further include a cap member which may be placed on the tubular member to occlude the passageway of the tubular member and the ear canal in situations where the listener may be exposed to adverse natural sound conditions.

The equipment may include at least one microphone for measuring ambient sound. By measuring ambient sounds a reverberation quality of the listener's environment may be obtained which enables virtual spatial audio to be generated having a reverberation which is congruent with the ambient sound environment of the listener. Instead, or in addition, the at least one microphone may be used to perform beamforming to improve a signal-to-noise ratio of soft real sounds for the listener. By playing the beamformed signal back as virtual spatial audio this may enhance the listener's hearing in difficult situations.

The equipment may include a microphone associated with each ear piece. Each microphone may be mounted in its associated ear piece.

The signal processor may be operable to compute the ambient reverberation of the listener's environment to generate, along with the database of HRTFs, the virtual spatial audio that has a reverberation matching the listener's environment and a direction that matches the target direction for the source of the virtual spatial audio.

This feature may be selectively operable by the listener.

It will be appreciated that the microphones are not used to pass ambient sound directly through to the listener's ear. According to a second aspect of the invention, there is provided a method for delivering augmented-reality audio, the method including: mounting an acoustically transparent ear piece within at least one outer ear of a listener, the acoustically transparent ear piece permitting the passage of sounds originating from a real sound source; connecting an acoustic actuator to the ear piece; and delivering virtual spatial audio to the listener's ear via the acoustic actuator and the acoustically transparent ear piece.

Preferably, the method includes mounting an acoustically transparent ear piece in each ear to effect binaural augmented-reality audio. The method may include configuring and mounting each ear piece to maximise frequency response so that low frequency signals can be passed to the listener's ear canal.

The method may include mounting each ear piece in a manner which causes minimal disruption to the filtering function of the listener's outer ear. Each piece may include a tubular member to be at least partially received within the ear canal, and the method may include allowing the passage of sounds from a real source to pass through the tubular member into the ear canal. The method may include processing auxiliary audio signals output by an auxiliary audio source to render the audio signals as virtual spatial audio.

Thus, the method may include accessing a database of head related transfer functions (HRTFs) and using the HRTFs for rendering the auxiliary audio signal as virtual spatial audio for delivery to each ear piece.

Further, the method may include manipulating various characteristics of the virtual spatial audio via inputs from the listener or from some other source.

If desired, the method may include enabling the listener to adjust the virtual spatial audio delivered to the listener. The method may include measuring ambient sounds to obtain a reverberation quality of the listener's environment which enables virtual spatial audio to be generated having a reverberation which is congruent with the ambient sound environment of the listener. Instead, or in addition, the method may include performing beamforming to improve a signal-to-noise ratio of soft real sounds for the listener.

Brief Description of the Drawing

The invention is now described by way of example with reference to the accompanying diagrammatic drawing which shows a schematic representation of equipment, in accordance with an embodiment of the invention, for the delivery of augmented-reality audio.

Detailed Description of Exemplary Embodiment

In the drawings, reference numeral 10 generally designates equipment, in accordance with an embodiment of the invention, for the delivery of augmented-reality audio. The equipment 10 includes a pair of acoustically transparent ear pieces 12.1 and 12.2, referred to collectively as the ear pieces 12. The equipment 10 further includes an acoustic actuator 14 associated with each ear piece 12.1, 12.2. Each acoustic actuator 14 is in communication with a signal processor 16.

In the embodiment of the invention illustrated, the signal processor 16 communicates with the acoustic actuators 14 via electrical leads 18. However, it will be appreciated that the signal processor 16 could communicate wirelessly with the actuators 14, for example, by infrared technology, Bluetooth technology, or the like.

Each ear piece 12.1, 12.2 is configured to minimally disrupt the ear filtering characteristics of a pinna, or an outer ear, 20 of a listener's ear 22. For this purpose, each ear piece 12.1, 12.2 includes an anchoring formation in the form of a ring-like element 24 which fits snugly in the concha 26 of the listener's ear 22. The ring-like element 24 is of a flexible material and provides for minimal disruption of the ear filtering function of the listener's outer ear 20.

A tubular member 28 projects from the ring-like element 24 to be received at least partially in an ear canal 30 of the listener's ear 22. As illustrated on an exaggerated scale, the tubular member 28 is of a smaller diameter than the diameter of the ear canal 30 so that sounds emanating from a real sound source can pass around the tubular member 28 into the ear canal 30. The tubular member 28 is formed integrally with the ring-like element 24 as a one-piece unit and is of a resiliency flexible, synthetic plastics material. Each tubular member 28 defines an open passageway through it. The acoustic actuator 14 is in communication with the passageway of its associated tubular member 28 to pass virtual audio to the tubular member 28 for delivery to the ear canal 30 of the listener.

The signal processor 16 comprises a data processor 32. An audio signal, output on a line 34 from an auxiliary audio source or device 36 is fed to the processor 32 for manipulation by the processor 32 to provide virtual spatial audio to the listener.

The signal processor 16 further comprises a database 38, in communication with the processor 32, the database 38 including a database of head related transfer functions (HRTFs). These HRTFs are generated in accordance with the applicant's method as described in the Applicant's International Patent Application having Publication No. WOO 1/54453 entitled "The generation of customised three-dimensional sound effects for individuals", the contents of which are incorporated in this specification by reference. Using these techniques, the sound output by the auxiliary audio device 36 is, after processing by the signal processor 16, delivered to the listener so that the auditory object sourced from the device 36 is rendered for the perception of the listener at some phantom or virtual location or locations in space around the listener.

The auxiliary audio device 36 could be any suitable device which emits audio signals such as, for example, a mobile telephone or other telecommunications device, an MP3 music player or other similar music playing device, or the like. The signal processor 16 may also take an input from the listener or from some other source in regard to how the signal should be rendered so as to generate various characteristics of the auditory object. Thus, the signal processor includes a controller 40 which is controllable by the listener to alter the virtual spatial audio presented to the listener. The characteristics of the auditory object which can be changed by the listener include, but are not limited to, the spatial location or the motion trajectory of the auditory object in extra-personal space, the loudness of the auditory object, its timbre or the ambient acoustic characteristics associated with the environment of the object such as reverberance.

The signal processor 16 further performs the filtering required to render the sounds for the right and left ear pieces 12.1 and 12.2, respectively. The equipment 10 includes a power supply 42 which provides power for the equipment 10 and, more particularly, the signal processor 16 and the acoustic actuators 14. The power supply 42 is, preferably, a battery power supply and ideally relies on rechargeable batteries.

Further, the equipment 10 include a microphone 44, one mounted in each ear piece 12, for measuring ambient sound. By measuring ambient sounds a reverberation quality of the listener's environment is obtained which enables virtual spatial audio to be generated having a reverberation which is congruent with the ambient sound environment of the listener. Instead, or in addition, each microphone 44 is used to perform beamforming to improve a signal-to-noise ratio of soft real sounds for the listener. By playing the beamformed signal back as virtual spatial audio this enhances the listener's hearing in difficult situations.

The signal processor 16 is operable to compute the ambient reverberation of the listener's environment to generate, along with the database of HRTFs, the virtual spatial audio that has a reverberation matching the listener's environment and a direction that matches the target direction for the source of the virtual spatial audio. This feature may be selectively operable by the listener using the controller 40. The microphones are not used to pass ambient sound directly through to the listener's ear 20.

In situations where the protection of hearing is paramount, the equipment 10 optionally includes plugs (not shown) which can be used to occlude the tubular members 28 and the ear canals 30 of the listener's ears 22.

In use, using a mobile telephone as an example of the audio device 36, the mobile telephone 36 is connected to the signal processor 16 of the equipment 10. The ear pieces 12.1 and 12.2 are placed in the right and left ears 22 of the listener, respectively, by, in each case, mounting the anchoring formation 24 in the concha 26 of its associated ear 22. Each ear piece 12.1 and 12.2 is arranged so that its tubular member 28 extends at least partially into the ear canal 30 of the ear 22. The acoustic actuators 14 are connected electrically to the signal processor 16.

With this arrangement, sound from a real source is passed by the acoustically transparent ear pieces 12.1 and 12.2 into the ear canals 30 of the listener. The sound from the real source is passed with minimal disruption to the filtering structures of the outer ear 20. In addition, an audio signal emanating from the mobile telephone 36 is processed by the signal processor 16 and is output to the acoustic actuators 14. The acoustic actuators 14 transmit the sounds into the ear canal 30. The audio output by the mobile telephone 36 is filtered by the HRTFs 38 which mimic the filtering function of the outer ears 20 of the listener so that the sound is perceived as being located externally of the listener to provide the virtual spatial audio.

Hence, it is an advantage of the invention that, by means of the equipment 10, sounds from both a real source and from an auxiliary audio device 36 can be appreciated by a listener simultaneously and with high fidelity. On the one hand, sounds from a real external sound source are passed without distortion through the acoustically transparent ear pieces 12. Audio from the auxiliary audio device 36 is rendered to the listener as virtual spatial audio using the Applicant's techniques of virtual audio space. The virtual spatial audio is combined with the real sound source by the signal processor 16 and the ear pieces 12. Thus, high fidelity rendering of the virtual spatial audio can be appreciated by the listener in conjunction with sounds from the real sound source. A particular advantage is that the equipment 10 is provided which minimally disrupts the outer ear filtering function of a listener's ears to enable real sounds to be heard authentically. A further advantage is that no microphones and no signal processing of the real sounds are needed, resulting in power savings and avoiding the disruptive combination of direct sound with a delayed version of itself. A further advantage is that the occlusion effect is avoided by using an ear piece that minimally disrupts the outer ear filtering function of a listener's ears.

It is another advantage of the invention that the equipment provides for the introduction of rendered signals from a variety of sources. This can include, but is not limited to, a mobile telephone or other communications device, voice or other sound interfaces from intelligent devices and other sounds from entertainment devices. The link to the signal processor 16 also allows the listener to control the characteristics of the inputs such as, but not limited to, the virtual location, loudness and tonal quality of the virtual spatial audio. The virtual spatial audio is provided to the listener in a form in which it cannot be heard by other listeners near the listener using the equipment. Generally, a listener is able to parse or separate multiple sources of information based on differences in their locations in space around the listener. This is referred to as the "cocktail party effect". The equipment enables the listener to listen to information received via the acoustic actuators and to deal with that information or to ignore it, as the case may be. It is still another advantage of the invention that the auxiliary audio device can be used to render information where the location of a particular sound may convey information at the same time as the listener is engaged in other activities. An example of such an application is in providing verbal or other spatial directions using a navigational guidance system for buildings or other terrain.

It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the invention as shown in the specific embodiments without departing from the scope of the invention as broadly described.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

CLAIMS:

1. Equipment for the delivery of augmented-reality audio, the equipment including: an acoustically transparent ear piece receivable within at least one outer ear of a listener, the acoustically transparent ear piece permitting the passage of sounds originating from a real sound source; an acoustic actuator connected to the acoustically transparent ear piece; and a signal processor in communication with the acoustic actuator for delivering virtual spatial audio to the listener through the ear piece.

2. The equipment of claim 1 which includes a pair of ear pieces, one for each ear, to effect binaural augmented-reality audio.

3. The equipment of claim 2 in which each ear piece is one of a dedicated right ear piece and a left ear piece.

4. The equipment of claim 2 or claim 3 in which each ear piece is configured so that the ear-filtering function of the listener's outer ear is minimally disrupted.

5. The equipment of any one of claims 2 to 4 in which each ear piece is configured to maximise frequency response so that low frequency signals can be passed to the listener's ear canal.

6. The equipment of any one of claims 2 to 5 in which each ear piece includes an anchoring formation for anchoring in position relative to a pinna of the ear.

7. The equipment of claim 6 in which the anchoring formation engages the ear concha in a manner which causes minimal disruption to the filtering function of the pinna.

8. The equipment of claim 6 or claim 7 in which a tubular member projects from the anchoring formation to be at least partially received within the ear canal, the tubular member being dimensioned so that it is a loose fit within the ear canal.

9. The equipment of claim 8 in which the tubular member defines an open passageway with the acoustic actuator being associated with the open passageway.

10. The equipment of any one of the preceding claims in which the signal processor is operable to process auxiliary audio signals output by an auxiliary audio source to render the audio signals as virtual spatial audio.

11. The equipment of claim 11 in which the signal processor includes a database of head related transfer functions (HRTFs), the HRTFs being are used for rendering the auxiliary audio signal as virtual spatial audio.

12. The equipment of claim 11 in which the signal processor, via inputs from the listener or from some other source, is operable to manipulate various characteristics of the virtual spatial audio.

13. The equipment of any one of the preceding claims which includes a controller for enabling the listener to control operation of at least the signal processor to adjust the virtual spatial audio delivered to the listener.

14. The equipment of any one of the preceding claims which includes at least one microphone for measuring ambient sound.

15. The equipment of claim 14 which includes a microphone associated with each ear piece.

16. A method for delivering augmented-reality audio, the method including: mounting an acoustically transparent ear piece within at least one outer ear of a listener, the acoustically transparent ear piece permitting the passage of sounds originating from a real sound source; connecting an acoustic actuator to the ear piece; and delivering virtual spatial audio to the listener's ear via the acoustic actuator and the acoustically transparent ear piece.

17. The method of claim 16 which includes mounting an acoustically transparent ear piece in each ear to effect binaural augmented-reality audio.

18 The method of claim 17 which includes configuring and mounting each ear piece to maximise frequency response so that low frequency signals can be passed to the listener's ear canal.

19. The method of claim 17 or claim 18 which includes mounting each ear piece in a manner which causes minimal disruption to the filtering function of the listener's outer ear.

20. The method of any one of claims 17 to 19 in which each ear piece includes a tubular member to be at least partially received within the ear canal, and in which the method includes allowing the passage of sounds from a real source to pass through the tubular member into the ear canal.

21. The method of any one of claims 17 to 20 which includes processing auxiliary audio signals output by an auxiliary audio source to render the audio signals as virtual spatial audio.

22. The method of claim 21 which includes accessing a database of head related transfer functions (HRTFs) and using the HRTFs for rendering the auxiliary audio signal as virtual spatial audio for delivery to each ear piece.

23. The method of claim 22 which includes manipulating various characteristics of the virtual spatial audio via inputs from the listener or from some other source.

24. The method of any one of claims 16 to 23 which includes enabling the listener to adjust the virtual spatial audio delivered to the listener.

25. The method of any one of claims 16 to 24 which includes measuring ambient sounds to obtain a reverberation quality of the listener's environment which enables virtual spatial audio to be generated having a reverberation which is congruent with the ambient sound environment of the listener.

26. The method of any one of claims 16 to 25 which includes performing beamforming to improve a signal-to-noise ratio of soft real sounds for the listener.