WO2013022483A1

WO2013022483A1 - Methods and apparatus for automatic audio adjustment

Info

Publication number: WO2013022483A1
Application number: PCT/US2012/024263
Authority: WO
Inventors: Thomas Josephe ZATO
Original assignee: Thomson Licensing
Priority date: 2011-08-05
Filing date: 2012-02-08
Publication date: 2013-02-14

Abstract

Methods and apparatus for audio adjustment are provided by the present principles that allow audio to be output to devices within an environment based on a user's location and orientation. The audio can originate from a source device, such as a portable media device, that the user is holding or from a fixed source. The principles described can provide, for example, audio from a portable media device to be heard on speakers in a room environment, such that the audio is heard in the same perspective regardless of the user's location or orientation. The method and apparatus determine the location and orientation of a user. The location and orientation information is used to adjust the audio levels or mix various audio channels, and assign particular audio channels to particular output devices.

Description

METHODS AND APPARATUS FOR AUTOMATIC AUDIO ADJUSTMENT

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application Serial No.

61/515439, filed August 5, 201 1 , which is incorporated by reference herein in its entirety.

FIELD OF THE INVENTION

The present principles relate to methods and apparatus for automatic adjustment of audio using a position of a user to maintain the user's perspective of the audio. The principles also apply to the user of a portable media device, such as a mobile media device.

BACKGROUND OF THE INVENTION

Portable media devices have recently permeated the marketplace, allowing users access to audio, video and information in virtually any place at any time. These devices include smartphones, tablets, media players, notebooks, laptops, gaming devices, etc.

While operating a portable media device with a video screen such as a tablet

(iPad, Samsung Galaxy, etc.) or a media player (such as an iPod, iTouch, iPhone, cell phone, etc.), a user can watch video content using the video screen. A user will typically wear headphones when watching such video in order to hear the audio associated with the video. Or, one of these devices can be used while playing audio without video.

Using headphones, however, to listen to audio is sometimes a very limiting experience because of the fidelity of such headphones. Moreover, a user can have in their home, or in a similar environment, an entertainment system having high quality audio speakers which are used when the user watches video on a large screen or listening to music in a living room or recreation room. However, the high quality audio speakers have been of little use to the user of a portable media device, who has been limited to the audio from his device speakers or listened through headphones from the device. Similarly, the high quality audio speakers do not reproduce sound with the proper balance and perspective if a listener is moving around his environment. A Front Right channel will sound different, for example, if a user turns in a different direction or stands at an unequal distance from each of the speakers.

The system described herein allows adjustment of audio being played over output devices such as speakers to be heard with the same perspective, regardless of a listener's movements within an environment. The audio can originate from the aforementioned portable media device that a user is holding, or the audio can originate from some other source.

The described principles assign the individual audio channels to particular speakers in a fixed location to maintain a consistent perspective (Right Front, Left Front, Right Rear, Left Rear, for example) of the audio. A system incorporating the principles described herein determines a user's location and orientation within an environment. The audio is adjusted accordingly to follow a user as he moves around such an environment and maintain the appropriate perspective.

SUMMARY OF THE INVENTION

The principles described herein relate to a method and an apparatus for automatic adjustment of audio based on the position of a user. The principles described also provide a solution for transferring audio from a portable media device to improved, high quality output devices with the audio's original perspective within an environment.

According to one aspect of the present principles, there is provided a method for transferring audio channels to output devices. The method comprises the steps of determining a location of a user, and determining the orientation of the user. The method also comprises a step of assigning at least one audio channel to at least one output device based on the location and orientation of the user for subsequent output on a corresponding output device.

According to another aspect of the present principles, there is provided an apparatus for transferring audio channels to output devices. The apparatus is comprised of circuitry to determine a location of a user and circuitry to determine an orientation of the user. The apparatus is further comprised of a processor for assigning at least one of a plurality of audio channels to at least one output device based on the location and orientation of the user for subsequent output on a corresponding output device.

These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which are to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Figures 1 a, b, and c show various configurations of audio speakers and a user of a portable media device.

Figure 2 shows a flow chart illustrating an exemplary embodiment of the present principles.

Figure 3 shows a flow chart illustrating another exemplary embodiment of the present principles.

Figure 4 shows a block diagram of an exemplary embodiment of the present principles.

Figure 5 shows a block diagram of another exemplary embodiment of the present principles. DETAILED DESCRIPTION

The principles described herein relate to automatic audio adjustment. Particularly, an object of the principles described herein is to allow a listener to hear audio played out on speakers in the listener's environment, from the same perspective, regardless of his location and orientation. Another object of the principles described herein is to allow a user of a portable media device to hear audio from his portable device played out on speakers in the user's environment, from the same perspective, regardless of his location and orientation. The audio can correspond to video played on the portable media device, or the user can simply be listening to audio without video. The speakers in a home are usually higher quality than speakers in the user's portable device or headphones, and the present principles allow a user to move around within an environment, and his perspective remains unchanged. Figures 1 a-c show various exemplary configurations illustrating the operation of a system incorporating the principles described herein. In one exemplary environment, there are four speakers (105, 1 10, 1 15, and 120) where audio can be played out. In addition, there is an audio receiver 150 which controls the audio that is played by the speakers (105, 1 10, 1 15, and 120). Audio receiver 150 is enabled with a wireless interface (RF, Bluetooth, 802.1 1 a, b, g, n s, for example) which receives audio information from a portable media device that is being operated by a user 130. The portable media device can have, for example, both a GPS detector and a gyroscope or other such devices that can determine the location and orientation of a user, and where such information can be transmitted to the audio receiver. Other more sophisticated methods can use optical means for detection of position of the listener's eyes. Another means is voice detection of the listener to determine the listener's direction. For the present example, the orientation information can be presented in degrees, although other units for location information can be used.

In this example, the configuration uses four channels of audio which will be output over the speakers. Under the principles described herein, however, it is understood that the system can support more than four channels if desired. The channels in this example are designated as Front Right, Front Left, Rear Right, and Rear Left. When the audio receiver receives the location and orientation information from the portable media device, the audio receiver can adjust the audio output based on the orientation of a user of the device. Specifically, the location information of a user can be used to determine whether the user is within an environment, and where he is within that environment, which contains speakers. The orientation information can then be used to automatically select which speakers are used for outputting specific audio channels.

In Figure 1 (a), a user of a media device is located within an environment and has an orientation of 0 degrees. In this case, speaker 105 outputs the "Front Left" audio, speaker 1 15 outputs the "Front Right" audio, speaker 1 10 outputs the "Rear Left" channel, and speaker 120 outputs the "Rear Right" audio.

The output audio will then change when audio receiver 150 receives orientation information in Figure 1 (b), indicating that user 130 changes their orientation to 270 degrees. In this example, the speaker 1 10 outputs the "Front Left" audio, speaker 105 outputs the "Front Right" audio, speaker 120 outputs the "Rear Left" channel, and speaker 1 15 outputs the "Rear Right" audio.

The output audio will then change again when audio receiver 150 receives orientation information as in Figure 1 (c), that user 130 changes their orientation to 180 degrees. In this example, speaker 120 outputs the "Front Left" audio, speaker 1 10 outputs the "Front Right" audio, speaker 1 15 outputs the "Rear Left" channel, and speaker 105 outputs the "Rear Right" audio. In another case, for example, the user 130 is directly facing speaker 1 10 at an angle of 225 degrees, a more sophisticated mixture of channels can be done where speaker 105 would produce the "Front Right" channel and speaker 120 would product the "Front Left" channel, with speaker 1 10 producing some combination of "Front Left" and "Front Right" channels and speaker 1 15 producing some combination of "Rear Left" and "Rear Right". The selection of the speaker for "Front Right" and "Front Left", while determined by the angle of the listener, can be blended smoothly as the listener's angle is changed such that there is not an abrupt shift in the speaker selection. The exact algorithm for blending and smoothing these transitions can be varied. It is usually desirable to include some hysteresis as the listener rotates his/her position to also contribute to a smoother transition of speakers.

Figure 2 shows a high level flow diagram for one exemplary method 200 for transferring audio from a source device, such as a portable media device, for example, to output devices, such as speakers, within an environment that keeps the original perspective of the audio. The method is comprised of a step 205 for determining a user's location information. This is helpful in determining, for example, whether a user is located in a specific room in a house or other building. User location can be important if multiple rooms in a house have speakers (such as a bedroom and a living room), and the user walks around to different rooms in the house. The location information can also be used to determine a specific location within the room or environment. In step 210, the orientation of a user is determined. This step can be performed in a number of ways, for example, by using the gyroscope information that can be determined by a portable media device. That is, many portable media devices can determine the orientation of a user, although other means such as visual identification, for example, can also be used in performing either step 205 or 210.

If it is determined in step 205 that a user is located in an environment with speakers, for example, and the user's orientation is communicated in step 210 to an audio device that controls the outputting of audio channels to output devices, then step 215 is performed. Specifically, there is a determination as to which speakers should be used for each particular audio channel from the portable media device. This can be done in a manner similar to the described arrangement in Figures 1 a-c with an audio receiver that obtains both the audio channels and location/orientation information about a user from a portable media device. An appropriate speaker for each audio channel is selected in step 215 based on the orientation of the user, for subsequent play out on the appropriate speakers.

In a more generalized version of the foregoing embodiment, the power levels of the audio channels can be weighted and the channels can be mixed in varying proportions to form modified audio channels that are output to the individual speakers. The audio levels can be adjusted, for example, to account for the user's distance to each of the speakers. With this additional feature, if the user is closer to some of the speakers, the audio level to those speakers can be reduced, for example, while increasing the levels from the far speakers, such that the audio has a consistent sound regardless of the user's position within the environment. This embodiment allows a virtually continuous sound experience regardless of the user's movements throughout an environment.

This exemplary embodiment using the present principles is shown in Figure 3, which shows a method 300 for transferring audio channels to output devices. In this case, a user has changed his location and orientation within an environment, such that he is facing a particular direction, and his distance from each of the speakers in the environment is unequal. The method is comprised of a step 305 for determining the user's location. The location information can be used to determine a specific location within a room or environment. In step 310, the orientation of a user is determined. This step can be performed in a number of ways, for example, by using the gyroscope information that can be determined by a portable media device, although other means such as visual identification can also be used in performing either step 305 or 310.

If it is determined in step 305 that a user is located in an environment with speakers, and the user's orientation is communicated to an audio device that controls the outputting of audio channels to output devices in step 310, then step 315 is performed. There is a determination in step 315 as to which audio channels should be used, at what levels, and in what combinations for subsequent output to individual speakers, based upon the orientation and location of the user. Under the present principles, the weighting and mixing of audio channels to allow the listener to hear the audio in any desired proportion is performed in step 320 of Figure 3, for subsequent play out to individual speakers.

A further example of the present principles is a case in which a mixture of speakers are used in the system each speaker having different power capabilities. A power balancing can be performed to allow a more powerful speaker to be scaled appropriately to match a weaker speaker. As an example, speakers 105 and 1 15 can be premium speakers that provide Front Left and Front Right audio tracks for a movie that is viewed on a screen behind the Audio Receiver 150. The user can reposition himself to an orientation shown in Figure 1 b when just listening to music, facing speakers 105 and 1 10. In this case, speaker 1 10 may be a weaker speaker set up originally as a Rear channel speaker for a surround sound video environment. For the music-only experience, in the orientation of Figure 1 b, speaker 1 10 may be

compensated differently for use as a Left channel audio such that its acoustic characteristics are better matched to be used when speaker 105 is used as a Right channel audio speaker for the intended music listening session. The processing adjustment would again be included in a processing block after block 215 shown in Figure 2, or as a sub-block of step 320 in Figure 3.

The audio channel adjustment techniques described herein can also be applied to a Voice Over IP (VOIP) or video conference application in which audio adjustment to the speakers may take into account, not only the position information of the listener, but also the location information of the person speaking, or presenting, on the other end of the audio connection. In this way, two similar systems providing a VOIP or video conferencing session between them, can continually compensate their audio output parameters for each individual speaker based on location information of the listener in addition to the location information of the presenter. If the presenter happens to move away from his microphone, the audio level can be automatically adjusted so that the listener's perception remains as if the presenter was still near the microphone. The audio level for this situation can be adjusted, either through modification of the audio output level, or modification of the microphone level. These methods should be considered within the scope of the present principles.

Another embodiment of the present principles is shown in Figure 4, which shows an apparatus 400 for transferring audio channels to output devices. The apparatus is comprised of circuitry 410 to determine a location of a user and circuitry 420 to determine an orientation of the user. The location and orientation can be determined in a number of ways, including, but not limited to, receiving wireless data from a device that the user is holding or manual information sent to circuitry 410 and 420. The apparatus if further comprised of a processor 430, whose first and second inputs are in signal communication with the outputs of circuitry 410 and circuitry 420, respectively. Processor 430 assigns at least one of a plurality of audio channels to at least one output device based on the location and the orientation of the user. The output of processor 430 is one or more audio channels directed to the appropriate output device. The audio channels are subsequently output on one or more of the output devices corresponding to their assignments. It should be understood that, under the present principles, processor 430 can also weight the various input audio channels or combine the audio channels in varying proportions before assigning the weighted or mixed output channels to particular output devices.

Another embodiment of the present principles is shown in Figure 5, which shows an apparatus 500 for transferring audio channels to output devices. In this embodiment, a user is holding a portable media device. The apparatus is comprised of circuitry 510 to determine a location of a user and circuitry 520 to determine an orientation of the user. Location and orientation information is sent, perhaps wirelessly, to circuitry 510 and 520 from the portable media device. The apparatus if further comprised of

Processor 530 whose first and second inputs are in signal communication with the outputs of circuitry 510 and circuitry 520, respectively. Processor 530 assigns at least one of a plurality of audio channels to at least one output device based on the location and the orientation of the user. The audio channels are subsequently output on one or more output devices corresponding to their assignments.

One or more implementations having particular features and aspects of the presently preferred embodiments of the invention have been provided. However, features and aspects of described implementations can also be adapted for other implementations. For example, these implementations and features can be used in the context of other video devices or systems. The implementations and features need not be used in a standard.

Reference in the specification to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation", as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

The implementations described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or computer software program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

Implementations of the various processes and features described herein can be embodied in a variety of different equipment or applications. Examples of such equipment include a web server, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment can be mobile and even installed in a mobile vehicle.

Additionally, the methods can be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) can be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact disc, a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions can form an application program tangibly embodied on a processor-readable medium. Instructions can be, for example, in hardware, firmware, software, or a combination. Instructions can be found in, for example, an operating system, a separate application, or a combination of the two. A processor can be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium can store, in addition to or in lieu of instructions, data values produced by an implementation.

As will be evident to one of skill in the art, implementations can use all or part of the approaches described herein. The implementations can include, for example, instructions for performing a method, or data produced by one of the described embodiments.

A number of implementations have been described. Nevertheless, it will be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes can be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this disclosure and are within the scope of this disclosure.

Claims

CLAIMS:

1. A method of transferring audio channels to output devices, comprising: determining a location of a user;

determining an orientation of said user;

assigning at least one of a plurality of audio channels to at least one output device based on said location and said orientation for subsequent output on a corresponding output device.

2. The method of Claim 1 , said determining steps further comprising receiving location information and orientation information from a portable media device being used by the user.

3. The method of Claim 1 , said assigning step comprises assigning at least one of a plurality of audio channels from a portable media device to at least one output device.

4. The method of Claim 1 , said assigning step comprises assigning said at least one audio channel to an output device comprising a speaker.

5. The method of Claim 1 , said assigning step comprises assigning the at least one of the plurality of audio channels to the at least one output device based on said location and said orientation such that the user maintains an original perspective of the audio channels.

6. The method of Claim 1 , said assigning step further comprising:

assigning audio channels from a source in a different location than the user.

7. The method of Claim 1 , further comprising weighting at least one of the plurality of audio channels based on said location and said orientation.

8. The method of Claim 1 , further comprising weighting and combining at least one of the plurality of audio channels based on said location and said orientation.

9. The method of Claim 1 , further comprising:

receiving location information and orientation information of all users in a videoconference; and

adjusting said audio channels based on said location and orientation.

10. The method of Claim 2, further comprising receiving location information and orientation information wirelessly from said portable media device.

11. An apparatus for transferring audio channels to output devices, comprising:

circuitry to determine a location of a user;

circuitry to determine an orientation of said user;

a processor for assigning at least one of a plurality of audio channels to at least one output device based on said location and said orientation for subsequent output on a corresponding output device.

12. The apparatus of Claim 1 1 , further comprising:

a receiver for receiving location information and orientation information from a portable media device being used by the user.

13. The apparatus of Claim 1 1 , said processor comprising circuitry to receive at least one of a plurality of audio channels from a portable media device for subsequent assignment to at least one output device.

14. The apparatus of Claim 1 1 , said processor assigning said at least one audio channel to an output device comprising a speaker.

15. The apparatus of Claim 1 1 , said processor assigning said at least one of the plurality of audio channels to at least one output device based on said location and said orientation so that the user maintains an original perspective of the audio channels.

16. The apparatus of Claim 1 1 , said processor further comprising circuitry for receiving audio channels from a source in a different location than the user.

17. The apparatus of Claim 1 1 , further comprising a weighting circuit for weighting at least one of the plurality of audio channels based on said location and said orientation.

18. The apparatus of Claim 1 1 , said processor further comprising a mixing circuit for weighting and combining a plurality of audio channels based on said location and said orientation.

19. The apparatus of Claim 1 1 , further comprising:

a receiver for location information and orientation information of a plurality of users in a videoconference; and

said processor assigning said audio channels based on said location information and said orientation information of said plurality of users.

20. The apparatus of Claim 12, further comprising a wireless receiver for location information and orientation information from said portable media device.