WO2022010453A1

WO2022010453A1 - Cancellation of spatial processing in headphones

Info

Publication number: WO2022010453A1
Application number: PCT/US2020/040901
Authority: WO
Inventors: Sunil Bharitkar; Mithra VANKIPURAM
Original assignee: Hewlett Packard Development Co LP
Current assignee: Hewlett Packard Development Co LP
Priority date: 2020-07-06
Filing date: 2020-07-06
Publication date: 2022-01-13
Anticipated expiration: 2023-01-06

Abstract

In example implementations, an apparatus is provided. The apparatus includes a communication interface, a correlation analysis device, and a processor. The communication interface is to receive an audio signal from headphones connected to the communication interface. The correlation analysis device is to detect spatial processing applied to the audio signal from the headphones and to identify a spatial filter applied by the headphones to perform the spatial processing. The processor is to apply a correction filter to cancel the spatial processing applied by the spatial filter of the headphones and apply a spatial filter of a host to the audio signal.

Description

CANCELLATION OF SPATIAL PROCESSING IN HEADPHONES

BACKGROUND

[0001] Audio signals from a source may be processed to create a more enjoyable experience for a user. For example, audio signals from a movie or video game may be processed to provide a surround sound experience. For example, speakers may be placed around a user, and the audio signals may be processed into different channels to be outputted by a respective speaker around the user to create the surround sound experience.

[0002] In some instances, a user may wear headphones to listen to the audio signals. Headphones may not have the ability to output audio signals in several different speakers. Rather, headphones output audio into a left channel and a right channel. However, some headphones may perform spatial processing on the headphones to make it sound like an audio signal is coming from a particular direction.

BRIEF DESCRIPTION OF THE DRAWINGS [0003] FIG. 1 is a block diagram of an example system to cancel spatial processing in headphones of the present disclosure;

[0004] FIG. 2 is a block diagram of example apparatus to cancel spatial processing in headphones of the present disclosure;

[0005] FIG. 3 is another block diagram of an example apparatus to cancel spatial processing in headphones of the present disclosure;

[0006] FIG. 4 is a block diagram of an example of how the spatial processing in the headphones is cancelled of the present disclosure;

[0007] FIG. 5 is a flow diagram of an example method for cancelling spatial processing in headphones; and

[0008] FIG. 6 is an example non-transitory computer readable storage medium storing instructions executed by a processor to cancel spatial processing in headphones of the present disclosure.

DETAILED DESCRIPTION

[0009] Examples described herein provide an apparatus and method to cancel spatial processing in headphones. As discussed above, headphones may be used to listen to audio signals. Some headphones may apply a spatial filter or processing on the audio signal to simulate a surround sound experience. [0010] In some instances, a host device connected to the headphones that generate the audio signals may also apply a spatial filter. The quality of the audio signal may be degraded when spatial processing of the audio signal is applied by both the headphones and the host device. This can lead to an unsatisfactory user experience.

[0011] The number of different headphones manufacturers and different number of spatial processing filters may create a large number of combinations of headphones with spatial processing filters. Moreover, as new headphones are continuously created, storing a known list would not be practical for identifying a particular combination of headphones and spatial processing filter to cancel the spatial processing performed by the headphones.

[0012] The present disclosure provides a method and apparatus that can detect and cancel spatial processing applied by the headphones based on the audio signal that is outputted by the headphones. The host device may receive a sample of the audio signal outputted by the headphones. A correlation analysis may be applied to the audio signal to detect if spatial processing has been applied to the audio signal by the headphones. If spatial processing is detected, the host device can determine which spatial filter was applied and apply a correction filter to negate spatial processing applied by the identified spatial filter. The host device may then apply the spatial filter of the host device to the audio signal without interference from the spatial filter applied by the headphones. [0013] FIG. 1 illustrates an example system 100 to cancel spatial processing applied by headphones 112. In an example, the system 100 may include an apparatus 102 and the headphones 112. The headphones 112 may be communicatively coupled to the apparatus 102.

[0014] In an example, the apparatus 102 may be a computing device with a processor and memory or may be a device with a digital signal processor (DSP) that is dedicated to processing audio signals, as discussed below. The apparatus 102 may be connected via a communication interface to the headphones 112 to transmit audio and receive audio back from the headphones 112. For example, the headphones 112 may be connected wirelessly (e.g., via a Bluetooth connection) or via a wired connection (e.g., a 3.5 millimeter audio jack, 1/4 quarter inch audio jack, a lighting connector, a universal serial bus (USB) connection, and the like).

[0015] In an example, the headphones 112 may include an output 114 (e.g., speakers in the ear cups of the headphones 112) and a microphone 118. In an example, the headphones 112 may include two outputs 114 (e.g., two channels, one for the left ear and one for the right ear). In an example, the headphones 112 may be an open back headphone that can capture some of the audio signals emitted by the output 114 by the microphone 118.

[0016] In an example, the headphones 112 may include a spatial processor 116. The spatial processor 116 may encode audio signals to simulate a surround sound experience via two channels (e.g., a left channel and a right channel) of the headphones 112.

[0017] In an example, the apparatus 102 may include an audio generator 104, a spatial processing detector 106, a spatial processing cancellation device 108, and a host spatial processor 110. The audio generator 104 may be a sound card of the apparatus 102 to generate an audio signal. The audio signal may be transmitted to the headphones 112. The audio signal may be part of a video game, a movie, a music file, or any other type of media that includes audio signals. The audio signal can be encoded or processed via the spatial processor 116 of the headphones 112. The audio signal processed by the spatial processor 116 (hereinafter processed audio signal) may be emitted by the output 114 and captured by the microphone 118.

[0018] In an example, the processed audio signal may be transmitted back to the apparatus 102. The processed audio signal may be received by the spatial processing detector 106. The spatial processing detector 106 may detect whether spatial processing has been performed on the audio signal generated by the audio generator 104. If the spatial processing detector 106 detects that spatial processing has been performed, then the processed audio signal may be passed to spatial processing cancellation device 108. The spatial processing cancellation device 108 may apply a correction filter to eliminate the spatial processing applied by the spatial processor 116 of the headphones 112.

[0019] After the spatial processing is removed from the processed audio signal (hereinafter original audio signal), the original audio signal may be passed to the host spatial processor 110. The host spatial processor 110 may apply spatial processing used by the apparatus 102 and transmit the spatially processed audio to the headphones 112. The apparatus 102 may continuously apply the spatial processing cancellation to the audio from the headphones 112 such that the host spatial processor 110 can be applied instead.

[0020] As noted above, some headphones may perform spatial processing. However, when spatial processing is applied by both the apparatus 102 and the headphones 112, the audio signal may be degraded. Thus, the apparatus 102 may be able to detect if the headphones 112 have applied any spatial processing to the audio signal generated by the audio generator 104, cancel the spatial processing applied by the spatial processor 116 of the headphones 112, and then apply the host spatial processor 110 of the apparatus 102. As a result, a single instance of the spatial processing (of the host device or the apparatus 102) may be applied to the audio signal generated by the audio generator 104. [0021] The detection of spatial processing on the headphones 112 can be done in-situ and real-time with user-generated content (e.g., movies or music or voice) before applying the host spatial processing or after applying host spatial processing running in cascade with the headphone spatial processing. In an example, the detection can be done when new headphones 112 are detected for the first time and a specific stimulus (e.g., log-sweep) is transmitted from the apparatus 102 to the headphones 112. The response on the adjacent microphone 118 can be measured. The headphones 112 may be open-back headphones that couple to open-space.

[0022] FIG. 2 illustrates a block diagram of another example of the apparatus 102. In an example, the apparatus 102 may include a processor 202, a correlation analysis device 204, a correction filter 206, and a communication interface 208. In an example, the processor 202 may be a central processing unit (CPU) of a computing device or may be a digital signal processor (DSP) that is designed to perform a specific function (e.g., encode and/or process audio signals). For example, the DSP or an application specific integrated circuit (ASIC) chip may be programmed to perform certain functions, such as spatial encoding.

[0023] The processor 202 may control the correlation analysis device 204 to detect if the audio signal 210 includes spatial processing applied by the headphones 112 and to detect a spatial filter that was applied. If spatial processing is detected, the processor 202 can apply the correction filter 206 associated with the spatial filter that is identified by the correlation analysis device 204 to eliminate the spatial processing applied by spatial filter of the headphones 112.

[0024] In an example, the correlation analysis device 204 may be part of the spatial processing detector 106, illustrated in FIG. 1 . In an example, the correction filter 206 may be part of the spatial processing cancellation device 108, illustrated in FIG. 1.

[0025] In an example, an audio signal 210 from the headphones 112 (e.g., audio signals emitted by the output 114 and captured by the microphone 108) may be received by the apparatus 102 via the communication interface 208. In an example, the audio signal 210 may be a first frame, or first few frames, of the audio signal that is received from the headphones 112. For example, the headphones 112 may be connected to the communication interface 208. An audio signal may be transmitted to the headphones 112. The audio signal emitted from the output 114 may be captured by the microphone 118 and transmitted to the communication interface 208 as the audio signal 210. The communication interface 208 may be a wireless interface or a wired interface to transmit audio signals to the headphones 112 and to receive the audio signal 210 from the headphones 112.

[0026] The processor 202 may provide the audio signal 210 to the correlation analysis device 204. The correlation analysis device 204 may detect the spatial processing that is applied to the audio signal from the headphones 112 and identify a spatial filter that was applied by the spatial processor 116.

[0027] In an example, the audio signal 210 may be received over a single channel or converted by the processor 202 from a two channel audio signal to a single channel audio signal. A correlation analysis may be performed between an audio signal generated by the audio generator 104 and the audio signal 210 to determine whether a difference between the two signals exceeds a threshold. The difference may be an average of the difference between each value of the two signals within a predefined time frame or a period of the audio signal. If the difference exceeds the threshold, the correlation analysis device 204 may determine that the audio signal 210 has been spatially processed.

[0028] In an example, if a correlation between the audio signal generated by the audio generator 104 and the audio signal 210 is greater than a threshold, the correlation analysis device 204 may determine that the audio signal 210 comes from an open-back headphone and may clean up the audio signal 210. For example, a voice-activity detector may be used to eliminate portions of the audio signal 210 that are associated with a voice. An ambient noise suppression filter may also be used to eliminate ambient noise from the environment that may also be captured by the microphone 118 of the headphones 112 and inserted into the audio signal 210.

[0029] After the audio signal 210 is cleaned to eliminate voice and ambient noise, a single channel of the audio signal 210 may be analyzed on a frame-by- frame basis for a pre-determined number of frames to determine if spatial processing has been applied to the audio signal generated by the audio generator 104.

[0030] In some examples, the correlation analysis device may include a plurality of trained classifiers that can detect spatial processing and identify the type of spatial filter that was applied (e.g., Motion Pictures Expert Group-H (MPEG-H), Dolby, digital theater systems (DTS), and the like). In an example, the classifiers can be trained using large amounts of content with known spatial processing and/or known spatial filters applied, such as audio signals from various cinematic media, music, game content data, and the like. The classifiers can be trained based on the known contents of the audio signals to determine the profile for audio signals that are spatially processed and what type of spatial filter is used.

[0031] In an example, the classifiers can be trained by filtering large quantities of audio signals or content with a transfer function (e.g., H(e^{j J})) that is associated with sound radiation from both loudspeaker drivers or outputs 114 of the headphones 112 to the microphone 118. The transfer function can be measured by applying a stimulus signal to the loudspeaker drivers of an open- back headphone and deconvolving an impulse response h(n) < H(e^{j J}) from the recording (e.g., dividing the measured frequency response from the stimulus frequency response). This can be done for a few different open-back headphones to approximate an average response for the transfer function H(e^{j J}).

[0032] Each frame of the audio signal 210 may be compared to the trained classifiers to determine if any of the comparisons provide a confidence probability that is greater than a confidence probability threshold (e.g., greater than 95% , greater than 99%, and so forth). If the confidence probability for one of the classifiers is greater than the confidence probability threshold, then the correlation analysis device 204 may detect that the audio signal 210 has been spatially processed. The spatial filter associated with the trained classifier that matches the audio signal 210 may be identified as the spatial filter.

[0033] The processor 202 may then apply the correction filter 206 associated with the spatial filter that is identified to the audio signal 210 to cancel the spatial processing applied by the headphones 112. Although a single correction filter 206 is shown in FIG. 2, it should be noted that a plurality of different correction filters 206 may be stored in memory, as shown in FIG. 3, and discussed in further details below. [0034] In an example, the correction filter 206 may be applied to the audio signal generated by the audio generator 104. Thus, the spatial filter of the host spatial processor 110 and the correction filter 206 may be applied to the audio signal generated by the audio generator 104. The correction filter 206 may then cancel any spatial filtering applied by the spatial processor 116 in the headphones 112. Thus, the correction filter 206 may be used to pre-process the audio signals 210 before the audio signals are transmitted from the apparatus 102 to the headphones 112.

[0035] FIG. 3 illustrates another block diagram of an example of the apparatus 102. FIG. 3 illustrates the apparatus 102 with the processor 202, the correlation analysis device 204, the communication interface 208, the audio generator 104, the host spatial processor 110, and a memory 214. In an example, the memory 214 may be a non-transitory computer readable medium (e.g., a random access memory (RAM), a hard disk drive, a solid state drive, and the like). The memory 214 may include correction filters 216 and a list of previously identified headphones 218.

[0036] In an example, the audio signal 210 may be received by the apparatus 102 from the headphones 112 via the communication interface 208. The audio signal 210 may be analyzed by the correlation analysis device 204 to detect if the audio signal 210 has been spatially processed, and if so, identify the spatial filter, as described above. If the audio signal 210 has been spatially processed, the processor 202 may select a correction filter associated with the identified spatial filter from the correction filters 216 stored in the memory 214. The selected correction filter may then be applied to an audio signal generated by the audio generator 104. In addition, the audio signal may be spatially processed by the host spatial processor 110 to apply a spatial filter of the host device or apparatus 102. The correction filter that is applied and the spatial filter that is applied may generate an audio signal 212 that is transmitted to the headphones 112.

[0037] In an example, each time headphones 112 are connected, the processor 202 may determine if the headphones 112 have been previously connected to the apparatus 102. For example, when the headphones 112 are connected, the headphones 112 may provide a unique identifier or identification number to the apparatus 102. The unique identifier may be stored in the list of previous identified headphones 218. In an example, the unique identifier may also be stored in a remote server in the cloud to allow access for multiple different apparatuses 102.

[0038] If the unique identifier is found in the list of previously identified headphones 218, then the spatial filter used by the spatial processor 116 has been previously identified. Thus, the apparatus 102 may skip analysis by the correlation analysis device 204 and may apply the previously identified correction filter 216 to the audio signal generated by the audio generator 104. [0039] However, if the unique identifier of the headphones is not found in the list of previously identified headphones 218, then the correlation analysis device 204 may be activated. The correlation analysis device 204 may analyze the first few frames of the audio signal 210 received from the headphones 112, as described above.

[0040] FIG. 4 illustrates an example of how an audio signal 402 may be analyzed by the correlation analysis device 204. The audio signal 402 may be the audio signal 210 that is received from the headphones 112. In addition, the audio signal 402 may be received as a single channel or converted into a single channel, as described above. The audio signal 402 may be divided into a predetermined number of frames 404i to 404_n (hereinafter also referred to individually as a frame 404 or collectively as frames 404). In an example, each frame 404 may be a predefined time period (e.g., 1 millisecond, 1 second, several seconds, and the like) or may be a period of the signal (e.g., if the audio signal 402 has a repeating pattern or sinewave).

[0041] Each frame 404i to 404_n may be applied to the trained classifier during inference, from which the output being the class probability determines the type of headphone spatial processing (e.g., MPEG, Dolby, dts, and the like). A confidence probability may be calculated for the comparison of the frames 404i to 404_n to the frame or frames 406 of the trained classifier. If the confidence probability is greater than a confidence probability threshold, then a match may be detected. [0042] In other words, the audio signal 402 may be determined to have been spatially processed. In response, a correction filter represented as a frame 408 may be applied to the frames 404i to 404_n. The correction filter may include a waveform that negates the waveform of the spatial processing that is added by the identified spatial filter. As a result, a frame 410 of the audio signal 402 may be generated that has the spatial processing may be eliminated. A host spatial processing 412 may be applied to the frame 410 to then generate a new frame 414 of an audio signal with the correction filter applied and the host spatial processing 412 applied. The audio signal may then be transmitted to the headphones 112.

[0043] FIG. 5 illustrates a flow diagram of an example method 500 for cancelling spatial processing in headphones of the present disclosure. In an example, the method 500 may be performed by the apparatus 102 or the apparatus 600 illustrated in FIG. 6, and described below.

[0044] At block 502, the method 500 begins. At block 504, the method 500 receives an audio signal from headphones. For example, after the headphones are connected to an apparatus or device that performs the method 500, an audio signal may be generated and transmitted to the headphones. The headphones may be open back headphones that include a microphone. The microphone may capture audio signals emitted by the speakers of the headphones. The audio signal captured by the microphone may be transmitted to the apparatus that performs the method 500.

[0045] In an example, the audio signal may be a single channel or converted into a single channel. The audio signal may be cleaned up to remove any voice signals and/or ambient noise in the audio signal that is captured by the microphone. The audio signal may be divided into frames to be analyzed on a frame-by-frame basis, as described above. The first few frames of the audio signal received from the headphones may be analyzed.

[0046] At block 506, the method 500 detects spatial processing on the audio signal. For example, the frames of the audio signal may be compared to trained classifiers. A confidence probability may be calculated for the frames of the audio signal compared to each trained classifier. If the confidence probability for the comparison of the frames of the audio signal to one of the trained classifiers is above a confidence probability threshold, a match may be detected. Thus, spatial processing may be detected on the audio signal and the spatial filter associated with the trained classifier may be identified. If none of the confidence probabilities are above the confidence probability threshold, then it may be determined that no spatial processing has been applied to the audio signal by the headphones.

[0047] At block 508, the method 500 cancels the spatial processing on the audio signal. For example, a correction filter may be applied to the audio signal. The correction filter may be an inverse signal that cancels the signal applied by the spatial filter used by the headphones. In other words, the correction filter may apply a signal that has a shape that is inverse to a shape of the signal generated by the spatial filter that is applied to the audio signal.

[0048] At block 510, the method 500 applies a spatial filter of the host to the audio signal. For example, the host device or apparatus that performs the method 500 may have a proprietary spatial filter. To avoid degrading the audio signal by applying two different spatial filters, the method 500 may cancel the spatial filtering applied by the headphones and apply the spatial filter of the host to the audio signal.

[0049] At block 512, the method 500 transmits the audio signal that is processed with the spatial filter of the host to the headphones. For example, subsequent audio signals generated by the apparatus that performs the method 500 may be processed with the correction filter and the spatial filter of the host. Thus, the spatial filter applied by the headphones may be canceled by the correction filter encoded audio signal, and the headphones may emit the audio signal with the spatial processing applied by the spatial filter of the host.

[0050] In some examples, when the headphones have been previously connected to the apparatus that performs the method 500, the headphones may be identified on a subsequent connection. Thus, the apparatus may know which correction filter to apply to the headphones. As a result, when the correction filter of the headphones is known, the method 500 may skip blocks 504 and 506. For example, the appropriate correction filter and the spatial filter of the host may be applied to audio signals that are generated. The audio signal may then be transmitted to the headphones. At block 514, the method 500 ends.

[0051] FIG. 6 illustrates an example of an apparatus 600. In an example, the apparatus 600 may be the apparatus 100. In an example, the apparatus 600 may include a processor 602 and a non-transitory computer readable storage medium 604. The non-transitory computer readable storage medium 504 may include instructions 606, 608, 610, 612, and 614 that, when executed by the processor 602, cause the processor 602 to perform various functions.

[0052] In an example, the instructions 606 may include instructions to detect a connection to headphones. The instructions 608 may include instructions to transmit an audio signal to the headphones. The instructions 610 may include instructions to receive the audio signal from the headphones. The instructions 612 may include instructions to detect spatial processing on the audio signal from the headphones. The instructions 614 may include instructions to apply a correction filter to cancel the spatial processing performed by the headphones on the audio signal.

[0053] It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, or variations therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.

Claims

1. An apparatus, comprising: a communication interface to receive an audio signal from headphones connected to the communication interface; a correlations analysis device to detect spatial processing applied to the audio signal form the headphones and to identify a spatial filter applied by the headphones to perform the spatial processing; and a processor to apply a correction filter to cancel the spatial processing applied by the spatial filter of the headphones and apply a spatial filter of a host to the audio signal.

2. The apparatus of claim 1 , further comprising: a memory communicatively coupled to the processor to store a plurality of different correction filters for known spatial filters.

3. The apparatus of claim 1 , further comprising: a memory communicatively coupled to the processor to store a list of previously identified headphones and associated spatial filters of the previously identified headphones.

4. The apparatus of claim 1 , further comprising: an audio signal generator to generate the audio signal that is to be transmitted to the headphones.

5. The apparatus of claim 1 , wherein the audio signal is an output of the headphones captured by and received from a microphone of the headphones.

6. A method, comprising: receiving an audio signal from headphones; detecting spatial processing on the audio signal; cancelling the spatial processing on the audio signal; applying a spatial filter of the host to the audio signal; and transmitting the audio signal that is processed with the spatial filter of the host to the headphones.

7. The method of claim 6, wherein the detecting comprises: identifying a spatial filter applied by the headphones.

8. The method of claim 7, wherein the cancelling comprises: applying a correction filter associated with the spatial filter that is identified to cancel the spatial processing.

9. The method of claim 6, wherein the detecting is performed on a frame- by-frame basis of the audio signal for a predetermined number of frames.

10. The method of claim 9, wherein the spatial processing is detected when a match is detected between the audio signal and a trained classifier above a confidence probability threshold.

11. A non-transitory computer readable storage medium encoded with instructions executable by a processor, the non-transitory computer-readable storage medium comprising: instructions to detect a connection to a headphones; instructions to transmit an audio signal to the headphones; instructions to receive the audio signal from the headphones; instructions to detect spatial processing on the audio signal from the headphones; and instructions to apply a correction filter to cancel the spatial processing performed by the headphones on the audio signal.

12. The non-transitory computer readable storage medium of claim 11 , further comprising: instructions to store an identification of the headphones and the correction filter that was applied in a memory.

13. The non-transitory computer readable storage medium of claim 12, further comprises: instructions to detect a subsequent connection to the headphones; instructions to determine that the headphones were previously connected based on the identification of the headphones stored in the memory; and instructions to apply the correction filter to audio signals received from the headphones.

14. The non-transitory computer readable storage medium of claim 11 , further comprising: instructions to apply a spatial filter of a host device to the audio signal after the spatial processing performed by the headphones is cancelled by the correction filter.

15. The non-transitory computer readable storage medium of claim 11 , wherein the correction filter comprises an inverse filter based on an impulse response to a spatial filter used by the headphones to perform the spatial processing.