Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
  • H04S7/30—Control circuits for electronic adaptation of the sound field
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/11—Application of ambisonics in stereophonic audio systems

Definitions

• a spatial audio signal decoder performs operations to convert spatial audio signals from an input spatial audio format to an output spatial audio format.
• Known spatial audio signal format decoding techniques include passive decoding and active decoding.
• a passive signal decoder performs decoding operations that are based upon the input spatial audio signal format and the output spatial audio signal format and, in some examples, can use external parameters such as frequency but may not depend upon spatial characteristics of the audio input signal itself, such as the direction of arrival of audio sources in the audio input signal, for example. In other words, a passive signal decoder performs one or more operations independent of the spatial characteristics of the input signal.
• An active signal decoder performs decoding operations that are based upon the input spatial audio signal format, the output spatial audio signal format and perhaps external parameters such as frequency, for example, as well as spatial characteristics of the audio input signal.
• An active signal decoder can perform operations that are adapted to the spatial characteristics of the audio input signal.
• Active and passive signal decoders lack universality. Passive signal decoders often blur directional audio sources. For example, passive signal decoders sometimes render a di screte point source in an input audio signal format to all of the channels of an output spatial audio format (corresponding to an audio playback system) instead of to a subset localized to the point-source direction. Active signal decoders, on the other hand, often focus diffuse sources by modeling such sources as directional, for example, as a small number of acoustic plane waves. As a result, an active signal decoder sometimes imparts directionality to nondirectional audio signals. For example, an active signal decoder sometimes renders nondirectional reverberations from a particular direction in an output spatial audio format (corresponding to an audio playback system) such that the spatial characteristics of the reverberation are not preserved by the decoder.
• a problem to be solved includes optimizing audio signal rendering for omnidirectional components of an ambisonic mix, such as using active and passive decoders.
• some decoders may not correctly handle directionless or zero-distance information in ambisonic signals, such as“in-head” or“voice-of-god” type directionless signals.
• Some examples of potentially problematic signals include a 0th (zeroth) order (e.g., W-component only) ambisonic signals, or ambisonics that exhibit symmetry about one or more axes in three- dimensional space.
• ambisonic signals where the same signal is
• a solution can include, among other things, separating omnidirectional or middle components from an ambisonic signal, and using separate active and passive decoders for directional and omnidirectional components, respectively.
• a 0th order or directionless signal such as can be intended for diffuse or middle-panned rendering, can be added to a W-component of an ambisonic signal.
• information about an energy ratio in the X, Y, and Z components can be used to renormalize the W component, such as with the remainder extracted as the 0th order or omnidirectional component.
• the renormalized directional part can then be decoded, such as using
• the extracted part (which, for example, may have been intentionally added to the W-component) can be passively decoded or can be rendered using a volumetric method to diffuse and distribute the sound energy.
• directional but volumetric sources can be included in a mix by applying regular ambisonic panning and processing on multiple decorrelated signals.
• FIG. 1 illustrates generally a block diagram showing an input audio signal in a first spatial audio format, a decoder, and output audio signal in a second spatial audio format.
• FIG. 2 illustrates generally an example of a spatial audio signal decoder.
• FIG. 3 illustrates generally an example of a system with active and passive decoders.
• FIG. 4 illustrates generally a first example of a signal decomposition.
• FIG. 5 illustrates generally a second example of a signal decomposition.
• FIG. 6 illustrates generally an example of a method of processing an input ambisonic signal.
• FIG. 7 illustrates generally an example of a method of generating a secondary ambisonic signal.
• FIG. 8 illustrates generally an example of a method of preparing ambisonic or other signals for further processing by an active or passive decoder.
• FIG. 9 illustrates generally a diagram of a machine in the form of a computer system within which a set of instructions may be executed for causing the machine to perform any one or more of the methods discussed herein.
• Terms such as spatial encoding, spatial coding, or spatial audio coding can refer to representing a sound scene or soundfield in terms of audio signals and side
• spatial format or spatial audio format or spatial audio signal can refer to audio signals and side information that represent a sound scene or soundfield.
• side information can include a definition of a format, such as directional characteristics corresponding to each of the audio channels in the format, and in some cases, can include signal-dependent information such as directions of sources present in audio signals.
• a spatial audio signal includes one or more constituents that may be referred to as audio signal components, or audio channels or sources.
• a spatial audio signal may be referred to as an audio signal in a spatial format.
• Spatial decoding can include processing an input spatial audio signal in a specified spatial audio format to generate an output spatial audio signal in a specified spatial audio format.
• decoding can correspond to transcoding from an input spatial audio format to a different spatial audio format or decoding can include generating signals for playback over a specified audio reproduction system, such as a multichannel loudspeaker layout.
• An audio reproduction system can, in some examples, correspond to a spatial audio format or immersive format, such as a surround sound format.
• active or parametric spatial audio systems can be configured to make determinations or assumptions about a direction of one or more audio signal sources.
• Some sources can be intended to be rendered or reproduced with substantially no distance, that is, they can be considered to originate from essentially all directions simultaneously.
• signals or sources can be sourced from various panned objects. Examples discussed herein can encode such sources into an ambisonic mix such that the sources can be properly decoded downstream by active or parametric decoders.
• information about one or more of an omnidirectional,“middle,” or non-diegetic component can be added to an ambisonic source signal, or ambisonic source mix. That is, such signals or components can be encoded together with other directional components in an ambisonic mix, such as without using extra channels or metadata.
• a decoder receiving such a mix can identify residual energy, or 0th (zeroth) order energy, such as before or after accounting for diffuse or directional components in the same mix.
• the residual energy can be rendered as either a middle channel (e.g., having non-directional components, having zero distance, or being a “voice-of-god” type channel) or as an omni-volumetric signal.
• information from an ambisonic source can be pre-rendered as a volumetric source.
• omnidirectional components can be extracted by a parametric decoder as a diffuse component in a desired or intended number of different directions.
• FIG. 1 illustrates generally a first block diagram 100 that includes a
• the spatial audio decoder 106 can be configured to decode the input spatial audio signal 102 to an output spatial audio signal 108 such as in an output spatial audio format suitable for a multichannel audio reproduction system 110.
• the spatial audio decoder 106 can transform an input signal, such as can be received in a first-order ambisonics B -format or other spatial audio format, to an output signal in a multichannel audio format suitable for playback using the multichannel audio reproduction system 1 10.
• the spatial audio decoder 106 can be implemented as an active or passive decoder.
• the decoder can perform transformation operations from the input spatial format to the output spatial format independent of spatial characteristics of the audio input signal, such as direction of arrival of the audio input signal.
• the spatial audio decoder 106 can be implemented as an active decoder that performs transformation operations from the input spatial format to the output spatial format based at least in part upon spatial characteri stics of the audio input signal, or the input spatial audio signal 102.
• a rendered output using the multichannel audio reproduction system 110 can include information about a point source 112, such as can be decoded using an active decoder, can include information about a volumetric source 1 14, such as can be decoded using an active or passive decoder, or can include information about a middle source (e g., at the reference listening position), such as can be decoded using an active or passive decoder.
• FIG. 2 illustrates generally a spatial audio signal decoder example 200.
• the decoder example 200 can represent a configuration of the spatial audio decoder 106 from the example of FIG. 1 .
• the decoder can be configured to map an input spatial audio signal in an input spatial format to an output spatial audio signal in an output spatial format.
• one example decoder can be configured as an active signal decoder 308, and another example decoder can be configured as a passive signal decoder 310.
• each input spatial audio signal can include multiple audio signal components and that each output spatial audio signal can include multiple audio signal components.
• the respective audio signal components may be referred to as channels.
• mapping operation includes an M-by-N spatial decoder matrix to map M input spatial audio signal components in an input spatial format to N spatial audio output signal components in an output spatial format.
• the mapping operations can be used as a basis to configure the decoder as an active signal decoder or a passive signal decoder.
• the value of M is four since the input spatial format is a first-order ambisonics B-format signal that has four signal components, and the value of N depends, for example, on the number of speakers in the multichannel audio reproduction system.
• the spatial format of the input spatial audio signal received by the example signal decoder can include B-format audio input signal components W, X, Y, and Z, such as having directivity patterns given by the respective elements in the vector (d)(Q) defined as
• W corresponds to an angular pair consisting of an azimuth angle Q and an elevation angle f with respect to a reference point for measurement.
• a spatial audio scene or soundfield can be encoded in the W, X, Y, and Z components in accordance with the directivity patterns defined in the above vector.
• a point source S at azimuth angle Q and elevation angle f is encoded in B-format components as
• ambisonics such as B-format ambisonics
• B-format ambisonics can be used to represent a soundfield by capturing or encoding a fixed set of signals corresponding to a single point in the soundfield.
• Each of the fixed set of signals in an ambisonic representation has a defined directivity pattern.
• the directivity patterns are designed such that ambisonic- encoded signals carry directional information for all of the sounds in an entire soundfield.
• An ambisonic encoder (not shown) encodes a soundfield in an ambisonic format, and the format can be independent from a specific loudspeaker layout which may be used to reconstruct an encoded soundfield.
• An ambisonic decoder decodes ambisonic format signals for a specific loudspeaker layout.
• Eric Benjamin, Richard Lee, and Aaron Heller in“Is My Decoder Ambisonic?” published in 125th AES Convention, San Francisco 2008, provides a general explanation of ambisonics.
• a signal decoder transforms an input audio signal in an input spatial format to an output audio signal in an output spatial format suitable for a five- loudspeaker layout as depicted in FIG. 1.
• the examples are not limited to the
• Example signal decoders can be configured to decode to a 5.1 loudspeaker layout, a 7.1 loudspeaker layout, an 1 1.1 loudspeaker layout, or other loudspeaker layout, for example.
• the signal decoder transforms an input audio signal in an input spatial format to an output audio signal in a two-channel binaural format. The examples are not limited to input audio signals in the first-order ambisonics B-format.
• the signal decoder transforms an input audio signal in a higher-order ambisonics for at to an output audio signal in an output spatial format.
• a decoder or other processor can include or use headtracking information to further enhance a playback experience for a listener.
• FIG. 3 illustrates generally a schematic block diagram of a spatial audio signal decoder system 300.
• the spatial audio signal decoder system 300 can include a computer or processor system that includes one or more processor devices configured to be operatively coupled to one or more computer memory storage devices (e.g., a non- transitory storage device) that store instructions to configure the processor devices to provide the processing blocks described with reference to FIG. 3.
• the spatial audio signal decoder system 300 can include one or more of a time- frequency transformation block 304, an active/passive decomposition block 306, an active signal decoder 308, a passive signal decoder 310, a combiner block 312, and an inverse time-frequency transformation block 314.
• the time-frequency transformation block 304 can be configured to receive a time-domain input spatial audio signal, such as the input signal 302 in an input spatial audio format, and can convert the input signals to a time-frequency domain input signal 318. Subsequent processing is carried out in a corresponding time-frequency domain.
• An alternative example first spatial audio signal decoder system (not shown) omits the time- frequency transformation block so that subsequent processing is carried out in the time domain.
• the active/passive decomposition block 306 can be configured to decompose the time-frequency domain input signal 318, such as in the time-frequency domain, to produce an active input component 320 and passive input component 322.
• the active input component 320 and the passive input component 322 can sum or add up to the time-frequency domain input signal 318.
• each of the active input component 320 and the passive input component 322 can have respective multiple components or component signals.
• the components can be in the same spatial audio format as the input signal 302 as received in the time domain.
• the input signal 302 can include an ambisonic signal such as can include X, Y, Z, and W components.
• the input signal 302 can comprise an ambisonic signal with an excess W component, such as can result from a multidirectional source signal having symmetry along one or more axes (e.g., axes X, Y, and Z of a three- dimensional space).
• a phantom center signal constructed from identical sources that are evenly panned on either side of a listener can be such a source.
• Such a phantom center signal can have a primary direction but can have a more ambiguous or less certain position.
• the active signal decoder 308 and the passive signal decoder 310 can transform their respective inputs from an input spatial format to respective decoder output spatial audio signals, such as including active decoder output signals 324 and passive decoder output signals 326, that can have a common output spatial format such as a common multichannel loudspeaker layout.
• respective decoder output spatial audio signals such as including active decoder output signals 324 and passive decoder output signals 326
• different respective ones of the decoders can transform respective decoder input audio signals to respective decoder output audio signals having different spatial formats.
• the active signal decoder 308 can receive the active input component 320 and, in response, provide the active decoder output signals 324.
• the output format of the active decoder output signals 324 can be determined by a
• the active signal decoder 308 can transform the active input component 320, such as can have a respective input spatial format, to active decoder output signals 324 having a particular or designated active signal output spatial format.
• the passive signal decoder 310 can receive the passive input component 322 and, in response, provide the passive decoder output signals 326.
• the output format of the passive decoder output signals 326 can be determined by a configuration of the passive signal decoder 310.
• the passive signal decoder 310 can transform the passive input component 322, such as having a respective input spatial format, to the passive decoder output signals 326, such as can have a specified passive signal output spatial format.
• the passive signal decoder 310 can partition the passive input component 322 into one or more frequency bands such that different processing can be applied to each frequency band.
• an example passive signal decoder 310 can be configured to perform a lower frequency range transformation operation for a frequency range of the passive input component 322 below a cutoff frequency and is configured to perform an upper frequency range transformation operation for a frequency range of the passive input component 322 above the cutoff frequency.
• the passive signal decoder 310 can be configured to apply decorrelation processing or filtering to at least a portion of the passive input component 322 to thereby provide a decorrelated passive decoder output signals 326.
• the passive signal decoder 310 can render the decorrelated signals as an envelopment source.
• An envelopment source can include sound source information that originates, or is perceived by a listener to originate, from essentially all directions, and may or may not change with listener movement in any direction.
• a reverb signal can be considered to be enveloping source or an envelopment source.
• the combiner block 312 can include a summation circuit to sum the respective output spatial audio signals from the decoders, such as the active decoder output signals 324 and the passive decoder output signals 326.
• the combiner block 312 can provide summed output signals 328 representing the combination of the active decoder output signals 324 and the passive decoder output signals 326.
• the combiner block 312 can perform other processing such as filtering or decorrelation. Filters, such as all-pass filters or others, can be applied to one or more channels of the passive decoder output signals 326 to decorrelate the channels prior to the combination with the active decoder output signals 324 by the combiner block 312.
• Decorrelation of the channels can lead to a more diffuse and less directional rendering, which is generally preferable for the passive or omnidirectional components.
• additional processing of the decoded signal components can be carried out before combining the decoded signal components. For example, different filters may be applied to the active and passive components.
• additional processing of the decoded signal components can be carried out after combining the decoded signal components, for example, equalization or other filtering can be performed or applied.
• the inverse time-frequency transformation block 314 can receive the summed output signals 328 from the combiner block 312.
• the inverse time- frequency transformation block 314 can be configured to convert the summed output signals 328, such as in the time-frequency domain, to a time-domain output, or time- domain spatial audio output signals 316.
• the time-domain spatial audio output signals 316 can be provided to a sound reproduction system, such as the multichannel audio reproduction system 1 10 from the example of FIG. 1 , or can be provided to another signal processor or network for distribution.
• FIG. 4 illustrates generally an example 400 of a first decomposition block 406.
• the active/passive decomposition block 306 from the example of FIG. 3 comprises or corresponds to the first decomposition block 406.
• the first decomposition block 406 can include a 0th order determination block 410 and a normalization block 412.
• the 0th order determination block 410 is configured to receive the time-frequency domain input signal 318 such as comprising an ambi sonic signal.
• the 0th order determination block 410 can be configured to perform various processing operations, as discussed elsewhere herein, to provide a directional component 402 and an omnidirectional component 404.
• the directional component 402 can include, for example, a first order ambisonic signal such as comprising X, Y, Z, and W components.
• the W component of the directional component 402 can be unbalanced relative to the other components of the directional component 402.
• the normalization block 412 can be configured to analyze the X, Y, and Z components and re-scale or normalize the W component to thereby provide a normalized output 408 that comprises balanced W, X, Y, and Z components. That is, the normalization block 412 can be used to prepare an ambisonic signal for output by correcting a perceived imbalance in energy with respect to the X, Y, Z, and W components of the signal.
• the normalized output 408 includes or comprises the active input component 320 such as can be provided to the active signal decoder 308.
• the normalization block 412 can be configured to prepare or provide an auxiliary ambisonic signal at its output by subtracting the omnidirectional component 404 from the directional component 402.
• the resulting auxiliary ambisonic signal can include one or more non-zero directional components.
• the omnidirectional component 404 can include or represent information about an omnidirectional signal, or less-directional signal, that is present in the time-frequency domain input signal 318.
• the omnidirectional component 404 can include or comprise the passive input component 322 such as can be provided to the passive signal decoder 310.
• the omnidirectional component 404 can include a middle channel, or in-head channel, or“voice of god” channel that can be rendered by the passive signal decoder 310.
• the omnidirectional component 404 can comprise an ambisonic signal with a W component and, in some examples, can omit X, Y, or Z components.
• a multidirectional source with perfect symmetry about all directional axes can comprise an ambisonic signal with only a W component.
• a“W-only” signal can be considered a 0th order ambisonic signal.
• FIG. 5 illustrates generally an example 500 of a second decomposition block 506.
• the active/passive decomposition block 306 from the example of FIG. 3, or the first decomposition block 406 from the example of FIG. 4 comprises or corresponds to the second decomposition block 506.
• the second decomposition block 506 can be configured to receive the time- frequency domain input signal 318 and route its various components to one or more of a direction estimation block 502, a subspace determination block 504, or the 0th order determination block 410.
• the direction estimation block 502 can provide an estimate 510 of a number and direction of arrival (DO A) of directional audio sources in the time- frequency domain input signal 318, in accordance with an input spatial audio format.
• DO A direction of arrival
• the subspace determination block 504 can determine the active input component 320 based upon the estimate 510 of the number and DOAs of directional sound sources in the time-frequency domain input signal 318.
• An example sub space determination block 504 determines the active input component 320 by projecting the active signal component onto a subspace that can be determined based on the number and DOAs of directional sound sources and the time-frequency domain input signal 318.
• the 0th order determination block 410 can be used to determine a value or values that can be used to scale a residual, passive, or omnidirectional component of the time-frequency domain input signal 318.
• the 0th order determination block 410 can determine or provide a scale factor for scaling information between the active input component 320 and the passive input component 322.
• the residual determination block 508 can determine the passive input component 322, for example, based on a difference between the time-frequency domain input signal 3 18 and the active input component 320 as-determined by the sub space determination block 504.
• the passive input component 322 can be determined first, and the active input component 320 can be determined thereafter based upon a difference between the received time-frequency domain input signal 318 and the passive input component 322.
• acti ve signals or the active input component 320 can include functionally directional source signals.
• the passive input component 322 can include a combination of non-directional or omnidirectional sources, such as can be detected using the 0th order determination block 410.
• the passive input component 322 can include diffuse or residual error-based signals as identified by the sub space determination block 504 and the residual determination block 508.
• Other examples of determining residual error and allocating residual signals are discussed by Goodwin et al. in“Spatial Audio Signal Decoder,” PCT application PCT/US2019/046936, filed August 16, 2019 (herein “Goodwin et al "), the entirety of which is hereby incorporated herein by reference in its entirety.
• FIG. 6 illustrates generally an example of a first method 600 that can include processing an input ambisonic signal to separate omnidirectional components that can be rendered using a passive decoder from other, directional components that can be rendered using an active decoder.
• the example first method 600 can include receiving a B -form at primary ambisonic signal such as including W, X, Y, and Z components.
• block 602 can include receiving the input spatial audio signal 102 such as comprising the input signal 302.
• block 602 can include receiving the input signal 302 using one or more of the active/passive decomposition block 306, the first decomposition block 406, or the second decomposition block 506.
• the first method 600 can include determining, such as for a first signal band or frequency band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal.
• the active/passive decomposition block 306 can be configured to receive the input signal 302 and analyze the X, Y, Z, and W components of the input signal 302 such as to identify any imbalance of the W
• the input signal 302 can be considered to have an
• omnidirectional component 404 that can be extracted from the input signal 302 such as for subsequent processing that can be different than the processing used for the other, directional component 402.
• the first method 600 can include determining a secondary ambisonic signal using information about a difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.
• the active/passive decomposition block 306 can include a signal processor configured to determine or generate the secondary ambisonic signal to represent the omnidirectional component 404.
• the secondary ambisonic signal can include a 0th order ambisonic signal, or can include a single channel signal that includes exclusively omnidirectional signal information.
• block 606 can include determining a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the input signal 302.
• determining the total energy can include determining whether a sum of squared magnitudes of the X, Y, and Z components is substanti ally equal to a square magnitude of the W component.
• determining the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the input signal 302 can include determining whether component attributes of the input signal 302 are mathematically equivalent to solutions of assumed panning equations and normalizations for the input signal 302, such as can have different values or requirements depending on a format used.
• the first method 600 can include generating an output signal that is based on the secondary ambisonic signal, such as using the passive signal decoder 310.
• the output can be a middle channel, or in-head type signal, sometimes referred to a“voice of god” signal because it can be interpreted by a li stener as either originating inside the listener's head or as originating from all directions simultaneously.
• the first method 600 can include determining or generating a re balanced primary ambisonic signal based on a difference between the primary ambi sonic signal as-received and the secondary ambisonic signal such as determined at block 606.
• block 610 can include using the normalization block 412 to scale the primary ambisonic signal or to generate a W component that corresponds to the X, Y, and Z components of the primary ambisonic signal, such as received at block 602.
• the first method 600 can include generating an output signal that is based on the re-balanced primary ambisonic signal, such as using the active signal decoder 308.
• FIG. 7 illustrates generally an example of a second method 700 that can include generating a secondary ambisonic signal, such as using the active/passive decomposition block 306.
• the second method 700 can begin at block 606, which is described above in the discussion of FIG. 6.
• the second method 700 can include i dentifying one or more first-order or higher-order ambisonic steering components for the secondary ambisonic signal.
• the one or more directional steering components can be based on respective steering components from the primary ambisonic signal.
• the second method 700 can include computing a residual error projection matrix for the W, X, Y, and Z components of the primary ambisonic signal, such as received at block 602.
• a residual error projection matrix can be computed or determined according to the systems and methods discussed by Goodwin et al., which is incorporated herein by reference.
• the residual can represent a passive signal component as a difference between an input signal and the primary ambisonic signal.
• the second method 700 can include determining an omni-signal return factor.
• the omni- signal return factor can be based on the determined difference between the total energy of X, Y, and Z components and the total energy of the W component of the input signal.
• the second method 700 can include updating the residual error projection matrix using the determined omni-signal return factor from block 706. That is, the omni-signal return factor from block 706 can indicate a total energy difference (e.g., an excess) of the W component relative to the X, Y, and Z components. Updating the projection matrix can include rebalancing the values such that the residual or excess W component is removed. In an example, the matrix can be rebalanced such that any excess W component is allocated to the residual or passive component.
• the second method 700 can include applying the updated residual error projection matrix to the primary ambisonic signal to thereby generate the secondary ambisonic signal.
• the generated secondary ambisonic signal can be provided to the passive signal decoder 310 for further processing or rendering.
• Goodwin et al. which is incorporated herein by reference, explains generally various systems and methods for determining a residual signal, or portions of an input signal that may not be adequately represented by directional components that can be detected in the input.
• an input signal including a front-left source with some volumetric attribute e.g., not a pure point source
• the decoder described by Goodwin et al. can, in an example, detect a center of the source and designate directionality based on the center (e.g., 45 degrees left). The decoder can then test the directionality hypothesis by forming a pure point source at the given direction (e.g., 45 degrees left).
• a difference between the pure point source and the original volumetric signal can, in theory, be designated as the residual and can thus be passively decoded (or, non-point source rendered), while the center or point source information can be actively decoded.
• an ambisonic signal that includes the phantom signal will cancel for left-right balance. Since the source is intended to be front center, the decoder of Goodwin et al. can produce a strong front center signal. The residual can be minimized because the left-right balance was cancelled out. Any information that the source was a center image made of respective sources on the left and right can be hidden in the relationship to the magnitude, rather than the direction, of the signal.
• FIG. 8 illustrates generally an example of a third method 800 that can include preparing different ambi sonic or other signals for further processing by an active or passive decoder, such as by the active signal decoder 308 or the passive signal decoder 310.
• the third method 800 can be performed at least in part using the active/passive decomposition block 306 and can begin at block 802 with receiving a primary ambisonic signal having information about a volumetric source.
• the primary ambisonic signal can include or comprise the input signal 302.
• the volumetric source can comprise a source with various directional cues and can have ambiguous or less-directional characteristics.
• the third method 800 can include determining a secondary ambisonic signal having one or more non-zero directional components.
• the non-zero directional components can be based on the directional cues associated with volumetric source.
• the secondary ambisonic signal can represent predominantly the less-directional characteristics from the volumetric source.
• the third method 800 can include generating a tertiary ambisonic signal.
• the tertiary ambisonic signal can be generated by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more of the non-zero directional components.
• the third method 800 can include outputting the one or more non zero directional components of the tertiary ambisonic signal to an active decoder, such as to the active signal decoder 308.
• Block 810 can include outputting the secondary ambisonic signal to a passive decoder, such as the passive signal decoder 310.
• an example of the input signal 302 can include or comprise a horizontal -only ambisonic signal comprising W, X, and Y components.
• any extracted or determined omnidirectional imbalance can be attributed to the“missing” Z component.
• a secondary ambisonic signal can be generated for W and Z, which could in turn be rendered as the missing height component.
• FIG. 9 is a di agrammatic representation of a machine 900 within which instructions 908 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 900 to perform any one or more of the methodologies discussed herein can be executed.
• the instructions 908 can cause the machine 900 to execute any one or more of the methods described herein.
• the instructions 908 can transform the general, non-programmed machine 900 into a particular machine 900 programmed to carry out the described and illustrated functions in the manner described.
• the machine 900 can operate as a standalone device or can be coupled (e.g., networked) to other machines or devices or processors. In a networked deployment, the machine 900 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
• the machine 900 can comprise a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 908, sequentially or otherwise, that specify actions to be taken by the machine 900. Further, while only a single machine 900 is illustrated, the term
• machine can be taken to include a collection of machines that individually or jointly execute the instructions 908 to perform any one or more of the methodologies discussed herein.
• the instructions 908 can include instructions stored using a memory circuit
• the machine 900 can include or use a processor circuit such as can be associated with any one or more of the various blocks, modules, processors, or other processing hardware or software discussed herein.
• the machine 900 can include various processors and processor circuitry, such as represented in the example of FIG. 9 as processors 902, memory 904, and I/O
• processors 902 e.g., a Central Processing Unit (CPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU), a graphics processing unit (GPU),
• RISC Reduced Instruction Set Computing
• CISC Complex Instruction Set Computing
• GPU Graphics Processing Unit
• DSP Digital Signal Processor
• ASIC ASIC
• RFIC Radio-Frequency Integrated Circuit
• processor 906 a processor 906 and a processor 910 that execute the instructions 908.
• processor 910 a processor 910 that execute the instructions 908.
• the term“processor” is intended to include multi-core processors that can comprise two or more independent processors (sometimes referred to as“cores”) that can execute instructions
• the machine 900 can include a single processor with a single core, a single processor with multiple cores (e.g., a multi -core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.
• a single processor with a single core e.g., a multi -core processor
• multiple processors with a single core e.g., multiple processors with multiples cores, or any combination thereof.
• the memory 904 can include a main memory 912, a static memory 914, or a storage unit 916, such as can be accessible to the processors 902 via the bus 944.
• the memory 904, the static memory 914, and storage unit 916 can store the instructions 908 embodying any one or more of the methods or functions or processes described herein.
• the instructions 908 can also reside, completely or partially, within the main memory 912, within the static memory 914, within the machine-readabl e medium 918 within the storage unit 916, within at least one of the processors (e.g., within a processor’s cache memory), or any suitable combination thereof, during execution thereof by the machine 900.
• the I/O components 942 can include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on.
• the specific I/O components 942 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones can include a touch input device or other such input mechani sms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 942 can include many other components that are not shown in FIG. 9. In various example embodiments, the I/O components 942 can include output components 928 and input components 930.
• the output components 928 can include visual components (e.g., a display such as a plasma display panel (PDF), a light emitting diode (LED) display, a liquid crystal display (LCD), a proj ector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth.
• visual components e.g., a display such as a plasma display panel (PDF), a light emitting diode (LED) display, a liquid crystal display (LCD), a proj ector, or a cathode ray tube (CRT)
• acoustic components e.g., speakers
• haptic components e.g., a vibratory motor, resistance mechanisms
• the input components 930 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.
• alphanumeric input components e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components
• point-based input components e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument
• tactile input components e.g., a physical button,
• the I/O components 942 can include biometric components 932, motion components 934, environmental components 936, or position components 938, among a wide array of other components.
• the biometric components 932 include components configured to detect a presence or absence of humans, pets, or other individuals or obj ects, or configured to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or el ectroencephalogram -b ased identification), and the like.
• the motion components 934 can include acceleration sensor components (e.g.,
• accelerometer e.g., Bosch Sensortec BMA150 accelerometer
• gravitation sensor components e.g., Bosch Sensortec BMA150 accelerometer
• rotation sensor components e.g., gyroscope
• the environmental components 936 can include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby obj ects), gas sensors (e.g., gas detection sensors to detection concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals
• illumination sensor components e.g., photometer
• temperature sensor components e.g., one or more thermometers that detect ambient temperature
• humidity sensor components e.g., humidity sensor components
• pressure sensor components e.g., barometer
• acoustic sensor components e.g., one or more microphones that detect background noise
• proximity sensor components e.g., infrared sensors that detect nearby
• location sensor components e.g., a GPS receiver component, an RFID tag, etc.
• altitude sensor components e.g., altimeters or barometers that detect air pressure from which altitude can be derived
• orientation sensor components e.g.,
• the I/O components 942 can include communication components 940 operable to couple the machine 900 to a network 920 or devices 922 via a coupling 924 and a coupling 926, respectively.
• the communication components 940 can include a network interface component or another suitable device to interface with the network 920.
• the communication components 940 can include wired communication components, wireless communication components, cellular communication components.
• the devices 922 can be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).
• the communication components 940 can detect identifiers or include components operable to detect identifiers.
• the communication components 940 can include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals).
• RFID Radio Frequency Identification
• NFC smart tag detection components e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes
• IP Internet Protocol
• Wi-Fi® Wireless Fidelity
• NFC beacon detecting an NFC beacon signal that can indicate a particular location
• the various memories can store one or more instructions or data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions
• instructions 908) when executed by processors or processor circuitry, cause various operations to implement the embodiments discussed herein.
• the instructions 908 can be transmitted or received over the network 920, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 940) and using any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol
• the instructions 908 can be transmitted or received using a transmission medium via the coupling 926 (e.g., a peer-to-peer coupling) to the devices 922.
• the coupling 926 e.g., a peer-to-peer coupling
• Aspect 1 can include or use subject matter such as a method comprising receiving a B -form at primary ambi sonic signal including W, X, Y, and Z components, determining, for a first signal band of the primary ambi sonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambi sonic signal, and determining a secondary ambi sonic signal using information about the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.
• Aspect 2 can include, and can optionally be combined with the subj ect matter of Aspect 1 to include generating an output signal based on the secondary ambisonic signal, the generating including scaling the secondary ambisonic signal for one or more of independent output, decoding, and spatial rendering.
• Aspect 3 can include, and can optionally be combined with the subj ect matter of Aspect 2 to include generating a re-balanced primary ambisonic signal based on a difference between the primary ambisonic signal as-received and the secondary ambisonic signal.
• Aspect 4 can include, and can optionally be combined with any one or more of Aspects 1-3 to include, determining the secondary ambisonic signal, including
• Aspect 5 can include, and can optionally be combined with the subj ect matter of Aspect 4 to include rendering the single-channel signal as a middle channel audio signal, wherein the middle channel audio signal is a signal configured to be heard or interpreted in-head or is a voice of god signal.
• Aspect 6 can include, and can optionally be combined with any one or more of Aspects 4-5 to include applying decorrelation processing to at least a portion of the secondary ambisonic signal to provide a decorrelated signal, and rendering the decorrelated signal as an envelopment source.
• Aspect 7 can include, and can optionally be combined with any one or more of Aspects 1-6 to include the secondary ambisonic signal including one or more first-order or higher-order ambisonic steering components, wherein the one or more directional steering components are based on respective steering components from the primary ambisonic signal.
• Aspect 8 can include, and can optionally be combined with the subj ect matter of Aspect 7 to include determining the secondary ambisonic signal, including: computing a residual error projection matrix for the W, X, Y, and Z components of the received primary ambisonic signal, determining an omni-signal return factor based on the determined difference between the total energy of X, Y, and Z components and the total energy of the W component of the primary ambisonic signal, updating the residual error projection matrix using the determined omni-signal return factor, and applying the updated residual error projection matrix to the primary ambisonic signal to thereby generate the secondary ambisonic signal.
• Aspect 9 can include, and can optionally be combined with any one or more of Aspects 1-8 to include the secondary ambisonic signal consisting of first or higher-order ambisonic signals including an omni-directional channel and one or more directional steering channels based on the received primary ambisonic signal.
• Aspect 10 can include, and can optionally be combined with any one or more of Aspects 1-9 to include generating a tertiary ambisonic signal comprising the primary ambisonic signal less the determined difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambi sonic signal.
• Aspect 1 1 can include, and can optionally be combined with any one or more of Aspects 1-10 to include preparing the primary ambisonic signal for output by scaling the W component of the primary ambisonic signal to correct a perceived imbalance in energy with respect to the X, Y, and Z components of the primary ambisonic signal.
• Aspect 12 can include, and can optionally be combined with any one or more of Aspects 1-1 1 to include generating a tertiary ambisonic signal by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more non-zero directional components.
• Aspect 13 can include, and can optionally be combined with the subject matter of Aspect 12 to include receiving the primary ambisonic signal including receiving information about a volumetric source having directional cues and ambiguous or less- directional characteristics, wherein determining the secondary ambisonic signal includes generating a signal having one or more non-zero directional components based on the directional cues while representing predominately the less-directional characteristics, and wherein the one or more non-zero directional components of the tertiary ambisonic signal predominately represent the directional cues.
• Aspect 13 can include outputting, to an active decoder, the one or more non-zero directional
• Aspect 14 can include, and can optionally be combined with the subj ect matter of Aspect 12 to include providing the tertiary ambisonic signal to an active decoder and providing the secondary ambisonic signal to a passive decoder.
• Aspect 15 can include, and can optionally be combined with any one or more of Aspects 1-14 to include determining the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal by determining whether a sum of squared magnitudes of the X, Y, and Z components is substantially equal to a square magnitude of the W component.
• Aspect 16 can include, and can optionally be combined with any one or more of Aspects 1-15 to include determining the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambi sonic signal by determining whether component attributes of the primary ambisonic signal are mathematically equivalent to solutions of assumed panning equations and normalizations for the primary ambisonic signal.
• Aspect 17 can include or use subj ect matter such as a computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform one or a combination of Aspects 1-16.
• Aspect 17 can include at least instructions that cause the computer to receive a B -form at primary ambisonic signal including W, X, Y, and Z components, and determine, for a first signal band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal, and determine a secondary ambisonic signal using information about the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.
• Aspect 18 can include or use subj ect matter such as a computing apparatus, the computing apparatus compri sing a processor and a memory storing instructions that, when executed by the processor, configure the apparatus to perform one or a
• Aspect 18 can include at least instructions that cause the computer to receive a B -form at primary ambisonic signal including W, X, Y, and Z components, and determine, for a first signal band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal, and determine a secondary ambisonic signal using information about the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.
• Aspect 19 can include or use subj ect matter such as a method (or a non- transitory com puter-readabl e storage medium, or a computing apparatus configured to perform the method or a portion thereof) that can include receiving a primary ambisonic signal including a W component and one or more of X, Y, and Z components, determining, for a first signal band of the primary ambisonic signal, a difference between a total energy of the one or more of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal, and determining a secondary ambisonic signal using information about the difference between the total energy of the one or more of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.
• a method or a non- transitory com puter-readabl e storage medium, or a computing apparatus configured to perform the method or a portion thereof
• Aspect 20 can include, and can optionally be combined with the subj ect matter of Aspect 19, such that the primary ambisonic signal includes a W component and two of the X, Y, and Z components, and the secondary ambi sonic signal includes a B-format ambisonic signal having X, Y, Z, and W components.
• Aspect 21 can include, and can optionally be combined with any one or more of Aspects 19 or 20 to include generating an output signal based on the secondary ambisonic signal, the generating including scaling the secondary ambisonic signal for one or more of independent output, decoding, and spatial rendering.
• Aspect 22 can include, and can optionally be combined with any one or more of Aspects 19-21 to include determining the secondary ambisonic signal by determining a single-channel signal that includes exclusively omnidirectional signal information.
• Aspect 23 can include, and can optionally be combined with any one or more of Aspects 19-22 to include the secondary ambisonic signal consisting of 1st or higher- order ambisonic signals including an omni-directional channel and one or more directional steering channels based on the received primary ambisonic signal.
• Aspect 24 can include, and can optionally be combined with any one or more of Aspects 19-23 to include generating a tertiary ambisonic signal by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more non-zero directional components and, optionally, providing the tertiary ambisonic signal to an active decoder and providing the secondary ambisonic signal to a passive decoder.
• the terms“a” or“an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of“at least one” or“one or more.”
• the term“or” is used to refer to a nonexclusive or, such that“A or B” includes“A but not B,”“B but not A,” and“A and B,” unless otherwise indicated.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Signal Processing (AREA)
• Acoustics & Sound (AREA)
• Mathematical Physics (AREA)
• Multimedia (AREA)
• Human Computer Interaction (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Health & Medical Sciences (AREA)
• Computational Linguistics (AREA)
• Mathematical Analysis (AREA)
• Theoretical Computer Science (AREA)
• Pure & Applied Mathematics (AREA)
• Mathematical Optimization (AREA)
• General Physics & Mathematics (AREA)
• Algebra (AREA)
• Stereophonic System (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=71787092

Family Applications (1)

Country Status (3)

Family Cites Families (1)

• 2020
  • 2020-05-29 KR KR1020217043424A patent/KR20220027891A/ko unknown
  • 2020-05-29 WO PCT/US2020/035270 patent/WO2020243535A1/en unknown
  • 2020-05-29 EP EP20746393.6A patent/EP3977447A1/de active Pending

Also Published As

Similar Documents

Legal Events