BR112016028215B1 - GETTING SCATTERED INFORMATION FOR HIGHER ORDER AMBISSONIC AUDIO RENDERERS - Google Patents

Abstract

SCATTERED INFORMATION GETTING FOR HIGH ORDER AMBISSONIC AUDIO RENDERS In general, techniques are described for getting the audio rendering information in a bitstream. A device configured to render higher-order ambisonic coefficients comprising a processor and memory can perform the techniques. The processor may be configured to obtain sparsity information indicative of sparsity from a matrix used to render the higher order ambisonic coefficients for a plurality of speaker feeds. Memory can be configured to store scatter information.

Description

[0001] This application claims the benefit of US Provisional Patent Application No. 62/023,662, filed July 11, 2014, entitled “SIGNALING AUDIO RENDERING INFORMATION IN A BITSTREAM,” and US Provisional Patent Application No. 62/005,829 , filed May 30, 2014, titled “SIGNALING AUDIO RENDERING INFORMATION IN A BITSTREAM,” the entire contents of each of the US Interim Applications are hereby incorporated by reference as if set forth herein in their respective entireties.

TECHNICAL FIELD

[0002] This revelation pertains to information rendering, and more specifically, information rendering for higher order ambisonic audio (HOA) data.

BACKGROUND

[0003] During the production of audio content, the sound engineer may render the audio content using a specific renderer in an attempt to adapt the audio content to the target settings of the speakers used to play the audio content. In other words, the sound engineer can render the audio content and play the rendered audio content using speakers arranged in the desired configuration. The sound engineer can then remix various aspects of the audio content, render the remixed audio content, and replay the rendered and remixed audio content using the speakers arranged in the targeted configuration. The sound engineer can iterate in this way until a certain artistic intent is provided by the audio content. In this way, the sound engineer can produce audio content that provides a certain artistic intent or that otherwise provides a certain sound field during playback (e.g. to accompany the video content played alongside the audio content) .

SUMMARY

[0004] In general, techniques are described for specifying audio rendering information in a representative bitstream of audio data. In other words, the techniques can provide a way in which to signal the audio rendering information used during the production of audio content to a playback device, which can then use the audio rendering information to render the audio content. Providing rendering information in this way allows the playback device to render the audio content in a manner intended by the sound engineer and thus potentially ensures proper reproduction of the audio content so that the artistic intent is potentially understood by the sound engineer. a listener. In other words, the rendering information used during rendering by the sound engineer is provided according to the techniques described in this disclosure so that the audio playback device can use the rendering information to render the audio content in a intended by the sound engineer, thus ensuring a more consistent experience during the production and playback of audio content compared to systems that do not provide this audio rendering information

[0005] In one aspect, a device configured to render higher-order ambisonic coefficients comprises one or more processors configured to obtain sparsity information indicative of the sparsity of a matrix used to render the higher-order ambisonic coefficients for a plurality of feeds. loudspeakers; and a memory configured to store sparsity information.

[0006] In another aspect, a method of rendering higher-order ambisonic coefficients comprises obtaining sparsity information indicative of a sparsity of a matrix used to render higher-order ambisonic coefficients to generate a plurality of speaker feeds.

[0007] In another aspect, a device configured to produce a bitstream comprises a memory configured to store a matrix, and one or more processors configured to obtain sparsity information indicative of a sparseness of the matrix used to render higher-order ambisonic coefficients for generate a plurality of speaker feeds.

[0008] In another aspect, a method of producing a bitstream comprises obtaining sparsity information indicative of a sparsity of a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds.

[0009] In another aspect, a device configured to render higher-order ambisonic coefficients comprises one or more processors configured to obtain signal symmetry information indicative of the signal symmetry of a matrix used to render the higher-order ambisonic coefficients to generate a plurality of speaker feeds; and a memory configured to store sparsity information.

[0010] In another aspect, a method of rendering higher-order ambisonic coefficients comprises obtaining signal symmetry information indicative of the signal symmetry of a matrix used to render the higher-order ambisonic coefficients to generate a plurality of high-order feeds. speakers.

[0011] In another aspect, a device configured to output a bitstream comprises a memory configured to store a matrix used to render the higher-order ambisonic coefficient to generate a plurality of speaker feeds, and one or more processors configured to signal the symmetry information indicative of the signal symmetry of the matrix.

[0012] In another aspect, a method of producing a bitstream comprises obtaining sparsity information indicative of a sparsity of a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds.

[0013] Details of one or more aspects of the techniques are presented in the accompanying drawings and description below. Other features, objects and advantages of the techniques will be evident from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG. 1 is a diagram illustrating spherical harmonic basis functions of various orders and suborders.

[0015] FIG. 2 is a diagram illustrating a system that can perform various aspects of the techniques described in this disclosure.

[0016] FIG. 3 is a block diagram illustrating in more detail an example of the audio encoding device shown in the example of Fig. 2 which can perform various aspects of the techniques described in this disclosure.

[0017] FIG. 4 is a block diagram illustrating the audio decoding device of Fig. 2 in greater detail.

[0018] FIG. 5 is a flowchart illustrating exemplary operation of an audio encoding device in performing various aspects of the vector-based synthesis techniques described in this disclosure.

[0019] FIG. 6 is a flowchart illustrating exemplary operation of an audio decoding device in performing various aspects of the techniques described in this disclosure.

[0020] FIG. 7 is a flowchart illustrating exemplary operation of a system, such as one of the systems shown in the examples of Figs. 2 in carrying out various aspects of the techniques described in this disclosure.

[0021] FIGs. 8A-8D are diagrams illustrating bitstreams formed according to the techniques described in this disclosure.

[0022] FIGs. 8E-8G are diagrams illustrating portions of bitstream or sidechannel information that can specify compressed spatial components in more detail.

[0023] FIG. 9 is a diagram illustrating an example of higher order ambisonic (HOA) dependent min and max gains within an HOA rendering matrix.

[0024] FIG. 10 is a diagram illustrating a partially sparse 6th order HOA rendering matrix for 22 speakers.

[0025] FIG. 11 is a flow diagram illustrating signaling symmetry properties.

DETAILED DESCRIPTION

[0026] The evolution of ambient sound has made many output formats available for entertainment today. Examples of such ambient sound formats for consumers are mostly “channels”, because they implicitly specify feeds to speakers in certain geometric coordinates. Consumer ambient sound formats include the popular 5.1 format (which includes the following six channels: Front Left (FL), Front Right (FR), Front Center or Center, Left Back or Left Ambient, Right Rear or Right Ambient, and (LFE)), the growing 7.1 format, various formats that include loudspeakers such as the 7.1.4 format and the 22.2 format (for example, for use with the Ultra High Definition Television standard). Non-consumer formats can span any number of loudspeakers (in symmetrical and non-symmetrical geometries), often referred to as “ambient arrays”. An example of this arrangement includes 32 speakers positioned in coordinates at the corners of a truncated icosahedron.

[0027] Input to a future MPEG encoder is optionally one of three possible formats: (i) traditional channel-based audio (as discussed above), which is intended to be played through speakers at pre-specified positions; (ii) object-based audio, which involves pulse code modulation (PCM) data for individual audio objects with associated metadata containing their location coordinates (among other information); and (iii) scene-based audio, which involves representing the sound field using coefficients of spherical harmonic basis functions (also called “spherical harmonic coefficients” or SHC, “Higher Order Ambisonics” or HOA and “HOA coefficients” The future MPEG encoder can be described in more detail in a document entitled “Call for Proposals for 3D Audio,” by the International Organization for Standardization/International Electrotechnical Commission (ISO)/(IEC) JTC1/SC29/WG11/N13411, released January 2013 in Geneva, Switzerland, and available at http://mpeg.chiariglione.org/sites/default/files/files/stan dards/parts/docs/w13411.zip.

[0028] There are several channel-based “ambient sound” formats on the market. They range, for example, from the 5.1 home theater system (which has been the most successful in terms of making inroads into living rooms beyond stereo) to the 22.2 system developed by NHK (Nippon Hoso Kyokai or Japan Broadcasting Corporation). Content creators (eg Hollywood studios) would like to produce a movie's soundtrack once and not spend the effort remixing it for every speaker setup. Recently, Standards Development Organizations have been considering ways to provide an encoding for a standardized bitstream and subsequent decoding that is adaptive and agnostic to speaker geometry (and number) and acoustic conditions at the playback location (involving a renderer). ).

[0029] To provide this flexibility to content creators, a hierarchical set of elements can be used to represent a sound field. The hierarchical set of elements can refer to a set of elements in which the elements are ordered such that a basic set of lower-ordered elements provides a complete representation of the modeled sound field. As the set is extended to include higher-order elements, the representation becomes more detailed, increasing resolution.

[0030] An example of a hierarchical set of elements is a set of spherical harmonic coefficients (SHC). The following expression demonstrates a description or representation of a sound field using SHC:

. Aqui

, c é a velocidade do som (~343 m/s) , {rr,θr,ϕr} é um ponto de referência (ou ponto de observação) , jn(•) é a função de Bessel esférica da ordem n, e

, são as funções de base harmônica esférica de ordem n e subordem m. Deve-se reconhecer que o termo em colchetes quadrados é uma representação do domínio de frequência do sinal (i.e.,S(ω,rr, θr,ϕr)) que pode ser aproximada pelas várias transformadas de tempo-frequência, como a transformada de Fourier discreta (DFT), a transformada de cosseno discreta (DCT) ou uma transformada em ondeleta. Outros exemplos de conjuntos hierárquicos incluem conjuntos de coeficientes de transformada em ondeleta e outros conjuntos de coeficientes de funções de base de multiresolução.[0031] The expression shows that the pressure pt at any point {rr,θr,ϕr} in the sound field, at time t, can be represented solely by the SHC,

. Here

, c is the speed of sound (~343 m/s) , {rr,θr,ϕr} is a reference point (or observation point), jn(•) is the spherical Bessel function of order n, and

, are the spherical harmonic basis functions of order n and suborder m. It should be recognized that the term in square brackets is a representation of the signal frequency domain (ie,S(ω,rr, θr,ϕr)) that can be approximated by various time-frequency transforms, such as the Fourier transform discrete (DFT), the discrete cosine transform (DCT) or a wavelet transform. Other examples of hierarchical sets include wavelet transform coefficient sets and other multiresolution base function coefficient sets.

[0032] FIG. 1 is a diagram illustrating spherical harmonic basis functions from zero order (n = 0) to fourth order (n = 4). As can be seen, for each order, there is an expansion of suborders m which is shown but not explicitly observed in the example in FIG. 1 for ease of illustration purposes.

pode ou ser fisicamente adquirido (ex., gravado) por várias configurações de arranjo de microfone ou, alternativamente, eles podem ser derivados de descrições baseada em canal ou baseada em objeto do campo de som. O SHC represente o áudio baseado na cena, onde o SHC pode ser inserido em um codificador de áudio para obter o SHC codificado que pode promover a transmissão ou armazenamento mais eficiente. Por exemplo, uma representação de quarta ordem que envolve coeficientes (1+4)2 (25, e portanto, de quarta ordem) pode ser usada.[0033] The SHC

can either be physically acquired (eg, recorded) by various microphone array configurations or, alternatively, they can be derived from channel-based or object-based descriptions of the sound field. SHC represents scene-based audio, where SHC can be inserted into an audio encoder to get encoded SHC which can promote more efficient transmission or storage. For example, a fourth-order representation involving coefficients (1+4)2 (25, and therefore fourth-order) can be used.

[0034] As noted above, SHC can be derived from a microphone recording using a microphone array. Several examples of how SHC can be derived from microphone arrays are described in Poletti, M., “Three-Dimensional Surround Sound Systems Based on Spherical Harmonics”, J. Audio Eng. Soc., Vol. 53, No. 11, November 2005, p. 1004-1025.

para o campo de som correspondente a um objeto de áudio individual podem ser expressos como:

onde i é

é a função de Hankel esférica (do segundo tipo) de ordem n, e {rs, θs, ϕs] é a localização do objeto. Conhecer a fonte de energia do objeto g(w) como uma função da frequência (por exemplo, usando técnicas de análise de frequência de tempo, como executar uma transformada de Fourier rápida no fluxo de PCM) nos permite converter cada objeto PCM e o local correspondente no SHC,

Além disso, pode ser mostrado (uma vez que o acima é uma decomposição linear e ortogonal) que os coeficientes

para cada objeto são aditivos. Desta maneira, uma multiplicidade de objetos PCM pode ser representada pelos coeficientes

(por exemplo, como uma soma dos vetores de coeficiente para os objetos individuais). Essencialmente, os coeficientes contêm informações sobre o campo de som (a pressão em função de coordenadas 3D), e o acima representa a transformação de objetos individuais para uma representação do campo de som global, na vizinhança do ponto de observação {rr,θr,ϕr}. As figuras restantes são descritas abaixo no contexto da codificação de áudio baseada em objeto e baseada em SHC.[0035] To illustrate how SHCs can be derived from an object-based description, consider the following equation. The coefficients

for the sound field corresponding to an individual audio object can be expressed as:

where i is

is the spherical Hankel function (of the second type) of order n, and {rs, θs, ϕs] is the location of the object. Knowing the energy source of the object g(w) as a function of frequency (e.g. using time frequency analysis techniques such as performing a fast Fourier transform on the PCM stream) allows us to convert each PCM object and location correspondent in the SHC,

Furthermore, it can be shown (since the above is a linear and orthogonal decomposition) that the coefficients

for each object are additive. In this way, a multiplicity of PCM objects can be represented by the coefficients

(for example, as a sum of the coefficient vectors for the individual objects). Essentially, the coefficients contain information about the sound field (the pressure as a function of 3D coordinates), and the above represents the transformation of individual objects to a representation of the global sound field, in the vicinity of the observation point {rr,θr, ϕr}. The remaining figures are described below in the context of object-based and SHC-based audio encoding.

[0036] FIG. 2 is a diagram illustrating a system 10 that can perform various aspects of the techniques described in this disclosure. As shown in the example of FIG. 2, the system 10 includes a content creator device 12 and a content consuming device 14. While described in the context of the content creator device 12 and the content consuming device 14, the techniques can be implemented in any context where SHCs (which may also be referred to as HOA coefficients) or any other hierarchical representation of a sound field are encoded to form a representative bitstream of the audio data. In addition, content creator device 12 may represent any form of computing device capable of implementing the techniques described in this disclosure, including a mobile phone (or cell phone), a tablet computer, a smartphone, or a desktop computer to provide some examples. . Similarly, content consuming device 14 may represent any form of computing device capable of implementing the techniques described in this disclosure, including a mobile phone (or cell phone), a tablet computer, a smartphone, a set-top box or a desktop computer. to provide some examples.

[0037] Content creator device 12 may be operated by a film studio or other entity that can generate multi-channel audio content for consumption by operators of content consumption devices, such as content consuming device 14. In some examples, the content creator device 12 can be operated by an individual user who would like to compress the HOA coefficients 11. Often, the content creator generates audio content in conjunction with video content. Content consuming device 14 may be operated by an individual. Content consuming device 14 may include an audio playback system 16, which may refer to any form of audio playback system capable of rendering SHC for playback as multi-channel audio content.

[0038] Content creator device 12 includes an audio editing system 18. Content creator device 12 obtains live recordings 7 in various formats (including directly as HOA coefficients) and audio objects 9, which the content creator device content 12 can edit using the audio editing system 18. A microphone 5 can capture the live recordings 7. The content creator can, during the editing process, render HOA coefficients 11 from audio objects 9 by listening the rendered speaker feeds in an attempt to identify various aspects of the sound field that need further editing. The content creator device 12 can then edit the HOA coefficients 11 (potentially indirectly through manipulation of different audio objects 9 from which the source HOA coefficients can be derived in the manner described above). The content creator device 12 may employ the audio editing system 18 to generate the HOA coefficients 11. The audio editing system 18 represents any system capable of editing audio data and producing the audio data as one or more coefficients of spherical source harmonics.

[0039] When the editing process is complete, content creator device 12 can generate a bitstream 21 based on HOA coefficients 11. That is, content creator device 12 includes an audio encoding device 20 that represents a device configured to encode or otherwise compress HOA coefficients 11 in accordance with various aspects of the techniques described in this disclosure for generating bitstream 21. Audio encoding device 20 may generate bitstream 21 for transmission, as an example, through a transmission channel, which may be a wired or wireless channel, a data storage device or the like. Bitstream 21 may represent an encoded version of HOA coefficients 11 and may include a primary bitstream and another side bitstream, which may be referred to as side channel information.

[0040] While shown in FIG. 2 as being directly transmitted to the content consuming device 14, the content creating device 12 may output the bitstream 21 to an intermediate device positioned between the content creating device 12 and the content consuming device 14. The intermediate device may store the bitstream 21 for later distribution to content consuming device 14, which can request the bitstream. The intermediate device may comprise a file server, a web server, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smartphone or any other device capable of storing the bitstream 21 for later retrieval by a decoder. audio. The intermediate device may reside in a content delivery network capable of streaming the bitstream 21 (and possibly in conjunction with the transmission of a corresponding video data stream) to subscribers, such as the content consuming device 14 , requesting bitstream 21.

[0041] Alternatively, content creator device 12 may store bitstream 21 on storage media such as a compact disc, digital video disc, high definition video disc, or other storage media, most of which are capable of being read by a computer and therefore may be referred to as computer-readable storage media or non-transient computer-readable storage media. In this context, broadcast channel can refer to the channels through which content stored for media is broadcast (and can include retail stores and other store-based distribution mechanisms). In any case, the techniques of this description should therefore not be limited in this regard to the example of FIG. two.

[0042] As further shown in the example of FIG. 2 , content creator device 14 includes an audio reproduction system 16. Audio reproduction system 16 may represent any audio reproduction system capable of reproducing multi-channel audio data. The audio reproduction system 16 may include a number of different renderers 22. The renderers 22 may each provide a different form of rendering, where the different forms of rendering may include one or more of several ways of performing audio positioning. vector-based amplitude (VBAP) and/or one or more of several ways to perform sound field synthesis. As used herein, "A and/or B" means "A or B", or both "A and B".

[0043] Audio reproduction system 16 may further include an audio decoding device 24. Audio decoding device 24 may represent a device configured to decode HOA coefficients 11' from bitstream 21, where HOA 11' may be similar to HOA 11 coefficients but differ due to lossy operations (eg quantization) and/or across the transmission channel. Audio reproduction system 16 may, after decoding bitstream 21, obtain HOA coefficients 11' and render HOA coefficients 11' to provide output speaker feeds 25. Speaker feeds 25 may drive one or more loudspeakers (which are not illustrated in the example of FIG. 2 for ease of illustration purposes).

[0044] To select the appropriate processor or, in some cases, generate an appropriate renderer, the audio reproduction system 16 may obtain speaker information 13 indicative of a number of speakers and/or a spatial geometry of the loudspeakers. - speakers. In some cases, the audio reproduction system 16 may obtain information from speaker 13 using a reference microphone and driving the speakers so as to dynamically determine information from speaker 13. In other cases or together with the dynamic determination of the information from the speaker 13, the audio reproduction system 16 can ask a user to interact with the audio reproduction system 16 and input the information from the speaker 13.

[0045] Audio playback system 16 may then select one of audio renderers 22 based on information from speaker 13. In some cases, audio playback system 16 may, when none of audio renderers 22 are within a threshold similarity measure (in terms of speaker geometry) for the speaker geometry specified in speaker information 13, generate one of audio renderers 22 based on speaker information 13 The audio playback system 16 may, in some cases, generate one of the audio renderers 22 based on speaker information 13 without first attempting to select an existing one of the audio renderers 22. One or more speakers 3 may then play the rendered speaker feeds 25.

[0046] In some cases, the audio playback system 16 may select any of the 22 audio renderers and may be configured to select one or more of the 22 audio renderers depending on the source from which the 21 bitstream is received ( such as a DVD player, a Blu-ray player, a smartphone, a tablet computer, a gaming system and a television to give a few examples). While any of the audio renderers 22 can be selected, often the audio processor used when creating the content provides a better (and possibly the best) form of rendering due to the content being created by the content creator 12 using this specific audio processor, i.e. the audio processor 5 in the example of FIG. 3. Selecting one of the audio renderers 22 that is the same or at least close (in terms of rendering form) can provide a better representation of the sound field and can result in a better ambient sound experience for the content consumer 14 .

[0047] In accordance with the techniques described in this disclosure, the audio encoding device 20 can generate the bitstream 21 to include the audio rendering information 2 ("render info 2"). The audio rendering information 2 may include a signal value that identifies an audio renderer used when generating multichannel audio content, i.e., the audio renderer 1 in the example of FIG. 3. In some cases, the signal value includes a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0048] In some cases, the signal value includes two or more bits that define an index that indicates that the bitstream includes an array used to render spherical harmonic coefficients for a plurality of speaker feeds. In some cases, when an index is used, the sign value further includes two or more bits that define a number of rows of the matrix included in the bitstream and two or more bits that define a number of columns of the matrix included in the bitstream. Using this information and given that each coefficient of the two-dimensional matrix is typically defined by a 32-bit floating point number, the bit size of the matrix can be calculated as a function of the number of rows, the number of columns and the size. of the floating point numbers that define each coefficient of the matrix, that is, 32 bits in this example.

[0049] In some cases, the signal value specifies a rendering algorithm used to render spherical harmonic coefficients for a plurality of speaker feeds. The rendering algorithm may include a matrix that is known to both the audio encoding device 20 and the decoding device 24. That is, the rendering algorithm may include applying a matrix in addition to other rendering steps, such as positioning ( for example VBAP, DBAP or simple placement) or NFC filtering. In some cases, the signal value includes two or more bits that define an index associated with one of a plurality of matrices used to render spherical harmonic coefficients for a plurality of speaker feeds. Again, both the audio encoding device 20 and the decoding device 24 can be configured with information indicating the plurality of matrices and the order of the plurality of matrices so that the index can uniquely identify a particular matrix of the plurality of matrices. Alternatively, the audio encoding device 20 may specify data in the bitstream 21 that define the plurality of matrices and/or the order of the plurality of matrices so that the index can uniquely identify a particular matrix of the plurality of matrices.

[0050] In some cases, the signal value includes two or more bits that define an index associated with one of a plurality of rendering algorithms used to render spherical harmonic coefficients for a plurality of speaker feeds. Again, both the audio encoding device 20 and the decoding device 24 can be configured with information indicating the plurality of rendering algorithms and the order of the plurality of rendering algorithms so that the index can uniquely identify a particular matrix of the plurality of matrices. Alternatively, the audio encoding device 20 may specify data in the bitstream 21 that define the plurality of matrices and/or the order of the plurality of matrices so that the index can uniquely identify a particular matrix of the plurality of matrices.

[0051] In some cases, audio encoding device 20 specifies audio rendering information 2 on an audio frame basis in the bitstream. In other cases, the audio encoding device 20 specifies the audio rendering information 2 once in the bitstream.

[0052] Decoding device 24 can then determine the audio rendering information 2 specified in the bitstream. Based on the signal value included in the audio rendering information 2, the audio playback system 16 can render a plurality of speaker feeds 25 based on the audio rendering information 2. As noted above, the value of signal may, in some cases, include a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds. In this case, the audio playback system 16 can configure one of the audio renderers 22 with the matrix, this using one of the audio renderers 22 to render the speaker feeds 25 based on the matrix.

[0053] In some cases, the signal value includes two or more bits that define an index indicating that the bitstream includes an array used to render HOA coefficients 11' for speaker feeds 25. Decoding device 24 can parse the array from the bitstream in response to the index, after which the audio playback system 16 can configure one of the audio renderers 22 with the parsed array and invoke this one of the processors 22 to render the speaker feeds 25. When the sign value includes two or more bits that define a number of rows of the matrix included in the bitstream and two or more bits that define a number of columns of the matrix included in the bitstream, the decoding device 24 can parse the matrix of the bitstream in response to the index and based on the two or more bits that define a number of rows and the two or more bits that define the number of columns in the manner described above.

[0054] In some cases, the signal value specifies a rendering algorithm used to render the HOA coefficients 11' for the speaker feeds 25. In these cases, some or all of the audio renderers 22 may execute these rendering algorithms . The audio player 16 can then use the specified rendering algorithm, for example one of the audio renderers 22, to render the speaker feeds 25 from the HOA coefficients 11'.

[0055] When the signal value includes two or more bits that define an index associated with one of a plurality of matrices used to render HOA coefficients 11' for speaker feeds 25, some or all of the audio renderers 22 can represent this plurality of matrices. In this way, the audio reproduction system 16 can render the speaker feeds 25 from the HOA coefficients 11' using one of the audio renderers 22 associated with the index.

[0056] When the signal value includes two or more bits that define an index associated with one of a plurality of rendering algorithms used to render HOA coefficients 11' for speaker feeds 25, some or all of the renderers of audio 34 can represent these rendering algorithms. In this way, the audio reproduction system 16 can render the speaker feeds 25 from the spherical harmonic coefficients 11' using one of the audio renderers 22 associated with the index.

[0057] Depending on how often this audio rendering information is specified in the bitstream, the decoding device 24 may determine the audio rendering information 2 on a per audio frame or a one-time basis.

[0058] By specifying the audio rendering information 3 in this way, the techniques can potentially result in better reproduction of multi-channel audio content and in accordance with the manner in which the content creator 12 intended the multi-channel audio content to be reproduced . As a result, the techniques can provide a more immersive ambient sound or multi-channel audio experience.

[0059] In other words and as noted above, Higher Order Ambisonics (HOA) may represent a way in which to describe the directional information of a sound field based on a spatial Fourier transform. Typically, the higher the order of Ambisonics N, the greater the spatial resolution, the greater the number of spherical harmonic coefficients (SH) (N+1)A2 and the greater the bandwidth required to transmit and store the data.

[0060] A potential advantage of this description is the possibility to reproduce this sound field in most any of the speaker configurations (eg 5.1, 7.1 22.2, etc.). The conversion of the sound field description into M speaker signals can be done through a static rendering matrix with (N+1)2 inputs and M outputs. Consequently, each speaker configuration may require a dedicated rendering matrix. There may be several algorithms to calculate the rendering matrix for a desired speaker configuration, which can be optimized for certain objective or subjective measures, such as the Gerzon criteria. For irregular speaker configurations, the algorithms can become complex due to iterative numerical optimization procedures such as convex optimization. To calculate a render matrix for irregular speaker layouts with no wait time, it can be beneficial to have sufficient compute resources available. Irregular speaker configurations can be common in home living room environments due to architectural restrictions and aesthetic preferences. Therefore, for the best reproduction of sound fields, a rendering matrix optimized for such a scenario may be preferred as it may allow sound field reproduction more accurately.

[0061] As an audio decoder generally does not require a lot of computational resources, the device may not be able to calculate an irregular rendering matrix in a consumer-friendly time. Various aspects of the techniques described in this disclosure may provide for the use of a cloud-based computing approach as follows: 1. The audio decoder may send over an Internet connection the speaker coordinates (and, in some cases, also SPL measurements taken with a calibration microphone) to a server; 2. The cloud-based server can calculate the rendering matrix (and possibly a few different versions, so that the client can later choose between these different versions); and 3. The server can then send the rendering matrix (or different versions) back to the audio decoder via the Internet connection.

[0062] This approach may allow the manufacturer to keep the manufacturing costs of an audio decoder low (because a powerful processor may not be needed to calculate these irregular rendering matrices), while also facilitating more optimal audio reproduction compared to rendering matrices typically designed for speaker configurations or geometries. The algorithm for calculating the rendering matrix can also be optimized after an audio decoder is shipped, potentially reducing costs for hardware overhauls or even recalls. The techniques can also, in some cases, gather a lot of information about different consumer product speaker configurations that can be beneficial for future product developments.

[0063] In some cases, the system shown in FIG. 3 may not flag audio rendering information 2 in bitstream 21 as described above, but rather flag this rendering audio information 2 as separate metadata from bitstream 21. Alternatively or in conjunction with the above, the system shown in FIG. 3 may signal a portion of the audio rendering information 2 in the bitstream 21 as described above and may signal a portion of this audio rendering information 3 as separate metadata from the bitstream 21. In some instances, the audio encoding device 20 may output these metadata, which can then be uploaded to a server or other device. The audio decoding device 24 can then download or otherwise retrieve this metadata, which is then used to augment the audio rendering information extracted from the bitstream 21 by the audio decoding device 24. The bitstream 21 is formed in accordance with the information rendering aspects of the techniques is described below with respect to the examples of FIGs. 8A-8D.

[0064] FIG. 3 is a block diagram illustrating in more detail an example of the audio encoding device 20 shown in the example of Fig. 2 which can perform various aspects of the techniques described in this disclosure. The audio encoding device 20 includes a content analysis unit 26, a vector-based decomposition unit 27 and a direction-based decomposition unit 28. Although briefly described below, more information about the audio encoding device 20 and the various aspects of compressing or otherwise encoding HOA coefficients are available in International Patent Application Publication No. WO 2014/194099 entitled “INTERPOLATION FOR DECOMPOSED REPRESENTATIONS OF A SOUND FIELD” filed May 29, 2014.

[0065] Content analysis unit 26 represents a unit configured to analyze the content of HOA coefficients 11 to identify whether HOA coefficients 11 represent content generated from a live recording or an audio object. The content analysis unit 26 can determine whether the HOA coefficients 11 were generated from a recording of a real sound field or an artificial audio object. In some cases, when the framed HOA coefficients 11 were generated from a recording, the content analysis unit 26 passes the HOA coefficients 11 to the vector-based decomposition unit 27. In some cases, when the HOA coefficients frames 11 were generated from a synthetic audio object, the content analysis unit 26 passes the HOA coefficients 11 to the direction-based decomposition unit 28. The directional-based synthesis unit 28 may represent a unit configured to perform a directional basis synthesis of HOA 11 coefficients to generate a directional basis 21 bitstream.

[0066] As shown in the example of FIG. 3, the vector-based decomposition unit 27 may include a linear invertible transform (LIT) unit 30, a parameter calculation unit 32, a reorder unit 34, a foreground selection unit 36, a compensation unit. 38, a psychoacoustic audio encoder unit 40, a bitstream generation unit 42, a sound field analysis unit 44, a coefficient reduction unit 46, a background selection (BG) unit 48, a spatiotemporal interpolation unit 50 and a quantization unit 52.

[0067] Linear invertible transform unit (LIT) 30 receives HOA coefficients 11 in the form of HOA channels, each channel representative of a block or frame of a coefficient associated with a given order, sub-order of spherical base functions ( which can be denoted as HOA[k], where k can denote the current frame or block of samples). The HOA coefficient matrix 11 can have dimensions D: M x (N+1)2.

[0068] The LIT unit 30 may represent a unit configured to perform a form of analysis referred to as a singular value decomposition. Although described with respect to SVD, the techniques described in this description can be performed with respect to any transformation or similar decomposition that provides linearly uncorrelated, energy-packed output sets. Also, reference to "sets" in this disclosure is generally intended to refer to non-zero sets unless specifically stated otherwise and is not intended to refer to the classical mathematical definition of sets which includes the so-called "empty set". An alternate transform may comprise a principal component analysis, which is often referred to as "PCA". Depending on the context, PCA can be referred to by several different names, such as the discrete Karhunen-Loeve transform, the Hotelling transform, proper orthogonal decomposition (POD) and eigenvalue decomposition (EVD) to name a few examples. The properties of such operations that are conducive to the underlying goal of audio data compression are “power compression” and “decoding” of multichannel audio data.

[0069] In any case, assuming that the LIT unit 30 performs a singular value decomposition (which, again, may be referred to as “SVD”) for the purposes of example, the LIT unit 30 can transform the HOA coefficients 11 into two or more sets of transformed HOA coefficients. The “sets” of transformed HOA coefficients can include vectors of transformed HOA coefficients. In the example of FIG. 3, the LIT unit 30 can perform SVD against HOA coefficients 11 to generate a so-called matrix V, a matrix S, and a matrix U. SVD, in linear algebra, can represent a factorization of a real or complex matrix X of y-by-z (where X can represent multichannel audio data, such as HOA 11 coefficients) as follows: X = USV* U can represent a real or complex y-by-y unitary matrix, where the y columns of U are known as the left singular vectors of multichannel audio data. S can represent a y-by-z diagonal matrix with non-negative real numbers on the diagonal, where the diagonal values of S are known as the singular values of the multichannel audio data. V* (which can denote a transposition of V) can represent a real or complex z-by-z unitary matrix, where the z columns of V* are known as the right singular vectors of the multichannel audio data.

[0070] In some examples, the matrix V* in the mathematical expression SVD referred to above is designated as the conjugate transposition of the matrix V to reflect that the SVD can be applied to matrices comprising complex numbers. When applied to matrices comprising only real numbers, the complex conjugate of the matrix V (or, in other words, the matrix V*) can be thought of as the transposition of the matrix V. Below, it is assumed, for ease of illustration, that the HOA 11 coefficients comprise real numbers, with the result that matrix V is produced via SVD instead of matrix V*. Furthermore, although denoted as the V matrix in this description, reference to the V matrix should be understood as referring to the transposition of the V matrix where appropriate. Although it is assumed to be the V matrix, the techniques can be applied in a similar way to HOA 11 coefficients with complex coefficients, where the output of the SVD is the V* matrix. Consequently, the techniques should not be limited in this regard to just providing the application of SVD to generate a matrix V, but may include the application of SVD to HOA coefficients 11 having complex components to generate a matrix V*.

[0071] In this way, the LIT unit 30 can perform SVD with respect to the HOA coefficients 11 to produce US[k] 33 vectors (which can represent a combined version of the S vectors and the U vectors) having dimensions D: M x (N2 )2, and vectors V[k] 35 having dimensions D: (N+1)2 x (N+1)2. Individual vector elements in the matrix US[k] can also be called XpS(k) while individual vectors in the matrix V[k] can also be called v(k~).

[0072] An analysis of the U, S and V matrices can reveal that the matrices carry or represent spatial and temporal characteristics of the underlying sound field represented above by X. Each of the N U-shaped vectors (of samples of length M) can represent separate audio signals normalized as a function of time (for the time period represented by M samples), which are orthogonal to each other and which have been decoupled from any spatial features (which may also be referred to as directional information). The spatial features, which represent the spatial shape and position (r, theta, fi) can instead be represented by the ith individual vectors, u(l\k'), in the matrix V (each of length (N+1)2) . The individual elements of each of the vectors v^Çk) can represent an HOA coefficient that describes the shape (including width) and position of the sound field for an associated audio object. Both vectors in matrix U and matrix V are normalized such that their root mean square energies are equal to unity. The energy of the audio signals in U is thus represented by the diagonal elements in S. Multiplying U and S to form US[k] (with the individual vector elements XPS(k)), thus represents the audio signal with energies. The ability of SVD decomposition to decouple the audio time signals (in U), their energies (in S), and their spatial characteristics (in V) can support various aspects of the techniques described in this disclosure. Furthermore, the model of synthesizing the underlying HOA coefficients [k], X, by a vector multiplication of US[k] and V[k] gives rise to the term “vector-based decomposition”, which is used throughout this document. .

[0073] Although described as being performed directly with respect to HOA 11 coefficients, the LIT unit 30 can apply the invertible linear transform to the derivatives of the HOA 11 coefficients. For example, the LIT unit 30 can apply SVD with respect to a density matrix power spectral derived from HOA coefficients 11. By performing SVD against the power spectral density (PSD) of the HOA coefficients rather than the coefficients themselves, the LIT 30 unit can potentially reduce the computational complexity of performing SVD in terms of one or more processor cycles and storage space , while achieving the same coding efficiency of the source audio as if the SVD were applied directly to the HOA coefficients.

[0074] Parameter calculation unit 32 represents a unit configured to calculate various parameters, such as a correlation parameter (R), directional property parameters (θ, Φ, r), and an energy property (e). Each of the parameters for the current frame can be denoted as R [k] , θ [k], Φ [k] , r [k] and e [k]. The parameter calculation unit 32 can perform an analysis and/or energy correlation (or so-called cross-correlation) against the US[k] vectors 33 to identify the parameters. The parameter calculation unit 32 can also determine the parameters for the previous frame, where the parameters of the previous frame can be denoted R[k-1], θ[k-1], Φ[k—1], r[k —1] and e[k—1], based on the previous frame of the US[k-1] vector and V[k-1] vectors. The parameter calculation unit 32 can supply the current parameters 37 and the previous parameters 39 to the reordering unit 34.

e uma matriz V[k] reordenada 35’ (que pode ser denotada matematicamente como

para uma unidade de seleção de som de primeiro plano (ou som predominante — PS) 36 (“unidade de seleção de primeiro plano 36”) e uma unidade de compensação de energia 38.[0075] The parameters calculated by the parameter calculation unit 32 can be used by the reordering unit 34 to reorder the audio objects to represent their evaluation or natural continuity over time. The reordering unit 34 can compare each of the parameters 37 of the first US vectors [k] 33 per turn against each of the parameters 39 for the second US vectors [k-1] 33. The reordering unit 34 can reorder (using, as an example, a Hungarian algorithm) the various vectors within the matrix US[k] 33 and the matrix V[k] 35 based on the current parameters 37 and the previous parameters 39 to produce a reordered US[k] matrix 33' ( which can be mathematically denoted as

and a 35' reordered matrix V[k] (which can be mathematically denoted as

to a foreground sound selection unit (or predominant sound — PS) 36 (“foreground selection unit 36”) and a power compensation unit 38.

[0076] The sound field analysis unit 44 may represent a unit configured to perform a sound field analysis with respect to HOA coefficients 11 in order to potentially achieve a target bit rate 41. The sound field analysis unit sound 44 can, based on analysis and/or a received target bit rate 41, determine the total number of instantiations of the psychoacoustic encoder (which may be a function of the total number of ambient or background channels (BGTOT) and the number of foreground channels or, in other words, predominant channels.The total number of psychoacoustic encoder instantiations can be denoted as numHOATransportChannels.

[0077] The sound field analysis unit 44 can also determine, again potentially reaching the target bit rate 41, the total number of foreground channels (nFG) 45, the minimum order of the background sound field (or , in other words ambient) (NBG or, alternatively, MinAmbHOAorder), the corresponding number of actual channels representing the minimum order of the background sound field (nBGa = (MinAmbHOAorder + 1)2), and indices (i) of the channels Additional HOA BGs to send (which may collectively be denoted as background channel information 43 in the example of FIG. 3. Background channel information 42 may also be referred to as background channel information 43. Each of the channels remaining to from numHOATransportChannels - nBGa, can be either an “additional background/environment channel”, an “active vector based predominant channel”, an “active directional based predominant signal”, or “completely inactive”. channel types can be the indic syntax element (as a “ChannelType”) by two bits (e.g. 00: directional based signal, 01: vector based predominant signal, 10: additional ambient signal, 11: idle signal). The total number of background or ambient signals, nBGa, can be given by (MinAmbHOAorder +1)2 + the number of times index 10 (in the example above) appears as a channel type in the bitstream for that frame.

[0078] The sound field analysis unit 44 can select the number of background (or in other words ambient) channels and the number of foreground (or in other words predominant) channels based on the rate bitrate 41, selecting more background and/or foreground channels when the target bitrate 41 is relatively higher (e.g., when the target bitrate 41 is equal to or greater than 512 Kbps). In one aspect the numHOATransportChannels can be set to 8 while the MinAmbHOAorder can be set to 1 in the header section of the bitstream. In this scenario, in each frame, four channels can be dedicated to represent the background or ambient portion of the sound field while the other 4 channels can, on a frame-by-frame basis, vary in channel type - for example, used as a additional background/environment channel or a foreground/dominant channel. The foreground/predominant signals can be one of the vector based or directional signals as described above.

[0079] In some cases, the total number of vector-based predominant signals for a frame can be given by the number of times the ChannalType index is 01 in the bitstream of that frame. In the above aspect, for each additional background/environment channel (eg corresponding to a ChannelType of 10), the corresponding information of which of the possible HOA coefficients (in addition to the first four) can be represented in that channel. The information, for fourth-order HOA content, can be an index to indicate HOA coefficients 5-25. The first four ambient HOA coefficients 1-4 can be sent all the time when minAmbHOAorder is set to 1, so the audio encoding device may only need to indicate one of the additional ambient HOA coefficients with an index of 5-25. Information could thus be sent using a 5-bit syntax element (for 4th-order content), which can be denoted as “CodedAmbCoeffIdx”. In either case, the sound field analysis unit 44 outputs the background channel information 43 and the HOA coefficients 11 to the background selection unit (BG) 36, the background channel information 43 to the reduction unit. of the coefficient 46 and the bitstream generation unit 42, and the nFG 45 for a foreground selection unit 36.

[0080] Background selection unit 48 may represent a unit configured to determine background or ambient HOA coefficients 47 based on background channel information (e.g., background sound field (NBG) and number (nBGa) ) and the indices (i) of the additional BG HOA channels to send). For example, when NBG is equal to one, the background selection unit 48 may select HOA coefficients 11 for each sampled audio frame having an order equal to or less than one. The background selection unit 48 may in this example then select HOA coefficients 11 having an index identified by one of the indices (i) as additional BG HOA coefficients, wherein the nBGa is provided to the bitstream generation unit 42 to be specified in the bitstream 21 so as to allow the audio decoding device, such as the audio decoding device 24 shown in the example of FIGs. 2 and 4, analyze the background HOA coefficients 47 from the bitstream 21. The background selection unit 48 may then output the ambient HOA coefficients 47 to the power compensation unit 38. The ambient HOA coefficients 47 may have dimensions D: M x [(NBG+1)2 + nBGa]. The ambient HOA coefficients 47 may also be referred to as "ambient HOA coefficients 47", where each of the ambient HOA coefficients 47 corresponds to a separate ambient HOA channel 47 to be encoded by the psychoacoustic audio encoder unit 40.

49) para a unidade de codificador de áudio psicoacústico 40, onde os sinais nFG 49 podem ter dimensões D: M x nFG e cada representa objetos de monoaudio. A unidade de seleção de primeiro plano 36 podem também produzir a matriz reordenada V[k] 35’ (ou v(1nFG)(k) 35’) que corresponde aos componentes de primeiro plano do campo de som para a unidade de interpolação espaço-temporal 50, onde um subconjunto da matriz reordenada V[k] 35’ correspondente aos componentes de primeiro plano pode ser denotado como matriz de primeiro plano V[k] 51k (que pode ser matematicamente denotada como

tendo dimensões D: (N+1)2 x nFG.[0081] The foreground selection unit 36 may represent a unit configured to select the reordered US[k] matrix 33' and the reordered V[k] matrix 35' representing the foreground or distinct components of the sound field based on nFG 45 (which can represent one or more indices that identify the foreground vectors). The foreground selection unit 36 can output nFG 49 signals (which can be denoted as a reordered US [k]1, „, nFG 49, FG1, ..., nfG[k] 49, or

49) to the psychoacoustic audio encoder unit 40, where the nFG signals 49 may have dimensions D:M x nFG and each represents monoaudio objects. The foreground selection unit 36 may also produce the reordered matrix V[k] 35' (or v(1nFG)(k) 35') which corresponds to the foreground components of the sound field for the space-space interpolation unit. temporal 50, where a subset of the reordered matrix V[k] 35' corresponding to the foreground components can be denoted as the foreground matrix V[k] 51k (which can be mathematically denoted as

having dimensions D: (N+1)2 x nFG.

[0082] Power offset unit 38 may represent a unit configured to perform power offset against ambient HOA coefficients 47 to compensate for power loss due to the removal of several of the HOA channels by background select unit 48. The power compensation unit 38 can perform an energy analysis with respect to one or more of the reordered matrix US[k] 33', the reordered matrix V[k] 35', the signals nFG 49, the foreground vectors V [k] 51k and ambient HOA coefficients 47 and then perform energy compensation based on energy analysis to generate energy compensated ambient HOA coefficients 47'. The power compensation unit 38 can output the power compensated ambient HOA coefficients 47' to the psychoacoustic audio encoder unit 40.

[0083] The spatiotemporal interpolation unit 50 may represent a unit configured to receive the 51k foreground V[k] vectors for the kth frame and the 51k-1 foreground V[k-1] vectors for the frame above (here the k-1 notation) and perform spatiotemporal interpolation to generate the interpolated foreground vectors V[k]. The spatiotemporal interpolation unit 50 can recombine the nFG signals 49 with the foreground vectors V[k] 51k to recover the reordered foreground HOA coefficients. The spatiotemporal interpolation unit 50 can then divide the reordered foreground HOA coefficients by the interpolated vectors V[k] to generate the interpolated nFG signals 49'. The spatiotemporal interpolation unit 50 can also output the foreground V[k] vectors 51k that were used to generate the foreground interpolated V[k] vectors so that an audio decoding device such as the audio decoding 24, can generate the interpolated foreground vectors V[k] and thus recover the foreground vectors V[k] 51k. The foreground vectors V[k] 51k used to generate the interpolated foreground vectors V[k] are denoted as the remaining foreground vectors V[k] 53. To ensure that the same V[k] and V[ k-1] are used in the encoder and decoder (to create the interpolated vectors V[k]) quantized/dequantized versions of the vectors can be used in the encoder and decoder. The spatiotemporal interpolation unit 50 can output the interpolated nFG signals 49' to the psychoacoustic audio encoder unit 46 and the interpolated foreground V[k] vectors 51k to the coefficient reduction unit 46.

[0084] Coefficient reduction unit 46 may represent a unit configured to perform coefficient reduction with respect to remaining foreground vectors V[k] 53 based on background channel information 43 to produce reduced vectors V[k ] foreground 55 to the quantization unit 52. The reduced foreground vectors V[k] 55 can have dimensions D: [(N+1)2 - (NBG+1)2-BGTOT] x nFG. The coefficient reduction unit 46 may, in this regard, represent a unit configured to reduce the number of coefficients in the remaining foreground vectors V[k] 53. In other words, the coefficient reduction unit 46 may represent a unit configured to eliminate the coefficients in the foreground V[k] vectors (which form the remaining foreground V[k] vectors 53) having little or no directional information. In some examples, the coefficients of the distinct or, in other words, foreground vectors V[k] corresponding to a first-order or zero-order basic function (which may be denoted as NBG) provide little directional information and therefore may be removed from the foreground V vectors (through a process that may be referred to as “coefficient reduction”). In this example, greater flexibility can be provided not only to identify the coefficients that correspond to NBG but to identify additional HOA channels (which can be denoted by the variable TotalOfAddAmbHOAChan) from the set of [(NBG +1)2+1, (N +1)2].

[0085] The quantization unit 52 may represent a unit configured to perform any form of quantization to compress the reduced foreground vectors V[k] 55 to generate coded foreground vectors V[k] 57, yielding the coded vectors V [k] 57 to the bitstream generating unit 42. In operation, the quantizing unit 52 may represent a unit configured to compress a spatial component of the sound field, i.e., one or more of the reduced vectors V[ k] foreground 55 in this example. The quantization unit 52 can perform any of the following 12 quantization modes, as indicated by a quantization mode syntax element denoted "NbitsQ": Value 0-3. 4: 5: of NbitsQ Reserved Quantization Mode Type Vector Scalar Quantization Without Huffman Encoding 6: 6-Bit Scalar Quantization without Huffman Encoding 7: 7-Bit Scalar Quantization without Huffman Encoding 8: 8-Bit Scalar Quantization without Huffman Encoding Huffman Encoding 16: 16-Bit Scalar Quantization without Huffman Encoding The quantization unit 52 can also perform predicted versions of any of the above types of quantization modes, where a difference between an element of (or a weight when the vector quantization is performed) of vector V of a previous frame and the element (or weight when vector quantization is performed) of vector V of a current frame is determined. The quantization unit 52 can then quantize the difference between the elements or weights of the current frame and the previous frame rather than the value of the vector V element of the current frame itself.

[0086] The quantization unit 52 can perform multiple forms of quantization with respect to each of the reduced foreground vectors [V][k] to obtain multiple encoded versions of the reduced foreground vectors [V][k]. The quantization unit 52 may select one of the encoded versions of the reduced foreground vectors [V][k] 55 as the encoded foreground vector V[k] 57. In other words, the quantization unit 52 may select a vector Unpredicted vector quantized V, predicted vector quantized V vector, non-Huffman encoded scalar quantized V vector, and Huffman encoded scalar quantized V vector to use as the output switched quantized V vector based on any combination of the criteria discussed in this revelation. In some examples, the quantization unit 52 may select a quantization mode from a set of quantization modes that includes a vector quantization mode and one or more scalar quantization modes and quantize an input vector V based on ( or according to) the selected mode. The quantization unit 52 can then provide the quantized vector V of the predicted vector (e.g., in terms of weight values or bits indicative thereof), vector V quantized per predicted vector (e.g., in terms of error values or bits indicative of the same), the non-Huffman encoded scalar quantized vector V and the Huffman encoded scalar quantized vector V for the bitstream generation unit 52 as the encoded foreground vectors V[k] 57. The quantization unit 52 can also provide the syntax elements indicative of the quantization mode (e.g. the NbitsQ syntax element) and any other syntax elements used to dequantize or otherwise reconstruct the vector V.

[0087] The psychoacoustic audio encoder unit 40 included in the audio encoding device 20 can represent multiple instances of a psychoacoustic audio encoder, each of which is used to encode a different audio object or HOA channel from each of the energy compensated ambient HOA coefficients 47' and the interpolated nFG signals 49' to generate encoded ambient HOA coefficients 59 and encoded nFG signals 61. The psychoacoustic audio encoder unit 40 can output the encoded ambient HOA coefficients 59 and the encoded nFG signals 61 to the bitstream generating unit 42.

[0088] The bitstream generating unit 42 included in the audio encoding device 20 represents a unit that formats data according to a known format (which may refer to a format known by a decoding device), thereby generating the vector-based bitstream 21 mode. The bitstream 21 may, in other words, represent encoded audio data, having been encoded in the manner described above. The bitstream generation unit 42 can represent a multiplexer in some examples, which can receive the coded foreground vectors V[k] 57, the coded ambient HOA coefficients 59, the coded nFG signals 61 and the channel information 43. The bitstream generation unit 42 can then generate a bitstream 21 based on the coded foreground vectors V[k] 57, the coded ambient HOA coefficients 59, the coded nFG signals 61, and the coded channel information. background 43. In this way, the bitstream generating unit 42 can thus specify the vectors 57 in the bitstream 21 to obtain the bitstream 21. The bitstream 21 can include a primary or main bitstream and one or more sidechannel bitstreams.

[0089] Various aspects of the techniques may also allow bitstream 46 generation unit, as described above, to specify audio rendering information 2 in bitstream 21. Although the current version of the upcoming 3D audio compression working draft provides flagging specific mix matrices within bitstream 21, the working draft does not provide the specification of renderers used in rendering HOA coefficients in bitstream 21. For HOA content, the equivalent of this mix matrix is the rendering matrix that converts the representation of HOA in the desired speaker feeds. Various aspects of the techniques described in this disclosure propose to further harmonize the sets of channel and HOA content characteristics by allowing the bitstream generation unit 46 to signal HOA rendering matrices within the bitstream (such as, for example, audio 2).

Tabela - Valor de usacConfigExtType (Tabela 1 no CD)

[0090] An exemplary signaling solution based on the coding scheme of mixing matrices and optimized for HOA is presented below. Similar to passing mix arrays, HOA render arrays can be flagged inside mpegh3daConfigExtension(). The techniques can provide a new ID_CONFIG_EXT_HOA_MATRIX extension type as set out in the following tables (italics and bold indicating changes to the existing table). Table - Syntax of mpegh3daConfigExtension() (Table 13 on CD)

Table - UsacConfigExtType Value (Table 1 on CD)

[0091] HOARenderingMatrixSet() bitfield can be equal in structure and functionality compared to DownmixMatrixSet(). Instead of inputCount(audioChannelLayout), the HOARenderingMatrixSet() can use the “equivalent” value NumOfHoaCoeffs, calculated in HOAConfig. Also, because the ordering of HOA coefficients can be fixed within the HOA decoder (see, eg, Appendix G on the CD), the HOARenderingMatrixSet does not need any equivalent to inputConfig(audioChannelLayout). Table 2 — Syntax of HOARenderingMatrixSet() (adopted from Table 15 on CD)

[0092] Various aspects of the techniques may also allow the bitstream generation unit 46, when compressing the HOA audio data (e.g., the HOA coefficients 11 in the example of FIG. 4), using a first compression scheme ( as the decompression compression scheme represented by vector-based decomposition unit 27), specify bitstream 21 so that bits corresponding to a second compression scheme (e.g. direction-based compression system or in directionality represented by the direction-based decomposition unit 28) are not included in the bitstream 21. For example, the bitstream generating unit 42 may generate the bitstream 21 so as not to include HOAPredictionInfo syntax elements or field that may be reserved for Use to specify prediction information between directional signals of the direction-based compression scheme. Examples of the bitstream 21 generated in accordance with various aspects of the techniques described in this description are shown in the examples of FIGs. 8E and 8F.

[0093] In other words, the prediction of directional signals can be part of the Predominant Sound Synthesis employed by the directional-based decomposition unit 28 and depends on the existence of ChannelType 0 (which may indicate a direction-based signal). When no direction-based signals are present within a frame, no directional signals prediction can be performed. However, the sideband information associated with HOAPredictionInfo() can, although unused, be written to each frame regardless of whether there are direction-based signals. When there is no directional signal within a frame, the techniques described in this description may allow the bitstream generation unit 42 to reduce the size of the sideband by not signaling HOAPredictionInfo in the sideband as set out in the following Table (where underlined italics denote additions): Table: HOAFrame Syntax

[0094] In this regard, the techniques may allow a device such as the audio encoding device 20 to be configured to, when compressing higher-order ambisonic audio data using a first compression scheme, specify a bitstream representative of a compressed version. higher-order ambisonic audio data that does not include bits corresponding to a second compression scheme also used to compress the higher-order ambisonic audio data.

[0095] In some cases, the first compression scheme comprises a vector-based decomposition compression scheme. In these and other cases, the vector-based decomposition compression scheme comprises a compression scheme that involves applying a single-value decomposition (or equivalents thereof described in more detail in this description) to the higher-order ambisonic audio data.

[0096] In these and other cases, the audio encoding device 20 can be configured to specify the bitstream that does not include the bits that correspond to at least one syntax element used to perform the second type of compression scheme. The second compression scheme may, as noted above, comprise a compression scheme based on directionality.

[0097] Audio encoding device 20 can also be configured to specify bitstream 21 so that bitstream 21 does not include the bits corresponding to a HOAPredictionInfo syntax element of the second compression scheme.

[0098] When the second compression scheme comprises a directionality-based compression scheme, the audio encoding device 20 can be configured to specify bitstream 21 so that bitstream 21 does not include the bits corresponding to a HOAPredictionInfo syntax element of the directionality-based compression scheme. In other words, the audio encoding device 20 can be configured to specify bitstream 21 so that bitstream 21 does not include the bits that correspond to at least one element of syntax used to perform the second type of compression schemes, at least least one syntax element indicative of a prediction between two or more direction-based signals. Again, when the second compression scheme comprises a directionality-based compression scheme, the audio encoding device 20 can be configured to specify bitstream 21 so that bitstream 21 does not include the bits corresponding to a HOAPredictionInfo syntax element of the directionality-based compression scheme, where the HOAPredictionInfo syntax element is indicative of a prediction between two or more direction-based signals.

[0099] Various aspects of the techniques may further allow the bitstream generation unit 46 to specify bitstream 21 in certain cases so that bitstream 21 does not include gain correction data. The bitstream generating unit 46 may, when the gain correction is suppressed, specify the bitstream 21 so that the bitstream 21 does not include the gain correction data. Examples of bitstream 21 generated in accordance with various aspects of the techniques as shown, as noted above, in the examples of FIGs. 8E and 8F.

[0100] In some cases, gain correction is applied when certain types of psychoacoustic encoding are performed given the relatively smaller dynamic range of these specific types of psychoacoustic encoding compared to other types of psychoacoustic encoding. For example, AAC has a relatively smaller dynamic range than unified voice and audio coding (USAC). When the compression scheme (such as a vector-based synthesis compression scheme or a direction-based compression scheme) involves USAC, bitstream generation unit 46 can signal in bitstream 21 that gain correction has been suppressed (e.g. , specifying a MaxGainCorrAmpExp syntax element in HOAConfig with a value of zero in bitstream 21), and then specifying bitstream 21 not to include gain correction data (in a HOAGainCorrectionData() field).

[0101] In other words, the MaxGainCorrAmpExp bitfield as part of HOAConfig (see Table 71 on CD) can control the extent to which the auto gain control module affects transport channel signals prior to USAC core encoding . In some cases, this module was developed for RM0 to improve the non-optimal dynamic range of the available AAC encoder implementation. With the switch from AAC to USAC core encoder during the integration phase, the dynamic range of the core encoder can improve and therefore the need for this gain control module may not be as critical as before.

[0102] In some cases, gain control functionality may be suppressed if MaxGainCorrAmpExp is set to 0. In such cases, the sideband information associated with HOAGainCorrectionData() may not be written to every HOA frame per the table above illustrating the “ HOAFrame syntax”. For the configuration where MaxGainCorrAmpExp is set to 0, the techniques described in this disclosure may not flag the HOAGainCorrectionData. Furthermore, in such a scenario, the inverse gain control module can still be bypassed, reducing the decoder complexity by about 0.05 MOPS per transport channel without any negative side effects.

[0103] In this regard, techniques can configure the audio encoding device 20 to, when gain correction is suppressed during compression of higher-order ambisonic audio data, specify the bitstream 21 representative of a compressed version of the audio data. higher order ambisonic audio so bitstream 21 does not include gain correction data.

[0104] In these and other cases, the audio encoding device 20 can be configured to compress the higher-order ambisonic audio data in accordance with a vector-based decomposition compression scheme to generate the compressed version of the audio data. higher-order ambisonics. Examples of the decompression compression scheme may involve applying a singular value decomposition (or its equivalents described in greater detail above) to the higher-order ambisonic audio data to generate the compressed version of the higher-order ambisonic audio data.

[0105] In these and other cases, the audio encoding device 20 can be configured to specify a MaxGainCorrAmbExp syntax element in bitstream 21 as zero to indicate that gain correction is suppressed. In some cases, audio encoding device 20 may be configured to specify, when gain correction is suppressed, bitstream 21 so that bitstream 21 does not include a HOAGainCorrection data field that stores gain correction data. . In other words, audio encoding device 20 can be configured to specify a MaxGainCorrAmbExp syntax element in bitstream 21 as zero to indicate that gain correction is suppressed and not include in the bitstream a HOAGainCorrection data field that stores the data. gain correction.

[0106] In these and other cases, the audio encoding device 20 may be configured to suppress gain correction when the compression of higher-order ambisonic audio data includes the application of unified audio speech coding and coding speech audio (USAC) to higher-order ambisonic audio data.

[0107] The previous potential optimizations for signaling various information in bitstream 21 can be adapted or otherwise updated in the manner described in greater detail below. Updates can be applied in conjunction with other updates discussed below, or used to update just various aspects of the optimizations discussed above. As such, each potential combination of updates for the optimizations described above is considered, including applying a single update described below for the optimizations described above or any particular combination of the updates described below for the optimizations described above.

O ID_CONFIG_EXT_HOA_MATRIX na tabela anterior fornece um recipiente no qual especificar a matriz de renderização, o recipiente denotado como “HoaRenderingMatrixSet()”.[0108] To specify a matrix in the bitstream, the bitstream generation unit 42 can, for example, specify an ID_CONFIG_EXT_HOA_MATRIX in an mpegh3daConfigExtension() of bitstream 21, as illustrated in bold and highlighted in the following Table. The following Table is representative of the syntax for specifying the mpegh3daConfigExtension() portion of bitstream 21: Table - mpegh3daConfigExtension() Syntax

The ID_CONFIG_EXT_HOA_MATRIX in the previous table provides a container in which to specify the rendering matrix, the container denoted as “HoaRenderingMatrixSet()”.

Como mostrado na Tabela acima, o HoaRenderingMatrixSet() inclui vários elementos de sintaxe diferentes, incluindo um numHoaRenderingMatrices, um HoaRendereringMatrixId, um CICPspeakerLayoutIdx, um HoaMatrixLenBits e um HoARenderingMatrix.[0109] The contents of the HoaRenderingMatrixSet() container can be defined according to the syntax established in the following Table: Table - HoaRenderingMatrixSet() Syntax

As shown in the Table above, the HoaRenderingMatrixSet() includes several different syntax elements, including a numHoaRenderingMatrices, a HoaRendereringMatrixId, a CICPspeakerLayoutIdx, a HoaMatrixLenBits, and a HoARenderingMatrix.

[0110] The numHoaRenderingMatrices syntax element can specify a number of HoaRendereringMatrixId definitions present in the bitstream element. The HoaRenderingMatrixId syntax element can represent a field that uniquely defines an ID for a standard HOA rendering matrix available on the decoder side or a passed HOA rendering matrix. In this regard, the HoaRenderingMatrixId can represent an example of the signal value that includes two or more bits that define an index that indicates that the bitstream includes a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds, or the value of signal that includes two or more bits that define an index associated with one of a plurality of matrices used to render spherical harmonic coefficients for a plurality of speaker feeds. The CICPspeakerLayoutIdx syntax element can represent a value that describes the output speaker layout for the given HOA rendering matrix and can correspond to a ChannelConfiguration element defined in ISO/IEC 23000 1-8. The HoaMatrixLenBits syntax element (which may also be referred to as “HoaRenderingMatrixLenBits”) can specify a length of the following bitstream element (for example, the HoaRenderingMatrix() container) in bits.

[0111] The HoaRenderingMatrix() container includes a NumOfHoaCoeffs followed by an outputConfig() container and an outputCount() container. The outputConfig() container can include channel configuration vectors specifying information about each speaker. The bitstream generating unit 42 can assume that this speaker information is known from the channel settings of the output layout. Each entry, outputConfig[i], can represent a data structure with the following members: AzimuthAngle (which can indicate the absolute value of the speaker's azimuth angle); AzimuthDirection (which can indicate the azimuth direction using, as an example, 0 for left and 1 for right); ElevationAngle (which can indicate the absolute value of the speaker's elevation angles); ElevationDirection (which can indicate the elevation direction using, for example, 0 for up and 1 for down); and isLFE (which can indicate whether the speaker is a Low Frequency Effect (LFE) speaker). The bitstream generation unit 42 can invoke a helper function, in some cases called “findSymmetricSpeakers”, which can further specify the following: pairType (which can store a value of SYMMETRIC (which means a symmetrical pair of two speakers) in some instance), CENTER, or ASYMMETRIC); and symmetricPair->originalPosition (which can indicate the position in the original channel configuration of the second (eg right) speaker in the group, for SYMMETRIC groups only). The outputCount() container can specify a number of speakers for which the HOA rendering matrix is defined.

Como mostrado na Tabela diretamente acima, o elemento de sintaxe numPairs é definido para o valor de saída a partir da invocação da função auxiliar findSymmetricSpeakers usando o outputCount e outputConfig e hasLfeRendering como entradas. O numPairs pode, portanto, indicar o número de pares de alto-falantes simétricos identificados na configuração de alto-falantes de saída que podem ser considerados para a codificação de simetria eficiente. O elemento de sintaxe precisionLevel na Tabela acima pode indicar uma precisão usada para quantização uniforme dos ganhos de acordo com a seguinte Tabela: Tabela - Tamanho da etapa de quantização uniforme de hoaGain em função de precisionLevel

[0112] The bitstream generation unit 42 can specify the HoaRenderingMatrix() container according to the syntax established in the following Table: Table - HoaRenderingMatrix() Syntax

As shown in the Table directly above, the numPairs syntax element is set to the output value from invoking the findSymmetricSpeakers helper function using the outputCount and outputConfig and hasLfeRendering as inputs. numPairs can therefore indicate the number of symmetrical speaker pairs identified in the output speaker configuration that can be considered for efficient symmetry encoding. The precisionLevel syntax element in the Table above can indicate a precision used for uniform quantization of gains according to the following Table: Table - Size of uniform quantization step of hoaGain as a function of precisionLevel

[0113] The gainLimitPerHoaOrder syntax element shown in the table above that establishes the syntax of HoaRenderingMatrix() can represent a flag indicating whether maxGain and minGain are specified individually for each order or for the entire HOA rendering matrix. The maxGain[i] syntax elements can specify a maximum real gain in the array for coefficients for the HOA order i expressed, as an example, in decibels (dB). The mixGain[i] syntax elements can specify a minimum real gain in the array for coefficients of the order HOA i expressed, as an example, in dB. The isFullMatrix syntax element can represent a flag that indicates whether the HOA rendering matrix is sparse or full. The firstSparseOrder syntax element can specify, in case the HOA render matrix was specified as sparse by the isFullMatrix syntax element, the first HOA order that is sparsely encoded. The isHoaCoefSparse syntax element can represent a bitmask vector derived from the firstSparseOrder syntax element. The lfeExists syntax element can represent a flag indicating whether one or more LFEs exist in outputConfig. The hasLfeRendering syntax element indicates whether the rendering array contains non-zero elements for one or more LFE channels. The zerothOrderAlwaysPositive syntax element can represent a flag indicative of whether the 0th order HOA has only positive values.

[0114] The isAllValueSymmetric syntax element can represent a flag indicative of whether all symmetric speaker pairs have equal absolute values in the HOA rendering matrix. The isAnyValueSymmetric syntax element represents a flag that indicates, when false, for example, whether some of the symmetric speaker pairs have equal absolute values in the HOA rendering matrix. The valueSymmetricPairs syntax element can represent a bitmask of length numPairs indicating the speaker pairs with value symmetry. The isValueSymmetric syntax element can represent a bitmask derived in the manner shown in Table 3 from the valueSymmetricPairs syntax element. The isAllSignSymmetric syntax element can indicate, when there are no value symmetries in the array, whether all pairs of symmetric speakers have at least sign-number symmetries. The isAnySignSymmetric syntax element can represent a flag indicative of whether there are at least some pairs of symmetric speakers with number sign symmetries. The signSymmetricPairs syntax element can represent a bitmask of length numPairs indicating the signal symmetry speaker pairs. The isSignSymmetric variable can represent a bitmask derived from the signSymmetricPairs syntax element in the manner shown above in the Table establishing the syntax of HoaRenderingMatrix(). The hasVerticalCoef syntax element can represent a flag indicative of whether the array is a horizontal-only HOA rendering array. The bootVal syntax element can represent a variable used in the decoding circuit.

[0115] In other words, bitstream generation unit 42 can parse audio renderer 1 to generate one or more of the above value symmetry information (e.g. any combination of one or more syntax element isAllValueSymmetric, element of isAnyValueSymmetric syntax, valueSymmetricPairs syntax element, isValueSymmetric syntax element, and valueSymmetricPairs syntax element) or get the value symmetry information. The bitstream generating unit 42 can specify the audio rendering information 2 in the bitstream 21 in the manner shown above so that the audio rendering information 2 includes the value signal symmetry information.

[0116] Also, in other words, bitstream generation unit 42 can also parse audio renderer 1 to generate one or more of the above signal symmetry information (e.g. any combination of one or more syntax elements isAllSignSymmetric, isAnySignSymmetric syntax element, signSymmetricPairs syntax element, isSignSymmetric syntax element, and signSymmetricPairs syntax element) or get the sign symmetry information. The bitstream generating unit 42 can specify the audio rendering information 2 in the bitstream 21 in the manner shown above so that the audio rendering information 2 includes the audio signal symmetry information.

[0117] When determining the value symmetry information and the signal symmetry information, the bitstream generation unit 42 can analyze the various values of the audio processor 1, which can be specified as a matrix. A rendering matrix can be formulated as a pseudo-inverse of an R matrix. In other words, to render (N+1)2 HOA channels (indicated as Z below) for L speaker signals (indicated by the column vector, p, of the loudspeaker L signals), the following equation can be given: Z = R * p. To arrive at the rendering matrix that produces the L speaker signals, the inverse of the R matrix is multiplied by the Z HOA channels as shown in the following equation: p = R-1 * Z. -speakers, L, is the same as the number of channels HOA Z, (N+1)2, the matrix R will not be square and a perfect inverse cannot be determined. As a result, the pseudo-inverse can be used instead, which is defined as follows: pinv(R) = RT(R * RT)-1, where RT denotes the transposition of the matrix R. Substituting R-1 into the equation above, the resolution for the loudspeaker signals L denoted by the column vector p can be mathematically denoted as follows: p = pinv(R) * Z = RT(R * RT)-1 * Z.

[0118] The entries of the matrix R are the values of the spherical harmonics for the speaker positions with (N+1)2 rows for the different spherical harmonics and L columns for the speakers. The bitstream generation unit 42 can determine speaker pairs based on the values for the speakers. By analyzing the values of the spherical harmonics for the speaker positions, the bitstream generation unit 42 can determine based on the values whose speaker positions are even (e.g., as pairs can have similar, nearly equal, or equal values , but with opposite signs).

[0119] After identifying the pairs, the bitstream generation unit 42 can determine for each pair whether the pairs have the same value or nearly the same value. When all pairs have the same value, bitstream generation unit 42 can set the syntax element isAllValueSymmetric to one. When all pairs do not have the same value, bitstream generation unit 42 can set the isAllValueSymmetric syntax element to zero. When one or more but not all pairs have the same value, bitstream generation unit 42 may set the isAnyValueSymmetric syntax element to one. When none of the pairs have the same value, bitstream generation unit 42 can set the isAnyValueSymmetric syntax element to zero. For pairs with symmetric values, the bitstream generation unit 42 can specify only one value instead of two separate values for the speaker pair, thus reducing the number of bits used to represent the audio rendering information 2 (for example, the array in this example) in bitstream 21.

[0120] When there are no value symmetries between the pairs, the bitstream generation unit 42 can also determine for each pair whether the loudspeaker pairs have sign symmetry (meaning that a loudspeaker has a negative value while the other speaker has a positive value). When all pairs have sign symmetry, bitstream generation unit 42 can set the syntax element isAllSignSymmetric to one. When all pairs do not have sign symmetry, bitstream generation unit 42 can set the syntax element isAllSignSymmetric to zero. When one or more but not all pairs have sign symmetry, the bitstream generation unit 42 may set the isAnySignSymmetric syntax element to one. When none of the pairs have sign symmetry, the bitstream generation unit 42 may set the isAnySignSymmetric syntax element to zero. For pairs with symmetrical signals, the bitstream generation unit 42 may specify only one or no signals instead of two separate signals for the speaker pair, thus reducing the number of bits used to represent the audio rendering information 2 (e.g. the array in this example) in bitstream 21.

[0121] The bitstream generation unit 42 can specify the DecodeHoaMatrixData() container shown in the Table establishing the syntax of HoaRenderingMatrix() according to the syntax shown in the following Table: Table - DecodeHoaMatrixData Syntax

[0122] The hasValue syntax element in the previous Table establishing the DecodeHoaMatrixData syntax can represent a flag indicative of whether the array element is sparsely encoded. The signMatrix syntax element can represent a matrix with the sign values of the HOA rendering matrix in, as an example, linearized vector form. The hoaMatrix syntax element can represent HOA rendering matrix values in, as an example, linearized vector form. The bitstream generation unit 42 can specify the DecodeHoaGainValeu() container shown in the Table establishing the DecodeHoaMatrixData syntax according to the syntax shown in the following Table: Table - DecodeHoaGainValue Syntax

[0123] The bitstream generation unit 42 can specify the readRange() container shown in the Table establishing the DecodeHoaGainValeu syntax according to the syntax specified in the following Table: Table 7 - ReadRange Syntax

[0124] Although not illustrated in the example of FIG. 3, audio encoding device 20 may also include a bitstream output unit that switches the bitstream output of audio encoding device 20 (e.g., between direction-based bitstream 21 and vector-based bitstream 21) based on whether a current frame should be encoded using directional based synthesis or vector based synthesis. The bitstream output unit may perform switching based on the syntax element emitted by the content analysis unit 26 indicating whether a direction-based synthesis has been performed (as a result of detecting that HOA coefficients were generated from an object audio) or a vector-based synthesis was performed (as a result of detecting that HOA coefficients were recorded). The bitstream output unit can specify the correct header syntax to indicate the switch or current encoding used for the current frame along with a respective one of the 21 bitstreams.

[0125] In addition, as noted above, the sound field analysis unit 44 can identify ambient HOA BGTOT 47 coefficients, which may change on a frame-by-frame basis (although sometimes the BGTOT may remain constant or equal across two or more adjacent frames (in time)). The change in BGTOT can result in changes in the coefficients expressed in the reduced foreground V[k] vectors. Changing BGTOT can result in background HOA coefficients (which may also be referred to as “ambient HOA coefficients”) that change on a frame-by-frame basis (although, again, sometimes BGTOT can remain constant or equal across two or more frames adjacent (in time)). The changes often result in an energy change for the aspects of the sound field represented by the addition or removal of additional ambient HOA coefficients and the corresponding removal of coefficients from or addition of coefficients to the reduced foreground V[k] vectors.

[0126] As a result, the sound field analysis unit 44 can further determine when the ambient HOA coefficients change from frame to frame and generate a flag or other syntactic element indicative of the change to the ambient HOA coefficient in terms of being used for represent the ambient components of the sound field (where the change may also be referred to as a “transition” of the ambient HOA coefficient or as a “transition” of the ambient HOA coefficient). In particular, coefficient reduction unit 46 can generate the flag (which may be denoted as an AmbCoeffTransition flag or an AmbCoeffIdxTransition flag) by supplying the flag to the bitstream generation unit 42 so that the flag can be included in the bitstream 21 (possibly as part of the side channel information).

[0127] The coefficient reduction unit 46 can, in addition to specifying the ambient coefficient transition indicator, also modify how the reduced foreground vectors V[k] 55 are produced. In an example, when determining that one of the coefficients HOA environment is in transition during the current frame, the coefficient reduction unit 46 can specify a vector coefficient (which may also be referred to as a “vector element” or “element”) for each of the V vectors of the reduced foreground vectors V[k] 55 which corresponds to the transitional ambient HOA coefficient. Again, the transitional ambient HOA coefficient can add to or remove from the total number of BGTOT of the background coefficients. Therefore, the resulting change in the total number of background coefficients affects whether or not the ambient HOA coefficient is included in the bitstream and whether the corresponding element of the V vectors is included for the specified V vectors in the bitstream in the second and third configuration modes described. above. More information on how the coefficient reduction unit 46 can specify reduced foreground V[k] vectors to overcome changes in energy is provided in Application Serial No. , filed on January 12, 2015.

[0128] FIG. 4 is a block diagram illustrating the audio decoding device 24 of Fig. 2 in greater detail. As shown in the example of FIG. 4 audio encoding device 24 may include an extractor unit 72, a renderer reconstruction unit 81, a directional base reconstruction unit 90 and a vector based reconstruction unit 92. audio decoding device 24 and the various aspects of decompressing or otherwise decoding HOA coefficients are available in International Patent Application Publication No. WO 2014/194099 entitled “INTERPOLATION FOR DECOMPOSED REPRESENTATIONS OF A SOUND FIELD” filed May 29, 2014.

[0129] The extraction unit 72 may represent a unit configured to receive the bitstream 21 and extract the audio rendering information 2 and the various encoded versions (e.g. a directional-based encoded version or a vector-based encoded version) of the HOA coefficients 11. In other words, the Higher Order Amnisonics (HOA) rendering matrices can be transmitted by the audio encoding device 20 to allow control over the HOA rendering process in the audio playback system 16. Transmission can be facilitated via the mpegh3daConfigExtension of Type ID_CONFIG_EXT_HOA_MATRIX shown above. The mpegh3daConfigExtension can contain multiple HOA render arrays for different speaker playback configurations. When the HOA render arrays are transmitted, the audio encoding device 20 signals, for each HOA render array signal, the associated target speaker layout which together with the HoaOrder determines the rendering array dimensions.

[0130] Streaming a single HoaRenderingMatrixId allows referencing a predefined HOA rendering matrix available on the audio playback system 16, or a HOA rendering matrix transmitted from outside the audio bitstream 21. In some cases, each HOA rendering matrix is assumed to be normalized in N3D and follows the ordering of the HOA coefficients as defined in bitstream 21.

[0131] The findSymmetricSpeakers function can, as noted above, indicate a number and position of all speaker pairs within the given speaker configuration that are symmetrical with respect, as an example, to the midplane of a listener in the so-called “sweet spot”. This helper function can be defined as follows: int findSymmetricSpeakers(int outputCount, SpeakerInformation* outputConfig, int hasLfeRendering); The extraction unit 72 can invoke the createSymSigns function to calculate a vector of 1.0 and -1.0 values which can then be used to generate the matrix elements associated with symmetric speakers. This createSymSigns function can be defined as follows: void createSymSigns(int* symSigns, int hoaOrder) { int n, m, k = 0; to (n = 0; n<=hoaOrder; ++n) { to (m = -n; m<=n; ++m) symSigns[k++] = ((m>=0)*2)-1; } }

[0132] Extraction unit 72 can invoke the create2dBitmask function to generate a bit mask to identify HOA coefficients that are only used in the horizontal plane. The create2dBitmask function can be defined as follows: void create2dBitmask(int* bitmask, int hoaOrder) { int n, m, k = 0; bitmask[k++] = 0; to (n = 1; n<=hoaOrder; ++n) { to (m = -n; m<=n; ++m) bitmask[k++] = abs(m)!=n; } }

[0133] To decode the HOA Rendering Matrix Coefficients, the extraction unit 72 can first extract the HoaRenderingMatrixSet() syntax element, which, as noted above, can contain one or more HOA rendering matrices that can be applied to get an HOA rendering for a desired speaker layout. In some cases, a given bitstream may not contain more than one instance of HoaRenderingMatrixSet(). The HoaRenderingMatrix() syntax element contains the HOA rendering matrix information (which may be denoted as processor 2 information in the example of FIG. 4). The extraction unit 72 can first read configuration information, which can guide the decoding process. Then, extraction unit 72 reads the array elements accordingly.

[0134] In some cases, extraction unit 72 initially reads the precisionLevel and gainLimitPerOrder fields. When the gainLimitPerOrder flag is set, extraction unit 72 reads and decodes the maxGain and minGain fields for each HOA order separately. When the gainLimitPerOrder flag is not set, extraction unit 72 reads and decodes the maxGain and minGain fields once and applies these fields to all HOA orders during the decoding process. In some cases, the minGain value must be between 0db and -69dB. In some cases, the maxGain value must be between 1dB and 111dB lower than the minGain value. FIG. 9 is a diagram illustrating an example of HOA dependent min and max gains within an HOA rendering matrix.

[0135] Extract unit 72 can then read the isFullMatrix flag, which can signal whether a matrix is defined as full or partially sparse. When the array is set to partially sparse, extracting unit 72 reads the following field (for example, the firstSparaseOrder syntax element), which specifies the HOA order from which the HOA rendering array is sparsely encoded. HOA rendering matrices can often be sparse for low order and become sparse for higher orders depending on the speaker playback configuration. FIG. 10 is a diagram illustrating a partially sparse 6th order HOA rendering matrix for 22 speakers. The sparsity of the matrix illustrated in FIG. 10 starts with the 26th HOA coefficient (HOA order 5).

[0136] Depending on whether one or more Low Frequency Effects (LFE) channels exist within the speaker playback configuration (indicated by the lfeExists syntax element), the extraction unit 72 can read the hasLfeRendering field. When hasLfeRendering is not set, the extraction unit 72 is configured to assume that the array elements related to the LFE channels are digital zeros. The next field read by extraction unit 72 is the zerothOrderAlwaysPositive indicator, which signals whether the array elements associated with the 0-order coefficient are positive. In this case where zerothOrderAlwaysPositive indicates that the zero-order HOA coefficients are positive, the extraction unit 72 determines that the numeric signals are not encoded for the rendering matrix coefficients corresponding to the zero-order HOA coefficients.

[0137] In what follows, the properties of the HOA rendering matrix can be signaled for speaker pairs symmetrical about the midplane. In some cases, there are two related symmetry properties a) value symmetry and b) signal symmetry. In the case of value symmetry, the matrix elements of the left speaker of the pair of symmetrical speakers are not encoded, but rather the extraction unit 72 derives those elements from the decoded matrix elements of the right speaker using the function createSymSigns helper that does the following: pairIdx = outputConfig[j].symmetricPair->originalPosition; hoaMatrix[i * outputCount + j] = hoaMatrix[i * outputCount + pairIdx]; and signMatrix[i * outputCount + j] = symSigns[i] * signMatrix[i * outputCount + pairIdx].

[0138] When a pair of speakers is not symmetrical in value, then the matrix elements can be symmetrical with respect to their number signals. When a pair of speakers is symmetrical in sign, the numeric signals of the left speaker array elements of the pair of symmetrical speakers are not encoded and the extraction unit 72 derives these numeric signals from the numeric signals of the symmetrical speaker pair. array associated with the right speaker using the createSymSigns helper function, which performs the following: pairIdx = outputConfig[j].symmetricPair->originalPosition; signMatrix[i * outputCount + j] = symSigns[i] * signMatrix[i * outputCount + pairIdx];

[0139] FIG. 11 is a diagram illustrating signaling symmetry properties. A pair of speakers cannot be defined as symmetrical in value and symmetrical in signal at the same time. The final decoding flag hasVerticalCoef specified if only array elements associated with circular (ie 2D) HOA coefficients are encoded. If hasVerticalCoef is not defined, array elements associated with HOA coefficients defined with the helper function create2dBitmask are set to digital zero.

[0140] That is, the extraction unit 72 can extract the audio rendering information 2 according to the process shown in FIG. 11. The extraction unit 72 can first read the isAllValueSymmetric syntax element from bitstream 21 (300). When the isAllValueSymmetric syntax element is set to one (or, in other words, a true boolean), the extraction unit 72 can iterate through the value of the numPairs syntax element, setting the valueSymmetricPairs array syntax element to a value of one (effectively indicating that all speaker pairs are symmetrical values) (302).

[0141] When the isAllValueSymmetric syntax element is set to zero (or, in other words, a false boolean), extraction unit 72 can then read the isAnyValueSymmetric syntax element (304). When the isAnyValueSymmetric syntax element is set to one (or, in other words, a true boolean), the extraction unit 72 can iterate through the value of the numPairs syntax element, setting the valueSymmetricPairs array syntax element to a read bit sequentially from the bitstream (306). The extraction unit 72 can also get the isAnySignSymmetric syntax element for any of the pairs having a valueSymmetricPairs syntax element set to zero (308). The extraction unit 72 can then iterate through the number of pairs again and, when valueSymmetricPairs equals zero, set a bit of signSymmetricPairs to a value read from bitstream 21 (310).

[0142] When the isAnyValueSymmetric syntax element is set to zero (or, in other words, a false boolean), extraction unit 72 can then read the isAllValueSymmetric syntax element from bitstream 21 (312). When the isAllSignSymmetric syntax element is set to a value of one (or, in other words, a true boolean), the extraction unit 72 can iterate through the value of the numPairs syntax element, setting the signSymmetricPairs array syntax element to a value of one (effectively indicating that all speaker pairs are symmetrical in sign) (316).

[0143] When the isAllSignSymmetric syntax element is set to zero (or, in other words, a false boolean), extraction unit 72 can then read the isAnySignSymmetric syntax element from bitstream 21 (316). The extraction unit 72 may iterate through the value of the numPairs syntax element, setting the syntax element of the signSymmetricPairs array to a bit read sequentially from the bitstream (318). The bitstream generation unit 42 may perform a reciprocal process with respect to that described above with respect to the extraction unit 72 to specify the value symmetry information, the signal symmetry information, or a combination of both the value symmetry information. and signal.

[0144] The renderer reconstruction unit 81 can represent a configuration unit d to reconstruct a processor based on the audio rendering information 2. That is, using the aforementioned properties, the renderer reconstruction unit 81 can read a array of array element gain values. To read the absolute gain value, renderer reconstruction unit 81 can invoke the DecodeGainValue() function. The renderer reconstruction unit 81 can invoke the ReadRange() function of the alphabet index to uniformly decode the gain values. When the decoded gain value is not a digital zero, the renderer reconstruction unit 81 can read the numerical signal value additionally (according to Table a below). When the array element is associated with an HOA coefficient that has been flagged as sparse (via isHoaCoefSparse) the hasValue flag precedes the value of gainValueIndex (see Table b). When the hasValue flag is zero, this element is set to digital zero and no gainValueIndex and sign are flagged. Tables a and b - Bitstream syntax examples for decoding an array element

[0145] Depending on the symmetry properties specified for speaker pairs, the rendering reconstruction unit 81 can derive the array elements associated with the left speaker from the right speaker. In this case, the audio rendering information 2 in bitstream 21 for decoding a matrix element for the left speaker is reduced or potentially completely omitted accordingly.

[0146] In this way, the audio decoding device 24 can determine the symmetry information to reduce a size of the audio rendering information to be specified. In some cases, the audio decoding device 24 may determine symmetry information to reduce a size of the audio rendering information to be specified and derive at least a portion of the audio processor based on the symmetry information.

[0147] In these and other cases, the audio decoding device 24 can determine the value symmetry information to reduce a size of the audio rendering information to be specified. In these and other cases, the audio decoding device 24 may derive at least a portion of the audio processor based on the value symmetry information.

[0148] In these and other cases, the audio decoding device 24 can determine the signal symmetry information to reduce a size of the audio rendering information to be specified. In these and other cases, the audio decoding device 24 may derive at least a portion of the audio processor based on the signal symmetry information.

[0149] In these and other cases, the audio decoding device 24 may determine sparsity information indicative of a sparsity of a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0150] In these and other cases, the audio decoding device 24 can determine a speaker layout for which a matrix is to be used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0151] Audio decoding device 24, in this regard, can then determine the audio rendering information 2 specified in the bitstream. Based on the signal value included in the audio rendering information 2, the audio playback system 16 may render a plurality of speaker feeds 25 using one of the audio renderers 22. The speaker feeds may trigger the speakers 3. As noted above, the signal value may, in some cases, include a matrix (which is decoded and provided as one of the 22 audio renderers) used to render spherical harmonic coefficients for a plurality of loudspeakers feeds. speaker. In this case, the audio playback system 16 can configure one of the audio renderers 22 with the matrix, this using one of the audio renderers 22 to render the speaker feeds 25 based on the matrix.

[0152] To extract and then decode the various encoded versions of the HOA coefficients 11 so that the HOA coefficients 11 are available to be processed using the obtained audio processor 22, the extraction unit 72 can determine from the syntax element noted above indicative of whether the HOA 11 coefficients were encoded using the various direction-based or vector-based versions. When a direction-based encoding has been performed, the extraction unit 72 can extract the directional-based version of the HOA coefficients 11 and the syntax elements associated with the coded version (which is denoted as directional-based information 91 in the example of FIG. 4 ), passing the directionally based information 91 to the directionally based reconstruction unit 90. The directionally based reconstruction unit 90 may represent a unit configured to reconstruct HOA coefficients in the form of HOA coefficients 11' based on information based on direction 91.

[0153] When the syntax element indicates that the HOA coefficients 11 have been encoded using a vector-based decomposition, the extraction unit 72 may extract the foreground encoded V[k] vectors 57 (which may include encoded weights 57 and/or or indices 63 or quantized scalar V vectors), the encoded ambient HOA coefficients 59 and the corresponding audio objects 61 (which may also be referred to as the encoded nFG signals 61). The audio objects 61 each correspond to one of the vectors 57. The extraction unit 72 may pass the coded foreground vectors V[k] to the vector reconstruction unit V 74 and the coded ambient HOA coefficients 59 along with the encoded nFG signals 61 to the psychoacoustic decoding unit 80.

[0154] The V vector reconstruction unit 74 may represent a unit configured to reconstruct the V vectors from the coded foreground vectors V[k] 57. The V vector reconstruction unit 74 may function in a reciprocal manner to of the quantization unit 52.

[0155] The psychoacoustic decoding unit 80 may work in a reciprocal manner to the psychoacoustic audio encoder unit 40 shown in the example of FIG. 3 in order to decode the ambient encoded HOA coefficients 59 and the encoded nFG signals 61 and thus generate energy compensated ambient HOA coefficients 47' and the interpolated nFG signals 49' (which may also be referred to as interpolated nFG audio objects 49). '). Psychoacoustic decoding unit 80 may pass energy compensated ambient HOA coefficients 47' to fading unit 770 and nFG signals 49' to foreground formulation unit 78.

[0156] The spatiotemporal interpolation unit 76 may function in a similar manner as described above with respect to the spatiotemporal interpolation unit 50. The spatiotemporal interpolation unit 76 may receive the vectors [V][k] of 55k reduced foreground vectors and perform spatiotemporal interpolation with respect to 55k reduced foreground vectors V[k] and 55k-1 reduced foreground vectors V[k-1] to generate interpolated foreground vectors V[k] 55k''. The spatiotemporal interpolation unit 76 can forward the interpolated foreground vectors V[k] 55k'' to the fading unit 770.

[0157] The extraction unit 72 may also output a signal 757 indicative of when one of the ambient HOA coefficients is transitioning to the fading unit 770, which may then determine which of the SHCBG 47' (where SHCBG 47' may also be designated as “ambient HOA coefficients 47'” or “ambient HOA coefficients 47'”) and elements of interpolated foreground vectors [V][k] 55k” must appear or fade. In some examples, the fading unit 770 may operate opposite each of the ambient HOA coefficients 47' and the elements of the interpolated foreground vectors V[k] 55k". That is, the fading unit 770 can perform either a fade in or out, or both a fade in or out with respect to a corresponding one of the ambient HOA coefficients 47', while performing a fade in or out or both a fade in and a fade out, with respect to a corresponding of the elements of the interpolated foreground vectors V[k] 55k”. The fading unit 770 can produce the adjusted ambient HOA coefficients 47'' for the HOA coefficient formulation unit 82 and adjusted foreground vectors V[k] 55k'' for the foreground formulation unit 78. , the fading unit 770 represents a unit configured to perform a fading operation with respect to various aspects of the HOA coefficients or their derivatives, e.g. in the form of ambient HOA coefficients 47' and elements of interpolated foreground vectors V[k] 55k''.

[0158] The foreground formulation unit 78 may represent a unit configured to perform matrix multiplication with respect to the adjusted foreground vectors V[k] 55k''' and the interpolated nFG signals 49' to generate the coefficients Foreground HOA 65. In this regard, the foreground formulation unit 78 can combine the audio objects 49' (which is another way of designating the interpolated nFG signals 49') with the vectors 55k"' to reconstruct the first plane or, in other words, predominant aspects of the HOA 11' coefficients. The foreground formulation unit 78 can perform a matrix multiplication of the interpolated nFG signals 49' by the fitted foreground vectors V[k] 55k"'.

[0159] The HOA coefficient formulation unit 82 may represent a unit configured to combine the foreground HOA coefficients 65 with the adjusted HOA coefficients 47" to obtain the HOA coefficients 11'. The primary notation reflects that HOA 11' coefficients may be similar to, but not equal to, HOA 11 coefficients. Differences between HOA 11 and 11' coefficients may result from loss due to transmission over lossy transmission media, quantization, or other lossmaking operations.

[0160] In addition, the extraction unit 72 and the audio decoding device 24 more generally can also be configured to operate in accordance with various aspects of the techniques described in this disclosure to obtain the bitstreams 21 that are potentially optimized in the ways described above regarding not including multiple syntax elements or data fields in certain instances.

[0161] In some cases, the audio decoding device 24 may be configured to, upon decompressing higher order ambisonic audio data compressed using a first compression scheme, obtain a bitstream 21 representative of a compressed version of the ambisonic audio data order that does not include corresponding bits for a second compression scheme also used to compress the higher order ambisonic audio data. The first compression scheme may comprise a vector-based compression scheme, the resulting vector defined in the spherical harmonic domain and sent through bitstream 21. The vector-based decomposition compression scheme may, in some instances, comprise a compression scheme which involves applying a singular value decomposition (or equivalents thereof as described in greater detail with respect to the example in FIG. 3) to the higher order ambisonic audio data.

[0162] Audio decoding device 24 can be configured to obtain bitstream 21 that does not include the bits that correspond to at least one syntax element used to perform the second type of compression scheme. As noted above, the second compression scheme comprises a directionality-based compression scheme. More specifically, the audio decoding device 24 can be configured to obtain bitstream 21 that does not include the bits corresponding to a HOAPredictionInfo syntax element of the second compression scheme. In other words, when the second compression scheme comprises a directionality-based compression scheme, the audio decoding device 24 can be configured to obtain bitstream 21 that does not include the bits corresponding to a HOAPredictionInfo syntax element of the compression scheme. based on directionality. As mentioned above, the HOAPredictionInfo syntax element can be indicative of a prediction between two or more signals based on directionality.

[0163] In some cases, either as an alternative or in conjunction with the previous examples, the audio decoding device 24 may be configured to, when gain correction is suppressed during compression of higher-order ambisonic audio data, obtaining the representative bitstream 21 of a compressed version of the higher order ambisonic audio data that does not include gain correction data. The audio decoding device 24 may, in these instances, be configured to decompress the higher order ambisonic audio data in accordance with a vector-based synthesis decompression scheme. The compressed version of the higher order ambisonic data is generated by applying a singular value decomposition (or its equivalents described in greater detail with respect to the example of FIG. 3 above) to the higher order ambisonic audio data. When SVD is applied or equivalents to the HOA audio data, the audio encoding device 20 specifies at least one of the indicative resultant vectors or bits in the bitstream 21, where the vectors describe spatial characteristics of corresponding foreground audio objects (such as as a width, location, and volume of the corresponding foreground audio objects).

[0164] More specifically, audio decoding device 24 can be configured to obtain a MaxGainCorrAmbExp syntax element from bitstream 21 with a value set to zero to indicate that gain correction is suppressed. That is, the audio encoding device 24 can be configured to obtain, when gain correction is suppressed, the bitstream so that the bitstream does not include a HOAGainCorrection data field that stores the gain correction data. Bitstream 21 may comprise a MaxGainCorrAmbExp syntax element having a value of zero to indicate that gain correction is suppressed and not include a HOAGainCorrection data field that stores the gain correction data. Gain correction suppression can occur when compression of the higher-order ambisonic audio data includes the application of a unified speech and audio and speech coding (USAC) to the higher-order ambisonic audio data.

[0165] FIG. 5 is a flowchart illustrating exemplary operation of an audio encoding device, such as the audio encoding device 20 shown in the example of Fig. 3 in performing various aspects of the vector-based synthesis techniques described in this disclosure. Initially, the audio encoding device 20 receives HOA coefficients 11 (106). The audio coding device 20 may invoke the LIT unit 30, which may apply a LIT against the HOA coefficients to produce the transformed HOA coefficients (e.g., in the case of SVD, the transformed HOA coefficients may comprise the vectors US[k ] 33 and the vectors V[k] 35) (107).

[0166] Audio encoding device 20 may then invoke parameter calculation unit 32 to perform the above-described analysis with respect to any combination of US[k]33 vectors, US[k-1]33 vectors, the vectors V[k] and/or V[k-1] 35 to identify various parameters in the manner described above. That is, the parameter calculation unit 32 can determine at least one parameter based on an analysis of the transformed HOA coefficients 33/35 (108).

[0167] The audio encoding device 20 can then invoke the reorder unit 34, which can reorder the transformed HOA coefficients (which, again in the context of SVD, can refer to the vectors US[k] 33 and V[ k] 35 based on the parameters to generate the 33'/35' reordered transformed HOA coefficients (or, in other words, the US[k] 33' vectors and the V[k] 35' vectors), as described above (109 The audio encoding device 20 may, during any of the preceding or subsequent operations, also invoke the sound field analysis unit 44. The sound field analysis unit 44 may, as described above, perform an analysis of sound field with respect to HOA coefficients 11 and/or transformed HOA coefficients 33/35 to determine the total number of foreground channels (nFG) 45, the background sound field (NBG) order, and the number ( nBGa) and the indices (i) of additional HOA BG channels to send (which may collectively be denoted os as background channel information 43 in the example of FIG. 3) (109).

[0168] Audio encoding device 20 may also invoke background selection unit 48. Background selection unit 48 may determine background or ambient HOA coefficients 47 based on background channel information 43 (110). The audio coding unit 20 can further invoke the foreground selection unit 36 which can select the reordered US[k] vectors 33' and the reordered V[k] vectors 35' representing the foreground or distinct components of the sound field based on nFG 45 (which may represent one or more indices that identify the foreground vectors) (112).

[0169] Audio encoding device 20 may invoke power compensation unit 38. Power compensation unit 38 may perform power compensation against ambient HOA coefficients 47 to compensate for power loss due to removal of several of the HOA coefficients by the background selection unit 48 (114) and thus generate the energy compensated ambient HOA coefficients 47'.

[0170] Audio encoding device 20 may also invoke spatiotemporal interpolation unit 50. Spatiotemporal interpolation unit 50 may perform spatiotemporal interpolation with respect to transformed HOA coefficients reordered 33'/35' to obtain the interpolated foreground signals 49' (which may also be referred to as the "interpolated nFG signals 49' ) and the remaining foreground directional information 53 (which may also be referred to as the "V[k] vectors 53") ( 116). The audio encoding device 20 can then invoke the coefficient reduction unit 46. The coefficient reduction unit 46 can perform a coefficient reduction with respect to the remaining foreground vectors V[k] based on the channel information of background 43 to obtain reduced foreground direction information 55 (which may also be referred to as the reduced foreground vectors V[k] 55) (118).

[0171] The audio coding device 20 can then invoke the quantization unit 52 to compress, in the manner described above, the reduced foreground vectors V[k] 55 and generate coded foreground vectors V[k] 57 ( 120).

[0172] Audio encoding device 20 may also invoke psychoacoustic audio encoder unit 40. Psychoacoustic audio encoder unit 40 may psychoacoustically encode each vector of energy compensated ambient HOA coefficients 47' and nFG signals interpolated 49' to generate coded ambient HOA coefficients 59 and coded nFG signals 61. The audio encoding device may then invoke the bitstream generation unit 42. The bitstream generation unit 42 may then generate the bitstream 21 based on the information encoded foreground directional 57, encoded ambient HOA coefficients 59, encoded nFG signals 61, and background channel information 43.

[0173] FIG. 6 is a flowchart illustrating exemplary operation of an audio decoding device, such as the audio decoding device 24 shown in Fig. 4 in carrying out various aspects of the techniques described in this disclosure. Initially, audio decoding device 24 may receive bitstream 21 (130). Upon receiving the bitstream, the audio decoding device 24 may invoke the extraction unit 72. Assuming, for purposes of discussion, that the bitstream 21 indicates that vector-based reconstruction is to be performed, the extraction unit 72 can parse the bitstream to retrieve the aforementioned information, passing the information to the vector-based reconstruction unit 92.

[0174] In other words, the extraction unit 72 can extract the coded foreground directional information 57 (which, again, may also be referred to as the coded foreground vectors V[k] 57), the ambient HOA coefficients 59 and encoded foreground signals which may also be referred to as encoded foreground nFG signals 59 or encoded foreground audio objects 59) from bitstream 21 in the manner described above (132).

[0175] Audio decoding device 24 may further invoke dequantization unit 74. Dequantization unit 74 may entropy decode and dequantize encoded foreground directional information 57 to obtain 55k reduced foreground directional information (136) . Audio decoding device 24 may also invoke psychoacoustic decoding unit 80. Psychoacoustic audio decoding unit 80 may decode ambient coded HOA coefficients 59 and encoded foreground signals 61 to obtain energy compensated ambient HOA coefficients 47 ' and interpolated foreground signals 49' (138). Psychoacoustic decoding unit 80 may pass energy compensated ambient HOA coefficients 47' to fading unit 770 and nFG signals 49' to foreground formulation unit 78.

[0176] The audio decoding device 24 may then invoke the spatiotemporal interpolation unit 76. The spatiotemporal interpolation unit 76 may receive the foreground reordered directional information 55k' and perform spatiotemporal interpolation with 55k/55k-1 foreground directional information reduced to generate 55k” (140) interpolated foreground directional information. The spatiotemporal interpolation unit 76 can forward the interpolated foreground vectors V[k] 55k'' to the fading unit 770.

[0177] Audio decoding device 24 may invoke fading unit 770. Fading unit 770 may receive or otherwise obtain syntax elements (e.g. from extraction unit 72) indicative of when the coefficients HOA environment compensated at 47' are in transition (for example, the AmbCoeffTransition syntax element). The fading unit 770 can, based on the transition syntax elements and the maintained transition state information, pop-up or fade the energy-compensated ambient HOA coefficients 47', providing 47" adjusted HOA coefficients for the formulation unit. of HOA coefficient 82. The fading unit 770 can also, based on the syntax elements and the maintained transition state information, make the one or more elements of the interpolated foreground vectors V[k] 55k appear or fade out.” corresponding outputs by outputting the adjusted foreground vectors V[k] 55k”' to the foreground formulation unit 78 (142).

[0178] Audio decoding device 24 may invoke foreground formulation unit 78. Foreground formulation unit 78 may perform matrix multiplication of nFG signals 49' by adjusted foreground directional information 55k” ' to get the foreground HOA coefficients 65 (144). The audio decoding device 24 can also invoke the HOA coefficient formulation unit 82. The HOA coefficient formulation unit 82 can add the foreground HOA coefficients 65 to the adjusted HOA coefficients 47" to obtain the HOA coefficients 11 ' (146).

[0179] FIG. 7 is a flowchart illustrating exemplary operation of a system, such as system 10 shown in the example of Fig. 2 in carrying out various aspects of the techniques described in this disclosure. As discussed above, content creator device 12 can use sound editing system 18 to create or edit captured or generated audio content (which is shown as HOA coefficients 11 in the example of FIG. 2). The content creator device 12 can then render the HOA coefficients 11 using the audio processor 1 to the generated multichannel speaker feeds, as discussed in more detail above (200). The content creator device 12 can then play back these speaker feeds using an audio playback system and determine if further adjustment or editing is needed to capture, for example, the desired artistic intent (202). When more adjustments are desired ("YES" 202), the content creator device 12 can remix the HOA coefficients (204), render the HOA coefficients (11) (200), and determine if further adjustments are needed (202). When no further adjustments are desired ("NO" 202), the audio encoding device 20 can encode the audio content to generate the bitstream 21 in the manner described above with respect to the example of FIG. 5) (206). Audio encoding device 20 can also generate and specify audio rendering information 2 in bitstream 21, as described in greater detail above (208).

[0180] Content consuming device 14 can then obtain audio rendering information 2 from bitstream 21 (210). Decoding device 24 can then decode bitstream 21 to obtain the audio content (which is shown as HOA coefficients 11' in the example of FIG. 2 in the manner described above with respect to the example of FIG. 6) (211) . The audio playback system 16 can then render the HOA coefficients 11' based on the audio rendering information 2 in the manner described above (212) and play the rendered audio content through the speakers 3 (214).

[0181] The techniques described in this disclosure may therefore allow, as a first example, a device that generates a bitstream representative of multi-channel audio content to specify audio rendering information. The device may, in this first example, include means for specifying audio rendering information that includes a signal value that identifies an audio processor used when generating multichannel audio content.

[0182] The device of the first example, the signal value including a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0183] In a second example, the device of the first example, where the signal value includes two or more bits that define an index that indicates that the bitstream includes an array used to render spherical harmonic coefficients for a plurality of high feeds - speaker.

[0184] The device of the second example, where the audio rendering information further includes two or more bits that define a number of rows of the matrix included in the bitstream and two or more bits that define a number of columns of the matrix included in the bitstream .

[0185] The device of the first example, where the signal value specifies a rendering algorithm used to render audio objects to a plurality of speaker feeds.

[0186] The device of the first example, where the signal value specifies a rendering algorithm used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0187] The device of the first example, the signal value including two or more bits that define an index associated with one of a plurality of matrices used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0188] The device of the first example, where the signal value includes two or more bits that define an index associated with one of a plurality of rendering algorithms used to render audio objects to a plurality of speaker feeds.

[0189] The device of the first example, where the signal value includes two or more bits that define an index associated with one of a plurality of rendering algorithms used to render spherical harmonic coefficients for a plurality of speaker feeds.

[0190] The device of the first example, the means for specifying the audio rendering information comprises means for specifying the audio rendering information on a per-frame basis of audio in the bitstream.

[0191] The device of the first example, the means for specifying the audio rendering information comprises means for specifying the audio rendering information at a single moment in the bitstream.

[0192] In a third example, a non-transient computer-readable storage medium that has instructions stored on it that, when executed, cause one or more processors to specify the audio rendering information in the bitstream, with the audio rendering identifies an audio processor used when generating multichannel audio content.

[0193] In a fourth example, a device for rendering multichannel audio content from a bitstream, the device comprising means for determining audio rendering information that includes a signal value that identifies an audio renderer used when generating the audio. multichannel audio content and means for rendering a plurality of speaker feeds based on the audio rendering information specified in the bitstream.

[0194] The device of the fourth example, wherein the signal value includes a matrix used to render spherical harmonic coefficients for a plurality of speaker feeds, and wherein the means for rendering the plurality of speaker feeds comprises means for rendering the plurality of speaker feeds based on the matrix.

[0195] In a fifth example, the device of the fourth example, where the signal value includes two or more bits that define an index that indicates that the bitstream includes an array used to render spherical harmonic coefficients for a plurality of high feeds - speaker, wherein the device further comprising means for parsing the matrix from the bitstream in response to the index and the means for rendering the plurality of speaker feeds comprising means for rendering the plurality of speaker feeds with based on the analyzed matrix.

[0196] The device of the fifth example, where the sign value further includes two or more bits that define a number of rows of the matrix included in the bitstream and two or more bits that define a number of columns of the matrix included in the bitstream, and wherein the means for parsing the array from the bitstream comprises means for parsing the array from the bitstream in response to the index and based on the two or more bits defining a number of rows and the two or more bits defining the number of columns .

[0197] The device of the fourth example, where the signal value specifies the rendering algorithm used to render audio objects for the plurality of speaker feeds, and where the means for rendering the plurality of speaker feeds speaker comprise means for rendering the plurality of speaker feeds from the audio objects using the specified rendering algorithm.

[0198] The fourth example device, where the signal value specifies the rendering algorithm used to render spherical harmonic coefficients for the plurality of speaker feeds, and the means for rendering the plurality of speaker feeds speaker comprise means for rendering the plurality of speaker feeds from the spherical harmonic coefficients using the specified rendering algorithm

[0199] The device of the fourth example, wherein the signal value includes two or more bits that define an index associated with one of the plurality of matrices used to render spherical harmonic coefficients for the plurality of speaker feeds, and wherein the means for rendering the plurality of speaker feeds comprises means for rendering the plurality of speaker feeds from the spherical harmonic coefficients using one of the plurality of matrices associated with the index.

[0200] The device of the fourth example, where the signal value includes two or more bits that define an index associated with one of the plurality of rendering algorithms used to render the audio objects to the plurality of speaker feeds, and wherein the means for rendering the plurality of speaker feeds comprises means for rendering the plurality of speaker feeds from the audio objects using one of the plurality of rendering algorithms associated with the index.

[0201] The device of the fourth example, wherein the signal value includes two or more bits that define an index associated with one of the plurality of rendering algorithms used to render spherical harmonic coefficients for a plurality of speaker feeds, and wherein the means for rendering the plurality of speaker feeds comprises means for rendering the plurality of speaker feeds from the spherical harmonic coefficients using one of the plurality of rendering algorithms associated with the index.

[0202] The device of the fourth example, the means for determining audio rendering information includes means for determining audio rendering information on a per-frame basis of audio from the bitstream.

[0203] The device of the fourth example, the means for determining the audio rendering information means includes determining the audio rendering information at a single time from the bitstream.

[0204] In a sixth example, a non-transient computer-readable storage medium having information stored on it that, when executed, causes one or more processors to determine audio rendering information that includes a signal value that identifies a renderer audio used when generating multichannel audio content; and render a plurality of speaker feeds based on the audio rendering information specified in the bitstream.

[0205] FIGs. 8A-8D are diagrams illustrating bitstreams 21A-21D formed according to the techniques described in this disclosure. In the example of FIG. 8A, bitstream 21A may represent an example of bitstream 21 shown in FIGS. 2-4 above. Bitstream 21A includes audio rendering information 2A that includes one or more bits that define a signal value 554. Signal value 554 can represent any combination of the types of information described below. Bitstream 21A also includes 558 audio content, which may represent an example of 7/9 audio content.

[0206] In the example of FIG. 8B, bitstream 21B may be similar to bitstream 21A where the signal value 554 of the audio rendering information 2B comprises an index 554A, one or more bits that define a row size 554B of the signed array, one or more bits that define a column size 554C of the signed matrix, and matrix coefficients 554D. Index 554A can be set using two to five bits, while each row size 554B and column size 554C can be set using two to sixteen bits.

[0207] Extract unit 72 can extract index 554A and determine if the index indicates that the array is included in bitstream 21B (where certain index values such as 0000 or 1111 may signal that the array is explicitly specified in bitstream 21B ). In the example of FIG. 8B, bitstream 21B includes an index 554A signaling that the array is explicitly specified in bitstream 21B. As a result, extraction unit 72 can extract row size 554B and column size 554C. Extractor unit 72 may be configured to calculate the number of bits to parse representing matrix coefficients as a function of row size 554B, column size 554C and a signaled bit size (not shown in FIG. 8A) or implicit of each matrix coefficient. Using the given number of bits, the extraction unit 72 can extract the matrix coefficients 554D, which the audio reproduction system 16 can use to configure one of the audio renderers 22 as described above. Although illustrated as signaling 2B audio rendering information only once in bitstream 21B, 2B audio rendering information may be signaled multiple times in bitstream 21B or at least partially or entirely on a separate out-of-band channel (as optional data). in some cases).

[0208] In the example of FIG. 8C, bitstream 21C may represent an example of bitstream 21 shown in FIGS. 2-4 above. Bitstream 21C includes audio rendering information 2C that includes a signal value 554, which in this example specifies an algorithm index 554E. Bitstream 21C also includes audio content 558. Algorithm index 554E can be set using two to five bits as noted above, where this algorithm index 554E can identify a rendering algorithm to be used when rendering audio content 558.

[0209] The extraction unit 72 can extract algorithm index 550E and determine if algorithm index 554E indicates that the array is included in bitstream 21C (where certain index values such as 0000 or 1111 may signal that the array is explicitly specified in bitstream 21C). In the example of FIG. 8C, bitstream 21C includes algorithm index 554E signaling that the array is not explicitly specified in bitstream 21C. As a result, the extraction unit 72 forwards the algorithm index 554E to the audio reproduction system 16, which selects a match (if available) from the rendering algorithms (which are designated as renderers 22 in the example of FIGs 2-4) . Although illustrated as signaling the audio rendering information 2C, once in bitstream 21C, in the example of FIG. 8C, the 2C audio rendering information may be signaled multiple times in the 21B bitstream or at least partially or fully on a separate out-of-band channel (as optional data in some cases).

[0210] In the example of FIG. 8D, bitstream 21D may represent an example of bitstream 21 shown in FIGS. 2-4 above. Bitstream 21D includes 2D audio rendering information that includes a signal value 554, which in this example specifies an array index 554F. Bitstream 21D also includes audio content 558. Array index 554F can be set using two to five bits as noted above, where this array index 554F can identify a rendering algorithm to be used when rendering audio content 558.

[0211] Extraction unit 72 can extract array index 550F and determine if array index 554F indicates that the array is included in bitstream 21D (where certain index values such as 0000 or 1111 may signal that the array is explicitly specified in bitstream 21C). In the example of FIG. 8D, bitstream 21D includes array index 554F signaling that the array is not explicitly specified in bitstream 21D. As a result, the extraction unit 72 forwards the matrix index 554F to an audio playback device, which selects a match (if available) from the renderers 22. Although illustrated as signaling the 2D audio rendering information, a single moment in the bitstream 21D, in the example of FIG. 8D, 2D audio rendering information can be signaled multiple times in the 21D bitstream or at least partially or fully on a separate out-of-band channel (as optional data in some cases).

[0212] FIGs. 8E-8G are diagrams illustrating portions of bitstream or sidechannel information that can specify compressed spatial components in more detail. FIG. 8E illustrates a first example of a frame 249A' of bitstream 21. In the example of FIG. 8E, frame 249A' includes ChannelSideInfoData (CSID) fields 154A-154C, HOAGainCorrectionData (HOAGCD) fields, and VVectorData fields 156A and 156B. The CSD field 154A includes the unitC 267, bb 266 and ba 265 along with the ChannelType 269, each of which is set to the corresponding values 01, 1, 0 and 01 shown in the example of FIG. 8E. The CSD field 154B includes the unitC 267, bb 266 and ba 265 along with the ChannelType 269, each of which is set to the corresponding values 01, 1, 0 and 01 shown in the example of FIG. 8E. The CSC field 154C includes the ChannelType field 269 with a value of 3. Each of the CSA fields 154A-154C corresponds to the respective one of transport channels 1, 2 and 3. In effect, each CSID field 154A-154C indicates whether the payload corresponding 156A and 156B are direction-based signals (when the corresponding ChannelType is equal to zero), vector-based signals (when the corresponding ChannelType is equal to one), an additional ambient HOA coefficient (when the corresponding ChannelType is equal to two) , or empty (when the ChannelType is equal to three).

[0213] In the example of FIG. 8E, frame 249A includes two vector-based signals (given ChannelType 269 equals 1 in CSID fields 154A and 154B) and an empty one (given ChannelType 269 equals 3 in CSC field 154C). Based on a portion of the above HOAconfig (not shown for ease of illustration), the audio decoding device 24 can determine that all 16 V-vector elements are encoded. Thus, VVectorData 156A and 156B each include all 16 vector elements, each uniformly quantized with 8 bits.

[0214] As shown further in the example of FIG. 8E, the 249A' structure does not include a HOAPredictionInfo field. The HOAPredictionInfo field can represent a field corresponding to a second directional-based compression scheme that can be removed according to the technique described in this disclosure when the vector-based compression scheme is used to compress HOA audio data.

[0215] FIG. 8F is a diagram illustrating a frame 249A" which is substantially similar to frame 249A except that the HOAGainCorrectionData has been removed from each transport channel stored in frame 249A". The HOAGainCorrectionData field may be removed from frame 249A” when gain correction is suppressed according to various aspects of the techniques described above.

[0216] FIG. 8G is a diagram illustrating a 249A”' frame which may look similar to 249A'' frame, except that the HOAPredictionInfo field is removed. Table 249A"' represents an example where both aspects of the techniques can be applied together to remove various fields that may not be necessary in certain circumstances.

[0217] The above techniques can be performed against any number of different contexts and audio ecosystems. A number of exemplary contexts are described below, although the techniques should be limited to example contexts. An example of an audio ecosystem might include audio content, movie studios, music studios, game audio studios, channel-based audio content, encoding engines, game audio stems, audio encoding/rendering engines game and delivery systems.

[0218] Movie studios, music studios and game audio studios can receive audio content. In some examples, the audio content may represent the output of an acquisition. Film studios can deliver channel-based audio content (for example, in 2.0, 5.1, and 7.1), such as using a digital audio workstation (DAW). Music studios can deliver channel-based audio content (eg in 2.0 and 5.1) as using a DAW. In either case, the encoding engines can receive and encode channel-based audio content based on one or more codecs (e.g., AAC, AC3, Dolby True HD, Dolby Digital Plus, and DTS Master Audio) for output by the systems. of distribution. Game audio studios may produce one or more stems of game audio, such as through the use of a DAW. The game's audio encoding/rendering engines can encode and/or render the audio stems for channel-based audio content for output by delivery systems. Another example context in which the techniques can be performed comprises an audio ecosystem that may include streaming audio recording objects, professional audio systems, consumer device capture, HOA audio format, on-device rendering, audio from consumer, TV and accessories and car audio systems.

[0219] Streaming recording audio objects, professional audio systems and capturing on consumer device can encode their output using HOA audio format. In this way, audio content can be encoded using the HOA audio format into a single representation that can be played back using on-device rendering, consumer audio, TV and accessories, and car audio systems. In other words, the single representation of the audio content can be played on a generic audio playback system (i.e. as opposed to a specific configuration like 5.1, 7.1, etc.), such as the 16 audio playback system.

[0220] Other examples of context in which the techniques can be performed include an audio ecosystem that can include both acquisition elements and playback elements. Acquisition elements can include wired and/or wireless acquisition devices (e.g. Eigen microphones), ambient sound capture on the device, and mobile devices (e.g. smartphones and tablets). In some examples, wired and/or wireless acquisition devices may be coupled to the mobile device via wired and/or wireless communication channels.

[0221] In accordance with one or more techniques of this disclosure, the mobile device may be used to acquire a sound field. For example, the mobile device may acquire a sound field through wired and/or wireless acquisition devices and/or capture ambient sound on the device (e.g., a plurality of microphones integrated into the mobile device). The mobile device can then encode the acquired sound field into HOA coefficients for playback by one or more of the playback elements. For example, a mobile device user can record (acquire a sound field of) a live event (e.g. a meeting, a conference, a play, a concert, etc.) and encode the recording in HOA coefficients. .

[0222] The mobile device may also use one or more of the playback elements to reproduce the HOA encoded sound field. For example, the mobile device may decode the HOA encoded sound field and output a signal to one or more of the playback elements that causes one or more of the playback elements to recreate the sound field. As an example, the mobile device may use wireless and/or wired communication channels to send the signal to one or more speakers (eg speaker arrays, soundbars, etc.). As another example, the mobile device may use docking solutions to output the signal to one or more docking stations and/or one or more built-in speakers (e.g. car stereo systems and/or smart homes). As another example, the mobile device can use headphone processing to output the signal to a set of headphones, for example to create realistic binaural sound.

[0223] In some examples, a certain mobile device can acquire a 3D sound field and play the same 3D sound field later. In some examples, the mobile device may acquire a 3D sound field, encode the 3D sound field in HOA, and transmit the encoded 3D sound field to one or more other devices (e.g., other mobile devices and/or other devices not mobile) for playback.

[0224] Yet another context in which the techniques can be performed includes an audio ecosystem that may include audio content, game studios, encoded audio content, rendering engines, and delivery systems. In some examples, game studios may include one or more DAWs that can support editing HOA signals. For example, one or more DAWs may include HOA plugins and/or tools that can be configured to work with (eg work with) one or more game audio systems. In some examples, game studios may produce new stem formats that support HOA. In any case, game studios can produce encoded audio content for rendering engines that can render a sound field for playback by distribution systems.

[0225] The techniques can also be performed with respect to exemplary audio acquisition devices. For example, the techniques may be performed in connection with an Eigen microphone which may include a plurality of microphones that are collectively configured to record a 3D sound field. In some examples, the plurality of Eigen microphone microphones may be located on the surface of a substantially spherical ball having a radius of approximately 4 cm. In some examples, the audio encoding device 20 may be integrated into the Eigen microphone so as to produce a bitstream 21 directly from the microphone.

[0226] Another exemplary audio acquisition context may include a production truck that may be configured to receive a signal from one or more microphones, such as one or more Eigen microphones. The production truck may also include an audio encoder, such as the audio encoder 20 of FIG. 3.

[0227] The mobile device may also, in some cases, include a plurality of microphones that are collectively configured to record a 3D sound field. In other words, the plurality of microphones may have X, Y, Z diversity. In some examples, the mobile device may include a microphone which can be rotated to provide X, Y, Z diversity with respect to one or more microphones of the mobile device. . The mobile device may also include an audio encoder, such as the audio encoder 20 of FIG. 3.

[0228] An enhanced video capture device can even be configured to record a 3D sound field. In some examples, the enhanced video capture device can be attached to a helmet of a user engaged in an activity. For example, the reinforced video capture device can be attached to a helmet of a user rafting in clear water. In this way, the enhanced video capture device can capture a 3D sound field representing the action around the user (e.g. water falling behind the user, another rafter talking in front of the user, etc...) .

[0229] The techniques can also be performed with respect to an enhanced mobile accessory device, which can be configured to record a 3D sound field. In some examples, the mobile device may be similar to the mobile devices discussed above, with the addition of one or more accessories. For example, an Eigen microphone can be connected to the aforementioned mobile device to form an enhanced mobile accessory device. In this way, the Enhanced Mobile Accessory Device can capture a higher quality version of the 3D sound field than just using sound capture components built into the Enhanced Mobile Accessory Device.

[0230] Example of audio playback devices that can perform various aspects of the techniques described in this disclosure are discussed further below. According to one or more techniques of this description, the speakers and/or soundbars can be arranged in any arbitrary configuration while still reproducing a 3D sound field. Furthermore, in some examples, the headphone playback devices may be coupled to a decoder 24 via a wired or wireless connection. According to one or more of the techniques in this description, a single generic representation of a sound field can be used to render the sound field in any combination of speakers, soundbars, and headphone playback devices.

[0231] A number of different exemplary audio playback environments may also be suitable for performing various aspects of the techniques described in this disclosure. For example, a 5.1 speaker playback environment, a 2.0 speaker playback environment (for example, stereo), a 9.1 speaker playback environment with full-height front speakers, a 22.2 speaker playback, a mobile device with ear button playback environment may be suitable environment to perform various aspects of the techniques described in this description.

[0232] According to one or more techniques of this description, a single generic representation of a sound field can be used to render the sound field in any of the aforementioned playback environments. Additionally, the techniques of this disclosure allow a renderer to render a sound field from a generic representation for playback in playback environments other than those described above. For example, if design considerations prohibit placing speakers correctly in a 7.1 speaker playback environment (for example, if you cannot place a right ambient speaker), the techniques in this disclosure allow that one renderer offsets the other 6 speakers, so playback can be achieved in a 6.1 speaker playback environment.

[0233] Additionally, a user can watch a sports game while wearing headphones. According to one or more techniques of this disclosure, the 3D sound field of the sports game can be acquired (e.g., one or more Eigen microphones can be placed in and/or around the baseball stadium), the HOA coefficients corresponding to the 3D sound field can be obtained and transmitted to a decoder, the decoder can reconstruct the 3D sound field based on HOA coefficients and output the reconstructed 3D sound field to a renderer, the renderer can get an indication of the type of environment of playback (e.g. headphones) and render the 3D sound field reconstructed into signals that cause the headphones to produce a representation of the 3D sound field of the sports game.

[0234] In each of the various instances described above, it is to be understood that the audio encoding device 20 may perform a method or otherwise comprise means for performing each step of the method for which the audio encoding device 20 is configured to run. In some cases, the means may comprise one or more processors. In some cases, the one or more processors may represent a special-purpose processor configured through instructions stored on non-transient computer-readable storage media. In other words, various aspects of the techniques in each of the sets of encoding examples can provide non-transient computer-readable storage media having instructions stored on it that, when executed, cause one or more processors to execute the method for which audio encoding device 20 has been configured to run.

[0235] In one or more examples, the functions described may be implemented in hardware, software, firmware or any combination thereof. If implemented in software, functions may be stored in or transmitted through one or more instructions or code on computer-readable media and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media that correspond to tangible media such as data storage media. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in the present disclosure. A computer program product may include computer readable media.

[0236] Similarly, each of the various instances described above, it is to be understood that the audio decoding device 24 may perform a method or otherwise comprise means for performing each step of the method for which the audio decoding device 24 audio 24 is set to run. In some cases, the means may comprise one or more processors. In some cases, the one or more processors may represent a special-purpose processor configured through instructions stored on non-transient computer-readable storage media. In other words, various aspects of the techniques in each of the sets of encoding examples can provide non-transient computer-readable storage media having instructions stored on it that, when executed, cause one or more processors to execute the method for which audio decoding device 24 has been configured to run.

[0237] By way of example, and not by way of limitation, such computer readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, memory flash or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to tangible, non-transient storage media. . Disk and floppy disk, as used herein, include compact disk (CD), laser disk, optical disk, digital versatile disk (DVD), diskette, and Blu-ray disk where diskettes generally reproduce data magnetically, while discs reproduce data optically. with lasers. Combinations of the above must also be included within the scope of computer readable media.

[0238] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASIC), field-programmable logic arrays (FPGA), or other equivalent logic or discrete circuits. Accordingly, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined codec. Also, the techniques can be fully implemented in one or more circuits or logic elements.

[0239] The techniques of the present disclosure can be implemented in a wide variety of devices or apparatus, including a wireless handset, an integrated circuit (IC), or a set of ICs (eg, a chip set). Various components, modules or units are described in the present disclosure to emphasize the functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be performed by different hardware units. Instead, as described above, multiple units may be combined into one codec hardware unit or provided by a collection of interoperable hardware units including one or more processors as described above in conjunction with software and/or firmware adequate.

[0240] Various aspects of the techniques have been described. These and other aspects of the techniques are within the scope of the claims that follow.

Claims (15)

1. Device configured to render higher-order ambisonic coefficients, the device characterized by comprising: one or more processors configured to: obtain, from a bitstream that includes an encoded version of the higher-order ambisonic coefficients, sparsity information indicative of a sparsity of a matrix used to render higher-order ambisonic coefficients for a plurality of speaker feeds; obtaining, from the bitstream, value symmetry information that indicates symmetry of matrix values; get, from the bitstream, a reduced number of bits used to represent the array; and based on the sparsity information, the value symmetry information and the reduced number of bits, reconstruct the matrix; and a memory coupled to one or more processors and configured to store sparse information. Device according to claim 1, characterized in that the one or more processors are further configured to determine a speaker format for which the matrix is to be used to render the plurality of speaker feeds from the coefficients. higher-order ambisonics. Device according to claim 1, characterized in that it further comprises a loudspeaker configured to reproduce a sound field represented by higher order ambisonic coefficients based on the plurality of loudspeaker feeds. Device according to claim 1, characterized in that one or more processors are additionally configured to obtain, from the bitstream, audio rendering information indicative of a signal value that identifies an audio renderer used when generating the content. multi-channel audio and render the plurality of speaker feeds based on the audio rendering information. The device of claim 4, characterized in that the signal value includes an index associated with the matrix used to render higher-order ambisonic coefficients for the multi-channel audio data, and that one or more processors are configured. to render the plurality of speaker feeds based on the array associated with the index included in the signal value. 6. Method for rendering higher-order ambisonic coefficients, the method characterized by comprising: obtaining, from a bitstream that includes a coded version of the higher-order ambisonic coefficients, sparsity information indicative of a sparseness of a matrix used to render higher order ambisonic coefficients for generating a plurality of speaker feeds; and obtaining, from a bitstream, value symmetry information that indicates matrix value symmetry; and obtaining, from a bitstream, a reduced number of bits used to represent the array; based on the value symmetry information, the sparsity information, and the reduced number of bits, reconstruct the array. A method as claimed in claim 6, further comprising determining a speaker format for which the matrix is to be used to render the plurality of speaker feeds from the higher order ambisonic coefficients. A method as claimed in claim 6, further comprising reproducing a sound field represented by higher order ambisonic coefficients based on the plurality of speaker feeds. A method as claimed in claim 6, further comprising obtaining from the bitstream audio rendering information indicative of a signal value that identifies an audio renderer used when generating the plurality of speaker feeds. ; and rendering the plurality of speaker feeds based on the audio rendering information. A method as claimed in claim 9, characterized in that the signal value includes an index associated with the matrix used to render the higher order ambisonic coefficients for the plurality of speaker feeds, and the method further comprises rendering the plurality of loudspeakers. speaker feeds based on the array associated with the index included in the signal value. 11. Device configured to produce a bitstream, the device characterized in that it comprises: a memory configured to store an array; and one or more memory-coupled processors, and configured to: obtain sparsity information indicative of a sparseness of the matrix used to render higher-order ambisonic coefficients to generate a plurality of speaker feeds; get value symmetry information that indicates the value symmetry of the array; based on the value symmetry information and the sparsity information, determine a reduced number of bits used to represent the matrix; and generating the bitstream to include an encoded version of the higher-order ambisonic coefficients, the value symmetry information, the sparsity information, and the reduced number of bits. Device according to claim 11, characterized in that one or more processors are further configured to determine a speaker format for which the matrix is to be used to render the plurality of speaker feeds from the coefficients. higher-order ambisonics. Device according to claim 11, characterized in that it additionally comprises a microphone configured to capture a sound field represented by higher order ambisonic coefficients. 14. Method for producing a bitstream, the method comprising: obtaining sparsity information indicative of a sparsity of a matrix used to render higher-order ambisonic coefficients to generate a plurality of speaker feeds; get value symmetry information that indicates the value symmetry of the array; and based on the value symmetry information and the sparsity information, reduce a number of bits used to represent the matrix; and generating the bitstream to include an encoded version of the higher-order ambisonic coefficients, the value symmetry information, the sparsity information, and the reduced number of bits. The method of claim 14, further comprising determining a speaker format for which the matrix is to be used to render the plurality of speaker feeds from the higher order ambisonic coefficients.

Images