JP6423009B2 - Obtaining symmetry information for higher-order ambisonic audio renderers - Google Patents

Description

Related applications

  [0001] This application is filed on July 11, 2014, US Provisional Application No. 62 / 023,662, entitled “SIGNALING AUDIO RENDERING INFORMATION IN A BITSTREAM”, and filed May 30, 2014. Claiming the benefit of US Provisional Application No. 62 / 005,829 entitled “SIGNALING AUDIO RENDERING INFORMATION IN A BITSTREAM”, each of the US provisional applications described above is hereby incorporated by reference in its entirety. Incorporated by reference as if described in.

  [0002] This disclosure relates to rendering information, and more particularly, to rendering information for higher order ambisonic (HOA) audio data.

  [0003] During the creation of audio content, a sound engineer renders the audio content using a specific renderer to adjust the audio content for the target configuration of the speakers used to play the audio content Can do. In other words, the acoustic engineer may render audio content and play the rendered audio content using speakers placed in the targeted configuration. The sound engineer then remixes the various aspects of the audio content, renders the remixed audio content, and renders the remixed audio content rendered using the speakers placed in the targeted configuration, again. obtain. The acoustic engineer can repeat this way until some artistic intent is provided by the audio content. In this way, the acoustic engineer provides audio content that provides some artistic intent (eg, to match video content that is screened with the audio content) or otherwise provides a sound field that is being played. Can be created.

  [0004] In general, techniques for specifying audio rendering information in a bitstream representing audio data are described. In other words, the technique provides a method for signaling audio rendering information used during audio content creation to a playback device, which then uses the audio rendering information to render the audio content. obtain. Providing rendering information in this way ensures that the playback device renders the audio content as intended by the acoustic engineer so that the artistic intent is potentially understood by the listener. Allows you to potentially ensure proper regeneration. In other words, the rendering information used during rendering by the acoustic engineer is provided according to the techniques described in this disclosure so that the audio playback device can render the audio content to render the audio content as intended by the acoustic engineer. Compared to systems that do not provide this audio rendering information, thereby ensuring a more consistent experience both during creation and playback of audio content.

  [0005] In one aspect, a device configured to render higher order ambisonic coefficients is a sparseness indicating a sparseness of a matrix used to render the higher order ambisonic coefficients in a plurality of speaker feeds. One or more processors configured to obtain information and a memory configured to store sparseness information.

  [0006] In another aspect, a method for rendering high-order ambisonic coefficients includes a sparseness indicating a sparseness of a matrix used to render the high-order ambisonic coefficients to generate a plurality of speaker feeds. Obtaining information.

  [0007] In another aspect, a device configured to create a bitstream includes a memory configured to store a matrix and a higher order ambisonic to generate a plurality of speaker feeds. One or more processors configured to obtain sparseness information indicative of the sparseness of the matrix used to render the coefficients.

  [0008] In another aspect, a method for creating a bitstream obtains sparseness information indicating a sparseness of a matrix used to render higher order ambisonic coefficients to generate multiple speaker feeds. Prepare to do.

  [0009] In another aspect, a device configured to render higher order ambisonic coefficients is a matrix code used to render higher order ambisonic coefficients to generate a plurality of speaker feeds. One or more processors configured to obtain sign symmetry information indicative of sign symmetry and a memory configured to store sparseness information.

  [0010] In another aspect, a method for rendering higher order ambisonic coefficients includes code symmetry that indicates code symmetry of a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds. Obtaining information.

  [0011] In another aspect, a device configured to create a bitstream stores a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds. A memory configured and one or more processors configured to encode symmetry information indicative of matrix code symmetry.

  [0012] In another aspect, a method for creating a bitstream obtains sparseness information indicating the sparseness of a matrix used to render higher-order ambisonic coefficients so as to generate multiple speaker feeds. Prepare to do.

  [0013] The details of one or more aspects of the techniques are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the technique will be apparent from the description and drawings, and from the claims.

The figure which shows the spherical harmonic basis function of various orders and suborders. FIG. 11 illustrates a system that can perform various aspects of the techniques described in this disclosure. FIG. 3 is a block diagram illustrating in more detail an example of an audio encoding device shown in the example of FIG. 2 that may perform various aspects of the techniques described in this disclosure. FIG. 3 is a block diagram illustrating the audio decoding device of FIG. 2 in more detail. 6 is a flowchart illustrating an example operation of an audio encoding device in performing various aspects of the vector-based synthesis techniques described in this disclosure. 6 is a flowchart illustrating an example operation of an audio decoding device in performing various aspects of the techniques described in this disclosure. 3 is a flowchart illustrating an example operation of a system, such as one of the systems shown in the example of FIG. 2, in performing various aspects of the techniques described in this disclosure. FIG. 3 illustrates a bitstream formed in accordance with the techniques described in this disclosure. FIG. 3 illustrates a bitstream formed in accordance with the techniques described in this disclosure. FIG. 3 illustrates a bitstream formed in accordance with the techniques described in this disclosure. FIG. 3 illustrates a bitstream formed in accordance with the techniques described in this disclosure. FIG. 3 illustrates in more detail a portion of a bitstream or side channel information that may specify a compressed spatial component. FIG. 3 illustrates in more detail a portion of a bitstream or side channel information that may specify a compressed spatial component. FIG. 3 illustrates in more detail a portion of a bitstream or side channel information that may specify a compressed spatial component. FIG. 4 is a diagram illustrating an example of HOA order dependent minimum and maximum gains in a higher order ambisonic (HOA) rendering matrix. FIG. 6 illustrates a partially sparse 6th order HOA rendering matrix for 22 loudspeakers. The flowchart explaining the signaling of a symmetry property.

  [0026] The development of surround sound now makes many output formats available for entertainment. Examples of such consumer surround sound formats are mostly “channel” based in that they implicitly specify a feed to a loudspeaker at certain geometric coordinates. The consumer surround sound format is a popular 5.1 format (which includes the following six channels: front left (FL), front right (FR), center or front center, and back Including left or surround left, backlight or surround right, and low frequency effect (LFE), developing 7.1 format, 7.1.4 format and 22.2 format (eg, ultra high definition) Includes various formats including height speakers (for use with television standards). A non-consumer format is often referred to as a “surround array” and can span any number of speakers (in symmetric and non-symmetrical geometry). An example of such an array includes 32 loudspeakers arranged at the corner coordinates of a truncated icosahedron.

[0027] The input to a future MPEG encoder is optionally intended to be played through a loudspeaker at three pre-specified locations: (i) (Ii) Discrete Pulse Code Modulation (PCM) data for a single audio object with associated metadata including position coordinates (among other information) Representing a sound field using accompanying object-based audio and (iii) spherical harmonic basis function coefficients (also called “spherical harmonic coefficients” or SHC, “higher ambisonics” or HOA, and “HOA coefficients”) Is one of scene-based audio with The future MPEG encoder is available at http://mpeg.chiariglione.org/sites/default/files/files/standards/parts/docs/w13411.zip , published in Geneva, Switzerland in January 2013 Possible International Organization for Standardization / International Electrotechnical Commission (ISO) / (IEC) JTC1 / SC29 / WG11 / N13411 entitled “Call for Proposals for 3D Audio”

  [0028] There are various “surround sound” channel-based formats on the market. These formats range, for example, from a 5.1 home theater system (most successful over stereo in terms of moving into the living room) to a 22.2 system developed by NHK (Nippon Hoso Kyokai or Japan Broadcasting Corporation). Content creators (eg, Hollywood studios) want to create a movie soundtrack at once, and do not want to make an effort to remix the soundtrack for each speaker configuration. Recently, standards development organizations have been able to adapt to and depend on the acoustic conditions at the location of the encoding (and number) of speakers and the playback location (with the renderer) into a standardized bitstream. Not thinking of a method for providing subsequent decoding and.

  [0029] To provide such flexibility to content creators, a hierarchical set of elements can be used to represent a sound field. A hierarchical set of elements may refer to a set of elements in which the elements are ordered such that a basic set of lower order elements provides a complete representation of the modeled sound field. As the set is expanded to include higher order elements, the representation becomes more detailed and resolution is improved.

  [0030] An example of a hierarchical set of elements is a set of spherical harmonic coefficients (SHC). The following equation shows a description or representation of a sound field that uses SHC.

[0031] This equation shows that the pressure p _i at any point {r _r , θ _r , φ _r } in the sound field at time t is SHC,

It can be expressed uniquely by here,

, C is the speed of sound (about 343 m / s), {r _r , θ _r , φ _r } is a reference point (or observation point), and j _n (•) is a spherical Bessel function of order n. ,

Is a spherical harmonic basis function of order n and sub-order m. The terms in square brackets are frequency domain representations of the signal that can be approximated by various time frequency transforms such as discrete Fourier transform (DFT), discrete cosine transform (DCT), or wavelet transform (ie, S (ω, r _r , It can be recognized that θ _r , φ _r )). Other examples of hierarchical sets include sets of wavelet transform coefficients and other sets of coefficients of multi-resolution basis functions.

  [0032] FIG. 1 is a diagram showing spherical harmonic basis functions from the 0th order (n = 0) to the 4th order (n = 4). As can be seen, for each order there is an extension of sub-order m, which is shown for simplicity of explanation but is not explicitly shown in the example of FIG.

  [0033] SHC

May be physically acquired (eg, recorded) by various microphone array configurations, or alternatively, they may be derived from a channel-based or object-based description of the sound field. SHC represents scene-based audio, where the SHC can be input to an audio encoder to obtain an encoded SHC that can facilitate more efficient transmission or storage. For example, a quaternary representation with (1 + 4) ² (25 and hence fourth order) coefficients may be used.

  [0034] As described above, the SHC may be derived from a microphone recording using a microphone array. Various examples of how SHC can be derived from a microphone array are described in Poletti, M, “Three-Dimensional Surround Sound Systems Based on Physical Harmonics”, J. Org. Audio Eng. Soc. Vol. 53, no. 11, November 2005, pages 1004-1025.

  [0035] To illustrate how the SHC can be derived from an object-based description, consider the following equation: Coefficients for the sound field corresponding to individual audio objects

Is

Where i is

And

Is a spherical Hankel function of order n (second type), and {r _s , θ _s , φ _s } is the location of the object. Knowing the object source energy g (ω) as a function of frequency (eg, using a time-frequency analysis technique, such as performing a fast Fourier transform on a PCM stream), the location corresponding to each PCM object SHC

It becomes possible to convert to. In addition, for each object

The coefficients can be shown to be additive (since the above equation is linear and orthogonal). In this way, many PCM objects

It can be represented by a coefficient (eg, as a sum of coefficient vectors for individual objects). In essence, these coefficients contain information about the sound field (pressure as a function of 3D coordinates), which is the total sound field in the vicinity of the observation points {r _r , θ _r , φ _r }. Represents a conversion from an individual object to a representation of The remaining numbers are described below in the context of object-based audio coding and SHC-based audio coding.

  [0036] FIG. 2 is an illustration of a system 10 that can perform various aspects of the techniques described in this disclosure. As shown in the example of FIG. 2, the system 10 includes a content creator device 12 and a content consumer device 14. Although described in the context of the content creator device 12 and the content consumer device 14, the technique can be any of SHC (which may also be referred to as a HOA coefficient) or sound field to form a bitstream representing audio data. It can be implemented in any context where other hierarchical representations are encoded. Moreover, the content creator device 12 can implement any of the techniques described in this disclosure, including a handset (or cellular phone), tablet computer, smartphone, or desktop computer, to name a few examples. In the form of a computing device. Similarly, content consumer device 14 implements the techniques described in this disclosure, including a handset (or cellular phone), tablet computer, smartphone, set-top box, or desktop computer, to name a few examples. May represent any form of computing device.

  [0037] Content creator device 12 may be operated by a movie studio or other entity that can generate multi-channel audio content for consumption by an operator of a content consumer device, such as content consumer device 14. In some examples, the content creator device 12 may be operated by an individual user who desires to compress the HOA factor 11. In many cases, content creators generate audio content along with video content. The content consumer device 14 can be operated by an individual. Content consumer device 14 may include an audio playback system 16 that may refer to any form of audio playback system capable of rendering an SHC for playback as multi-channel audio content.

  [0038] The content creator device 12 includes an audio editing system 18. The content creator device 12 obtains live recordings 7 (directly included as HOA coefficients) and audio objects 9 in various formats, and the content creator device 12 edits them using an audio editing system 18. Can do. The microphone 5 can capture a live recording 7. During the editing process, the content creator can render the HOA coefficient 11 from the audio object 9 and listen to the rendered speaker feed in an attempt to identify various aspects of the sound field that require further editing. . The content creator device 12 then edits the HOA coefficient 11 (potentially through manipulation of various objects of the audio object 9 from which the source HOA coefficient can be derived in the manner described above). be able to. The content creator device 12 can employ an audio editing system 18 to generate the HOA coefficient 11. Audio editing system 18 represents any system capable of editing audio data and outputting the audio data as one or more source spherical harmonic coefficients.

  [0039] Upon completion of the editing process, the content creator device 12 may generate the bitstream 21 based on the HOA factor 11. That is, the content creator device 12 has a device configured to encode or otherwise compress the HOA coefficient 11 in accordance with various aspects of the techniques described in this disclosure to generate the bitstream 21. An audio encoding device 20 is represented. Audio encoding device 20 may generate bitstream 21 for transmission over a transmission channel, which may be a wired or wireless channel, a data storage device, etc., by way of example. Bitstream 21 may represent an encoded version of HOA coefficient 11 and may include a main bitstream and another side bitstream that may also be referred to as side channel information.

  [0040] Although shown in FIG. 2 as being sent directly to the content consumer device 14, the content creator device 12 is located between the content creator device 12 and the content consumer device 14. The bitstream 21 can be output to the intermediate device that has been selected. The intermediate device can store the bitstream 21 for later delivery to the content consumer device 14 that may request the bitstream. The intermediate device may be a file server, web server, desktop computer, laptop computer, tablet computer, mobile phone, smartphone, or any other capable of storing the bitstream 21 for later retrieval by an audio decoder. The device may be provided. The intermediate device is in a content distribution network capable of streaming the bitstream 21 (possibly with a corresponding video data bitstream) to a subscriber, such as a content consumer device 14 that requests the bitstream 21. May be present.

  [0041] Alternatively, the content creator device 12 may store the bitstream 21 on a storage medium, such as a compact disk, digital video disk, high definition video disk, or other storage medium, Most are readable by a computer and are therefore sometimes referred to as computer-readable or non-transitory computer-readable storage media. In this context, a transmission channel may refer to a channel through which content stored on these media is transmitted (and may include retail stores and other store-based distribution mechanisms). Thus, in any event, the techniques of this disclosure should not be limited to the example of FIG. 2 in this regard.

  As further shown in the example of FIG. 2, the content consumer device 14 includes an audio playback system 16. Audio playback system 16 may represent any audio playback system capable of playing multi-channel audio data. Audio playback system 16 may include a number of different renderers 22. Each of the renderers 22 can provide a different form of rendering, wherein the different forms of rendering are one or more of various ways to perform vector-base amplitude panning (VBAP) and / or One or more of various ways of performing sound field synthesis may be included. As used herein, “A and / or B” means “A or B” or both “A and B”.

  [0043] The audio playback system 16 may further include an audio decoding device 24. Audio decoding device 24 may represent a device configured to decode HOA coefficient 11 ′ from bitstream 21, which may be similar to HOA coefficient 11 but with lossy operations (eg, quantum ) And / or transmission over the transmission channel. The audio playback system 16 can render the HOA coefficients 11 ′ to output the loudspeaker feed 25 after decoding the bitstream 21 to obtain the HOA coefficients 11 ′. The loudspeaker feed 25 can drive one or more loudspeakers (not shown in the example of FIG. 2 for ease of explanation).

  [0044] In order to select an appropriate renderer or, in some cases, to generate an appropriate renderer, the audio playback system 16 may determine the number of loudspeakers and / or the spatial geometry of the loudspeakers. The loudspeaker information 13 indicating the target arrangement can be acquired. In some cases, the audio playback system 16 can drive the loudspeaker in such a way as to obtain the loudspeaker information 13 using a reference microphone and dynamically determine the loudspeaker information 13. In other cases, or in conjunction with the dynamic determination of the loudspeaker information 13, the audio playback system 16 may prompt the user to interface with the audio playback system 16 and enter the loudspeaker information 13.

  [0045] The audio playback system 16 may then select one of the audio renderers 22 based on the loudspeaker information 13. In some cases, the audio playback system 16 may have any of the audio renderers 22 relative to the loudspeaker geometry specified in the loudspeaker information 13 (in terms of loudspeaker geometry). One of the audio renderers 22 may be generated based on the loudspeaker information 13 when not within some similarity measure threshold. The audio playback system 16 may in one case select one of the audio renderers 22 based on the loudspeaker information 13 without first trying to select an existing one of the audio renderers 22. Can be generated. The one or more speakers 3 may then play the rendered loudspeaker feed 25.

  [0046] In some cases, the audio playback system 16 can select any one of the audio renderers 22, and the bitstream 21 is received from the source (DVD player, Blu-ray to provide several examples). Depending on the (registered trademark) player, smartphone, tablet computer, game console, television receiver, etc.) can be configured to select one or more of the audio renderers 22. Any one of the audio renderers 22 can be selected, but the audio renderer often used when creating the content is that the content is this one of the audio renderers, ie the example of FIG. Now, due to the fact that it was created by the content creator 12 using the audio renderer 5, it provides a better (probably the best) form of rendering. Choosing one of the audio renderers 22 that is the same or at least close (in terms of the form of rendering) can provide a better representation of the sound field and good surround sound for the content consumer 14. Can bring experiences.

  [0047] In accordance with the techniques described in this disclosure, audio encoding device 20 may generate bitstream 21 to include audio rendering information 2 ("render info 2"). The audio rendering information 2 can include a signal value that identifies the audio renderer used when generating multi-channel audio content, ie, the audio renderer 1 in the example of FIG. In some cases, the signal value includes a matrix that is used to render spherical harmonic coefficients into multiple speaker feeds.

  [0048] In some cases, the signal value includes two or more bits that define an index indicating that the bitstream includes a matrix used to render spherical harmonic coefficients into multiple speaker feeds. Including. In some cases, when an index is used, the signal value further includes two or more bits that define the number of matrix rows included in the bitstream, and the number of matrix columns included in the bitstream. And two or more bits that define Using this information and assuming that each coefficient of a two-dimensional matrix is typically defined by a 32-bit floating point number, the size in terms of the bits of the matrix is the number of rows, the number of columns, and It can be calculated as a function of the size of a floating point number defining each coefficient of the matrix, i.e. in this example 32 bits.

  [0049] In some cases, the signal value specifies a rendering algorithm that is used to render spherical harmonics into multiple speaker feeds. The rendering algorithm can include a matrix that is known to both the audio encoding device 20 and the decoding device 24. That is, the rendering algorithm can include matrix application in addition to other rendering steps such as panning (eg, VBAP, DBAP, or simple panning) or NFC filtering. In some cases, the signal value includes two or more bits that define an index associated with one of the plurality of matrices used to render the spherical harmonic coefficients into the plurality of speaker feeds. Again, both audio encoding device 20 and decoding device 24 may use a plurality of matrices and a plurality of matrices so that the index can uniquely identify a particular one of the plurality of matrices. And information indicating the order of. Alternatively, audio encoding device 20 may specify a plurality of matrices and / or a plurality of matrix orders such that an index can uniquely identify a particular one of the plurality of matrices. Data in stream 31 may be specified.

  [0050] In some cases, the signal value is two or more bits that define an index associated with one of a plurality of rendering algorithms used to render the spherical harmonics into a plurality of speaker feeds. including. Again, both audio encoding device 20 and decoding device 24 may use a plurality of rendering algorithms and a plurality of rendering algorithms such that the index can uniquely identify a particular one of the plurality of matrices. It may be composed of information indicating the order of the rendering algorithm. Alternatively, the audio encoding device 20 may define a plurality of matrices and / or a plurality of matrix orders such that the index can uniquely identify a particular one of the plurality of matrices. You can specify the data inside.

  [0051] In some cases, audio encoding device 20 specifies audio rendering information 2 for each audio frame in the bitstream. In other cases, the audio encoding device 20 specifies the audio rendering information 2 once in the bitstream.

  [0052] The decoding device 24 may then determine the audio rendering information 2 specified in the bitstream. Based on the signal values included in the audio rendering information 2, the audio playback system 16 may render a plurality of speaker feeds 25 based on the audio rendering information 2. As described above, the signal values may in some cases include a matrix that is used to render spherical harmonic coefficients into multiple speaker feeds. In this case, the audio playback system 16 may configure one of the audio renderers 22 with that matrix, and this one of the audio renderers 22 to render the speaker feed 25 based on that matrix. Use one.

  [0053] In some cases, the signal value includes two or more bits that define an index indicating that the bitstream includes a matrix used to render the HOA 11 'to the speaker feed 25. The decoding device 24 can parse the matrix from the bitstream in response to the index, so that the audio playback system 16 constructs one of the audio renderers 22 with the parsed matrix and renders the speaker feed 25. This one of the renderers 22 may be called to do so. Decoding when the signal value includes two or more bits defining the number of rows of the matrix included in the bitstream and two or more bits defining the number of columns of the matrix included in the bitstream Device 24 may respond to the index and parse the matrix from the bitstream based on the two or more bits defining the number of rows and the two or more bits defining the number of columns as described above.

  [0054] In some cases, the signal value specifies the rendering algorithm used to render the HOA 11 'to the speaker feed 22. In these cases, some or all of the audio renderers 22 may perform these rendering algorithms. Audio playback device 16 may then utilize one of the specified rendering algorithms, eg, audio renderer 22, to render speaker feed 25 from HOA 11 '.

  [0055] When the signal value includes two or more bits defining an index associated with one of a plurality of matrices used to render the HOA 11 'to the speaker feed 25, of the audio renderer 22 Some or all may represent this multiple matrix. Thus, the audio playback system 16 may render the speaker feed 25 from the HOA 11 'using one of the audio renderers 22 associated with the index.

  [0056] of the audio renderer 34 when the signal value includes two or more bits defining an index associated with one of a plurality of rendering algorithms used to render the HOA 11 'to the speaker feed 25 Some or all of these may represent these rendering algorithms. Thus, the audio playback system 16 may render the speaker feed 25 from the spherical harmonics 11 'using one of the audio renderers 22 associated with the index.

  [0057] Depending on the frequency with which this audio rendering information is specified in the bitstream, decoding device 24 determines audio rendering information 2 on a per-audio-frame-basis basis or once. Can do.

  [0058] By specifying the audio rendering information 3 in this manner, the present technique potentially allows for better playback of multi-channel audio content according to how the content creator 12 intended to play multi-channel audio content. Can bring. As a result, the present technique may provide a more immersive surround sound or multi-channel audio experience.

  [0059] In other words, and as described above, higher order ambisonic (HOA) may represent a method for describing directivity information of a sound field based on spatial Fourier transform. Typically, the higher the ambisonics order N, the higher the spatial resolution and the larger the number of spherical harmonic (SH) coefficients (N + 1) ^ 2, which is required to transmit and store data. The bandwidth is larger.

[0060] A potential advantage of this description is the possibility of playing this sound field with most arbitrary loudspeaker settings (eg 5.1, 7.12, 22.2, etc.). The conversion from the sound field description to M loudspeaker signals can be done via a static rendering matrix with (N + 1) ² inputs and M outputs. As a result, every loudspeaker setting may require a dedicated rendering matrix. Several algorithms may exist to calculate the rendering matrix for the desired loudspeaker settings, which is optimized for some objective or subjective measure, such as the Gerzon criterion obtain. With irregular loudspeaker settings, the algorithm can be complicated due to iterative numerical optimization procedures, such as convex optimization. It may be beneficial to have sufficient computational resources available to calculate a rendering matrix for irregular loudspeaker placement without latency. Irregular loudspeaker settings can often occur in home living room environments due to structural constraints and aesthetic preferences. Thus, for best sound field reproduction, a rendering matrix optimized for such a scenario may be preferred in that it may allow for more accurate sound field reproduction.

[0061] Since an audio decoder typically does not require a lot of computational resources, the device cannot calculate an irregular rendering matrix in a consumer-friendly time. Various aspects of the techniques described in this disclosure may provide for using a cloud-based computing approach such as the following:
1. The audio decoder may send loudspeaker coordinates to the server (and in some cases also SPL measurement results obtained with a calibration microphone) via an Internet connection.
2. The cloud-based server may calculate the rendering matrix (and possibly a few different versions so that the consumer can later choose from these different versions).
3. The server may then send the rendering matrix (or a different version) back to the audio decoder over the internet connection.

  [0062] This approach is compared to the rendering matrices normally designed for regular speaker configurations or geometries (required for powerful processors to calculate these irregular rendering matrices May allow manufacturers to keep the cost of manufacturing the audio decoder low, while also facilitating more optimal audio playback. The algorithm for calculating the rendering matrix can also be optimized after the audio decoder is shipped, potentially reducing the cost of hardware revisions or even recalls. The technique may also collect a lot of information about the different loudspeaker settings of the consumer product that in some cases may be beneficial for future product development.

  [0063] In some cases, as described above, the system shown in FIG. 3 may not signal audio rendering information 2 in bitstream 21, but instead as metadata separated from bitstream 21. This audio rendering information 2 can be signaled. 3 or in combination with that described above, the system shown in FIG. 3 may signal part of the audio rendering information 2 in the bitstream 21 as described above, and this audio rendering as metadata separated from the bitstream 21. Part of information 3 may be signaled. In some examples, the audio encoding device 20 may output metadata that may then be uploaded to a server or other device. The audio decoding device 24 may then download or otherwise retrieve this metadata, which is then used to augment the audio rendering information extracted from the bitstream 21 by the audio encoding device 24. A bitstream 21 formed in accordance with the rendering information aspect of the technology is described below with respect to the example of FIGS. 8A-8D.

  [0064] FIG. 3 is a block diagram illustrating in more detail an example of the audio encoding device 20 shown in the example of FIG. 2 that may perform various aspects of the techniques described in this disclosure. The audio encoding device 20 includes a content analysis unit 26, a vector-based decomposition unit 27, and a directivity-based decomposition unit 28. Although briefly described below, more information regarding the audio encoding device 20 and various aspects of compressing or otherwise encoding the HOA coefficients can be found in “INTERPOLATION FOR DECOMPOSED” filed May 29, 2014. It is available in International Patent Application Publication No. WO 2014/194099 entitled “REPRESENTATIONS OF A SOUND FIELD”.

  [0065] The content analysis unit 26 analyzes the content of the HOA coefficient 11 to identify whether the HOA coefficient 11 represents content generated from a live recording or content generated from an audio object. Represents a configured unit. The content analysis unit 26 can determine whether the HOA coefficient 11 was generated from an actual sound field recording or an artificial audio object. In some cases, content analysis unit 26 passes HOA coefficient 11 to vector-based decomposition unit 27 when framed HOA coefficient 11 is generated from the recording. In some cases, the content analysis unit 26 passes the HOA coefficient 11 to the directivity-based synthesis unit 28 when the framed HOA coefficient 11 is generated from the synthesized audio object. The directivity-based combining unit 28 may represent a unit configured to perform directivity-based combining of the HOA coefficients 11 to generate the directivity-based bitstream 21.

  [0066] As shown in the example of FIG. 3, the vector-based decomposition unit 27 includes a linear lossless transformation (LIT) unit 30, a parameter calculation unit 32, a reordering unit 34, a foreground selection unit 36, and energy compensation. Unit 38, psychoacoustic audio coder unit 40, bitstream generation unit 42, sound field analysis unit 44, coefficient reduction unit 46, background (BG) selection unit 48, spatiotemporal interpolation unit 50, A quantization unit 52.

[0067] A linear reversible transform (LIT) unit 30 receives HOA coefficients 11 in the form of HOA channels, where each channel is a block or frame of coefficients associated with a given order, suborder of spherical basis functions ( HOA [k], where k may represent the current frame or block of samples). The matrix of HOA coefficients 11 may have dimension D: M × (N + 1) ² .

  [0068] The LIT unit 30 may represent a unit configured to perform a form of analysis called singular value decomposition. Although described with respect to SVD, the techniques described in this disclosure may be performed for any similar transformation or decomposition that provides a linearly uncorrelated, energy-compressed set of outputs. Also, references to “sets” in this disclosure are generally intended to refer to non-zero sets, unless specified otherwise, and are classical for sets including so-called “empty sets”. It is not intended to refer to any mathematical definition. An alternative transformation may comprise principal component analysis, often referred to as “PCA”. Depending on the context, PCA may be referred to by several different names such as discrete Karhunen-Loeve transform, Hotelling transform, eigenorthogonal decomposition (POD), and eigenvalue decomposition (EVD), to name a few examples . The characteristics of such operations that lead to the goal behind compressing audio data are “energy compression” and “decorrelation” of multi-channel audio data.

[0069] In any case, assuming that LIT unit 30 performs singular value decomposition (also sometimes referred to as "SVD") as an example, LIT unit 30 may convert HOA coefficient 11 into transformed HOA. It can be converted into two or more sets of coefficients. A “set” of transformed HOA coefficients may include a vector of transformed HOA coefficients. In the example of FIG. 3, the LIT unit 30 can perform SVD on the HOA coefficient 11 to generate a so-called V matrix, S matrix, and U matrix. SVD, in linear algebra, may represent a factorization of a y × z real matrix or complex matrix X, where X may represent multi-channel audio data such as HOA coefficient 11 in the following form:
X = USV *
U may represent a y × y real unitary or complex unitary matrix, where the y columns of U are known as the left singular vector of multichannel audio data. S may represent a y × z rectangular diagonal matrix with non-negative real numbers on the diagonal, where the diagonal value of S is known as a singular value of multi-channel audio data. V * (which may indicate a conjugate transpose of V) may represent a z × z real or complex unitary matrix, where the z columns of V * are known as the right singular vectors of multichannel audio data .

  [0070] In some examples, the V * matrix in the SVD formula referenced above is shown as a conjugate transpose of the V matrix to reflect that SVD can be applied to matrices with complex numbers. When applied to a matrix with only real numbers, the complex conjugate of the V matrix (ie, in other words, the V * matrix) may be considered a transpose of the V matrix. In the following, for ease of explanation, it is assumed that the HOA coefficient 11 comprises a real number, so that a V matrix is output by the SVD instead of a V * matrix. Moreover, although shown in this disclosure as a V matrix, references to the V matrix should be understood as referring to transposition of the V matrix where appropriate. Although assumed to be a V matrix, the technique may be applied to the HOA coefficients 11 with complex coefficients in a similar manner, where the output of the SVD is a V * matrix. Thus, the present technique should not be limited in this respect only to providing an application of SVD to generate a V matrix, but to an HOA coefficient 11 having a complex component to generate a V * matrix. Application of SVD may be included.

[0071] In this way, the LIT unit 30 has a US [k] vector 33 (which may represent a combined version of the S and U vectors) with dimension D: M × (N + 1) ² and dimension D. : SVD can be performed on the HOA coefficient 11 to output the V [k] vector 35 with (N + 1) ² × (N + 1) ² . Individual vector elements in the US [k] matrix may also be referred to as X _ps (k), while individual vectors in the V [k] matrix may be referred to as v (k).

[0072] Analysis of the U, S, and V matrices may reveal that they convey or represent the spatial and temporal characteristics of the underlying sound field represented above by X . Each of the N vectors in U (of M samples in length) are orthogonal to each other as a function of time (during the time period represented by M samples) (Which may also be referred to as directional information) may represent a separated, separated audio signal. Spatial properties representing the spatial shape and position (r, θ, φ) are instead expressed as individual i th vectors in the V matrix (each of length (N + 1) ² ), v ⁽ⁱ⁾ It can be represented by (k). Each individual element of the v ⁽ⁱ⁾ (k) vector may represent a HOA coefficient that describes the shape (including width) and position of the sound field for the associated audio object. Both the vectors in the U matrix and the vectors in the V matrix are normalized so that their root mean square energy is equal to one. Thus, the energy of the audio signal in U is represented by the diagonal elements in S. Thus, multiplying U and S to form US [k] (with individual vector elements _XPS (k)) represents an audio signal with true energy. The ability of SVD decomposition to decouple audio time signals (in U), their energy (in S), and their spatial properties (in V) supports various aspects of the techniques described in this disclosure. Can do. Furthermore, a model that synthesizes the underlying HOA [k] coefficient X by vector multiplication of US [k] and V [k] yields the term “vector-based decomposition” that is used throughout this document.

  [0073] Although described as being performed directly on the HOA coefficient 11, the LIT unit 30 may apply a linear reversible transform to the derivative of the HOA coefficient 11. For example, the LIT unit 30 can apply SVD on the power spectral density matrix derived from the HOA coefficient 11. By performing SVD on the power spectral density (PSD) of the HOA coefficient rather than the HOA coefficient itself, LIT unit 30 potentially calculates to perform SVD on one or more of processor cycles and storage space. The same source audio coding efficiency can be achieved as if SVD was applied directly to the HOA coefficients, while reducing the overall complexity.

  [0074] The parameter calculation unit 32 represents a unit configured to calculate various parameters such as correlation parameters (R), directivity characteristic parameters (θ, φ, r), and energy characteristics (e). Each of the parameters for the current frame may be denoted as R [k], θ [k], φ [k], r [k], and e [k]. The parameter calculation unit 32 can perform energy analysis and / or correlation (or so-called cross-correlation) on the US [k] vector 33 to identify the parameters. The parameter calculation unit 32 can also determine parameters for the previous frame, where the previous frame parameters are stored in the previous frame of the US [k−1] and V [k−1] vectors. Based on this, R [k−1], θ [k−1], φ [k−1], r [k−1], and e [k−1] may be indicated. The parameter calculation unit 32 can output the current parameter 37 and the previous parameter 39 to the sorting unit 34.

  [0075] The parameters calculated by the parameter calculation unit 32 may be used by the reordering unit 34 to reorder the audio objects to represent the natural evaluation or temporal continuity of the audio objects. The reordering unit 34 compares each of the parameters 37 from the first US [k] vector 33 against each of the parameters 39 for the second US [k−1] vector 33 in turn. Can do. The reordering unit 34 is the reordered US [k] matrix 33 '

And the rearranged V [k] matrix 35 '(in mathematical terms)

To the foreground sound (or dominant sound-PS (predominant sound) selection unit 36 ("foreground selection unit 36")) and the energy compensation unit 38 Based on the parameter 39, the various vectors in the US [k] matrix 33 and the V [k] matrix 35 can be reordered (using the Hungarian method as an example).

[0076] The sound field analysis unit 44 may represent a unit configured to perform sound field analysis on the HOA coefficient 11 to potentially achieve the target bit rate 41. Based on the analysis and / or the received target bit rate 41, the sound field analysis unit 44 determines the total number of instances of the psychoacoustic coder (total number of environment or background channels (BG _TOT ) and foreground channels, or in other words Which can be a function of the number of dominant channels, the total number of instantiations of the psychoacoustic coder can be denoted as numHOATransportChannels.

[0077] The sound field analysis unit 44 is also used to potentially achieve the target bit rate 41, as well as the total number of foreground channels (nFG) 45 and the minimum order of the background (or environmental) sound field. (N _BG , or alternatively MinAmbHOOrder), the corresponding number of actual channels representing the minimum order of the background sound field (nBGa = (MinAmbHOOrder + 1) ² ), and the index of the additional BG HOA channel to send (i ) (Which can be generically shown as background channel information 43 in the example of FIG. 3). Background channel information 42 may also be referred to as environmental channel information 43. Each of the remaining channels in numHOATransportChannels-nBGa is either “additional background / environment channel”, “active vector-based dominant channel”, “active directivity-based dominant signal”, or “completely non- It can be either “active”. In one aspect, the channel type may be a syntax element indicated by 2 bits (as “ChannelType”) (eg, 00: directivity-based signal, 01: vector-based dominant signal, 10: additional Environment signal, 11: inactive signal). The total number of background or environmental signals, nBGa, can be given by the number of times (MinAmbHOAorder + 1) ² + (in the above example) index 10 appears as the channel type in the bitstream for that frame.

  [0078] In any case, the sound field analysis unit 44 selects the number of background (or in other words environmental) channels and the number of foreground (or in other words dominant) channels based on the target bit rate 41. However, when the target bit rate 41 is relatively high (for example, when the target bit rate 41 is 512 Kbps or more), more background channels and / or foreground channels can be selected. In one aspect, in the header section of the bitstream, numHOATransportChannels may be set to 8, while MinAmbHOOrder is set to 1. In this scenario, four channels may be reserved in each frame to represent the background part or the environment part of the sound field, while the other four channels depend on the channel type for each frame. It can vary and can be used, for example, as either an additional background / environment channel or a foreground / dominant channel. The foreground / dominant signal can be one of either a vector-based signal or a directivity-based signal, as described above.

  [0079] In some cases, the total number of vector-based dominant signals for a frame may be given by the number of times the ChannelType index is 01 in the bitstream of that frame. In the above aspect, for each additional background / environment channel (eg, corresponding to a ChannelType of 10) which of the possible HOA coefficients (other than the first 4) can be represented in that channel Corresponding information. The information may be an index for indicating the HOA coefficients 5 to 25 for the fourth-order HOA content. The first four environmental HOA coefficients 1-4 may be sent whenever minAmbHOAorder is set to 1, so the audio encoding device is one of the additional environmental HOA coefficients with an index of 5-25. It may be necessary to show only one. That information may therefore be sent using a 5-bit syntax element (for 4th order content), which may be denoted as “CodedAmbCoeffIdx”. In any case, the sound field analysis unit 44 sends the background channel information 43 and the HOA coefficient 11 to the background (BG) selection unit 36, and the background channel information 43 to the coefficient reduction unit 46 and the bit stream generation unit 42. , NFG45 is output to the foreground selection unit 36.

[0080] The background selection unit 48 is based on background channel information (eg, background sound field (N _BG ) and number of additional BG HOA channels to send (nBGa) and index (i)), A unit configured to determine the background or environmental HOA factor 47 may be represented. For example, when N _BG is equal to 1, the background selection unit 48 can select the HOA coefficient 11 for each sample of an audio frame having an order of 1 or less. The background selection unit 48 can then select, in this example, the HOA coefficient 11 having the index specified by one of the indices (i) as an additional BG HOA coefficient, where nBGa is To be specified in the bitstream 21 to enable an audio decoding device such as the audio decoding device 24 shown in the examples of FIGS. 2 and 4 to parse the background HOA coefficient 47 from the bitstream 21 , Provided to the bitstream generation unit 42. The background selection unit 48 can then output the environmental HOA coefficient 47 to the energy compensation unit 38. The environmental HOA factor 47 may have a dimension D: M × [(N _BG +1) ² + nBGa]. The environmental HOA coefficients 47 may also be referred to as “environmental HOA coefficients 47”, where each of the environmental HOA coefficients 47 corresponds to a separate environmental HOA channel 47 to be encoded by the psychoacoustic audio coder unit 40. To do.

[0081] The foreground selection unit 36 reorders the US [k] representing the foreground or distinct components of the sound field based on the nFG 45 (which may represent one or more indices identifying the foreground vector). It may represent a unit configured to select the matrix 33 ′ and the sorted V [k] matrix 35 ′. The foreground selection unit 36 (reordered US [k] _{1, ..., nFG} 49, FG _{1, ..., nfG} [k] 49, or

NFG signal 49 can be output to psychoacoustic audio coder unit 40, where nFG signal 49 has dimension D: M × nFG and can each represent a mono audio object. The foreground selection unit 36 also ^outputs a rearranged V [k] matrix 35 ′ (or v ^(1..nFG) (k) 35 ′) corresponding to the foreground component of the sound field to the spatiotemporal interpolation unit 50. Where the subset of the sorted V [k] matrix 35 ′ corresponding to the foreground component is shown as a foreground V [k] matrix 51 _k with dimension D: (N + 1) ² × nFG. Get (this is

As mathematically).

[0082] The energy compensation unit 38 is a unit configured to perform energy compensation on the environmental HOA coefficient 47 to compensate for energy loss due to removal of various of the HOA channels by the background selection unit 48. Can be represented. The energy compensation unit 38 includes a rearranged US [k] matrix 33 ′, a rearranged V [k] matrix 35 ′, an nFG signal 49, a foreground V [k] vector 51 _k , and an environmental HOA coefficient 47. An energy analysis may be performed on one or more of the following, and then energy compensation may be performed based on the energy analysis to generate an energy compensated environmental HOA coefficient 47 '. The energy compensation unit 38 can output the energy-compensated environmental HOA coefficient 47 ′ to the psychoacoustic audio coder unit 40.

[0083] The spatio-temporal interpolation unit 50 performs the foreground V [k] vector 51 _k for the k th frame and the foreground V [k−1] for the previous frame (hence the notation k−1). Representing a unit configured to receive the vector 51 _k−1 and perform spatiotemporal interpolation to generate an interpolated foreground V [k] vector. The spatiotemporal interpolation unit 50 can recombine the nFG signal 49 with the foreground V [k] vector 51 _k to recover the sorted foreground HOA coefficients. The spatiotemporal interpolation unit 50 can then divide the sorted foreground HOA coefficients by the interpolated V [k] vector to produce an interpolated nFG signal 49 '. The spatiotemporal interpolation unit 50, so that it can restore the foreground V [k] vector 51 _k generated thereby foreground V [k] vector audio decoding device is interpolated, such as an audio decoding device 24, the interpolated The foreground V [k] vector 51 _k used to generate the foreground V [k] vector can be output. The foreground V [k] vector 51 _k that was used to generate the interpolated foreground V [k] vector is shown as the remaining foreground V [k] vector 53. In order to ensure that the same V [k] and V [k−1] are used in the encoder and decoder (to create the interpolated vector V [k]), the vector quantization is performed. A quantized / dequantized version may be used in the encoder and decoder. The spatiotemporal interpolation unit 50 may output the interpolated nFG signal 49 ′ to the psychoacoustic audio coder unit 46 and the interpolated foreground V [k] vector 51 to the coefficient reduction unit 46.

[0084] Coefficient reduction unit 46 performs coefficient reduction on the remaining foreground V [k] vector 53 based on background channel information 43 to output reduced foreground V [k] vector 55 to quantization unit 52. May represent a unit configured to perform The reduced foreground V [k] vector 55 may have dimension D: [(N + 1) ² − (N _BG +1) ² −BG _TOT ] × nFG. The coefficient reduction unit 46 may represent a unit configured in this respect to reduce the number of coefficients in the remaining foreground V [k] vector 53. In other words, coefficient reduction unit 46 includes units configured to remove coefficients in the foreground V [k] vector (forming the remaining foreground V [k] vector 53) that have little or no directivity information. Can be represented. In some examples, the coefficients of the clear or in other words foreground V [k] vectors, corresponding to the 1st and 0th order basis functions (which may be denoted N _BG ) provide little directivity information. Therefore, it can be removed from the foreground V vector (through a process that can be referred to as “coefficient reduction”). In this example, not only to identify the coefficient corresponding to N _BG , but also to identify an additional HOA channel (which may be indicated by the variable TotalOfAddAmbHOAChan) from the set of [(N _BG +1) ² +1, (N + 1) ² ] Greater flexibility may be provided.

[0085] Quantization unit 52 performs any form of quantization to compress reduced foreground V [k] vector 55 to generate coded foreground V [k] vector 57, and coding May represent a unit configured to output the generated foreground V [k] vector 57 to the bitstream generation unit 42. In operation, the quantization unit 52 may represent a unit configured to compress one or more of the spatial components of the sound field, ie, the reduced foreground V [k] vector 55 in this example. Quantization unit 52 may perform any one of the following 12 quantization modes, as indicated by the quantization mode syntax element denoted “NbtsQ”:
NbtsQ value Quantization mode type
0-3: Reserved
4: Vector quantization
5: Scalar quantization without Huffman coding
6: 6-bit scalar quantization with Huffman coding
7: 7-bit scalar quantization with Huffman coding
8: 8-bit scalar quantization with Huffman coding
. . . . . .
16: 16-bit scalar quantization with Huffman coding
The quantization unit may also perform a predictive version of any of the above types of quantization modes, where the difference is the V-vector of the previous frame (or when vector quantization was performed). Weight) and the determined V-vector element of the current frame (or weight when vector quantization is performed). Quantization unit 52 then quantizes the difference between the current frame element or weight of the current frame and the previous frame element or weight, rather than the value of the V vector element of the current frame itself. obtain.

  [0086] Quantization unit 52 obtains multiple versions of quantization for each of the reduced foreground V [k] vectors 55 to obtain multiple encoded versions of the reduced foreground V [k] vectors 55. Can be executed. Quantization unit 52 may select one of the coded versions of reduced foreground V [k] vector 55 as coded foreground V [k] vector 57. The quantization unit 52 is configured to use an unpredicted vector quantized V-vector, a predicted vector quantum for use as an output-switched and quantized V-vector based on any combination of criteria described in this disclosure. One of a quantized V vector, a non-Huffman coded scalar quantized V vector, and a Huffman coded scalar quantized V vector can be selected. In some examples, the quantization unit 52 selects a quantization mode from a set of quantization modes, including a vector quantization mode and one or more scalar quantization modes, and based on the selected mode (Or therefore) the input V-vector can be quantized. Quantization unit 52 then performs an unpredicted vector quantised V-vector (eg, with respect to a weight value or a bit indicating it), and a predicted vector-quantized V-vector (eg, with respect to an error value or a bit indicating it). , A non-Huffman-coded scalar quantized V-vector and a selected one of the Huffman-coded and scalar-quantized V-vectors are provided to the bitstream generation unit 52 as coded foreground V [k] vectors 57 be able to. The quantization unit 52 also includes a syntax element indicating the quantization mode (eg, NbitsQ syntax element) and any other syntax element used to dequantize or otherwise reconstruct the V vector. And can give.

  [0087] The psychoacoustic audio coder unit 40 included within the audio encoding device 20 may represent multiple instances of the psychoacoustic audio coder, each of which is encoded with an encoded environmental HOA coefficient 59. In order to generate the nFG signal 61, it is used to encode each different audio object or HOA channel of the energy compensated environmental HOA coefficient 47 'and the interpolated nFG signal 49'. The psychoacoustic audio coder unit 40 can output the encoded environmental HOA coefficient 59 and the encoded nFG signal 61 to the bitstream generation unit 42.

  [0088] A bitstream generation unit 42 included within the audio encoding device 20 formats the data to conform to a known format (which may refer to a format known by the decoding device), thereby providing a vector-based This represents a unit that generates the bitstream 21. In other words, the bitstream 21 may represent encoded audio data that has been encoded in the manner described above. Bitstream generation unit 42 may represent a multiplexer in some examples, which includes a coded foreground V [k] vector 57, an encoded environmental HOA coefficient 59, and an encoded nFG signal 61. And the background channel information 43 can be received. The bitstream generation unit 42 then generates a bit based on the coded foreground V [k] vector 57, the encoded environmental HOA coefficient 59, the encoded nFG signal 61, and the background channel information 43. Stream 21 can be generated. In this way, the bitstream generation unit can thereby specify a vector 57 in the bitstream 21 to obtain the bitstream 21. Bitstream 21 may include a main or main bitstream and one or more side channel bitstreams.

  [0089] Various aspects of the techniques may also allow the bitstream generation unit 46 to specify audio rendering information 2 in the bitstream 21, as described above. The current version of the upcoming 3D audio compression working draft provides signaling specific downmix matrices in the bitstream 21, while the working draft is used in the rendering HOA coefficients 11 in the bitstream. Does not provide a renderer specification. For HOA content, the equivalent of such a downmix matrix is a rendering matrix that converts the HOA representation into the desired loudspeaker feed. Various aspects of the techniques described in this disclosure may include HOA and channel content by allowing bitstream generation unit 46 to signal a HOA rendering matrix in a bitstream (eg, as audio rendering information 2). Propose further harmony with the feature set.

  [0090] One exemplary signaling solution optimized for HOA and based on a downmix matrix coding scheme is represented below. Similar to the transmission of the downmix matrix, the HOA rendering matrix can be signaled in mpegh3daConfigExtension (). The technique provides a new extension type ID_CONFIG_EXT_HOA_MATRIX as described in the following table italics and bold changes to existing tables (showing changes to existing tables in italics and bold): obtain.

  [0091] The bitfield HOARenderingMatrixSet () may be equal in structure and functionality compared to DownmixMatrixSet (). Instead of inputCount (audioChannelLayout), HOA RenderingMatrixSet () may use an “equivalent” NumOfHoaCoeffs value calculated in HOAConfig. Further, since the order of the HOA coefficients can be fixed in the HOA decoder (see, eg, Appendix G in the CD), HOARenderingMatrixSet need not be equivalent to any inputConfig (audioChannelLayout).

  [0092] Various aspects of the present technique may also include HOA audio data in which the bitstream generation unit 46 uses a first compression scheme (such as the decomposition compression scheme represented by the vector-based decomposition unit 27) (eg, When compressing the HOA 11 in the example of FIG. 4, a second compression scheme (eg, a directional-based compression scheme represented by a directional-based decomposition unit or a directional-based compression scheme (directionality-) It may be possible to specify a bitstream 21 such that the bits corresponding to the based compression scheme)) are not included in the bitstream 21. For example, the bitstream generation unit 42 generates the bitstream 21 such that use does not include a HOAPreductionInfo syntax element or field that may be reserved to specify prediction information between directional signals of a directional-based compression scheme. obtain. An example of a bitstream 21 generated in accordance with various aspects of the techniques described in this disclosure is shown in the examples of FIGS. 8E and 8F.

  [0093] In other words, directional signal prediction may be part of the dominant sound synthesis that is used by the directional base decomposition unit 28 and depends on the presence of ChannelType 0 (which may indicate a directional base signal). If the directional base signal is not present in the frame, directional signal prediction cannot be performed. However, even if not used, the associated sideband information HOAPreductionInfo () is written in every frame regardless of the presence of the directional base signal. If a directional signal is not present in the frame, the techniques described in this disclosure are such that the bitstream generation unit 42 is described in the following table (where an italic with an underline indicates addition): It may be possible to reduce the size of the sideband by not signaling HOAPreductionInfo in the sideband:

  [0094] In this regard, the present technique provides for the higher order ambisonic audio data when a device, such as the audio encoding device 20, compresses the higher order ambisonic audio data using the first compression technique. It may be possible to be configured to specify a bitstream that represents a compressed version of higher-order ambisonic audio data that does not include bits corresponding to a second compression scheme that is further used to compress.

  [0095] In some cases, the first compression technique comprises a vector-based decomposition compression scheme. In these and other cases, the vector-based decomposition compression scheme comprises a compression scheme that includes application of singular value decomposition (or equivalent to that described in more detail in this disclosure) to higher-order ambisonic audio data.

  [0096] In these and other cases, the audio encoding device 20 specifies a bitstream that does not include bits corresponding to at least one syntax element used to perform the second type of compression scheme. Can be configured to. The second compression scheme may comprise a directivity-based compression scheme, as described above.

  [0097] The audio encoding device 20 may also be configured to specify the bitstream 21 such that the bitstream 21 does not include bits corresponding to the HOAPPredictionInfo syntax element of the second compression scheme.

  [0098] If the second compression scheme comprises a directional-based compression scheme, the audio encoding device 20 may ensure that the bitstream 21 does not include bits corresponding to the HOAPreductionInfo syntax element of the directional-based compression scheme. It may be configured to specify a stream 21. In other words, the audio encoding device 20 specifies the bitstream 21 such that the bitstream 21 does not include bits corresponding to at least one syntax element used to perform the second type of compression scheme. The at least one syntax element indicates a prediction between two or more directional base signals. In other words, when the second compression technique comprises a directional-based compression scheme, the audio encoding device 20 ensures that the bitstream 21 does not include bits that correspond to the HOAPreductionInfo syntax element of the directional-based compression scheme. , May be configured to specify the bitstream 21. Here, the HOAPPredictionInfo syntax element indicates a prediction between two or more directional base signals.

  [0099] Various aspects of the techniques may further allow the bit scheme generation unit 46 to specify the bitstream 21 in certain cases where the bitstream 21 does not include gain correction data. The bitstream generation unit 46 may specify the bitstream 21 such that the bitstream 21 does not include gain correction data when gain correction is suppressed. An example of a bitstream 21 generated according to various aspects of the techniques is shown in the examples of FIGS. 8E and 8F, as described above.

  [0100] In some cases, assuming a relatively smaller dynamic range of certain types of psychoacoustic coding compared to other types of psychoacoustic coding, When the type is executed, gain correction is applied. For example, AAC has a dynamic range that is relatively smaller than unified speech and audio coding (USAC). If the compression scheme (such as a vector-based compression scheme or a directivity-based compression scheme) includes USAC, the bitstream generation unit 46 determines that the gain correction (in the HOAConfig having a value of zero in the bitstream) is the syntax element MaxGainCorArExpExp. May be signaled into the suppressed bitstream (by specifying), and then the bitstream 21 may be specified to not include gain correction data (in the HOAGainCollectionData () field).

  [0101] In other words, the bit field MaxGainCorrAmpExp (see Table 71 in the CD) as part of HOAConfig may control the range in which the automatic gain control module affects the transport channel signal prior to USAC core coding. In some cases, this module was developed to improve the non-ideal dynamic range of AAC encoder implementations where RM0 is available. With changes from the AAC to the USAC core coder during the integration phase, the dynamic range of the core encoder can be improved, so the need for this gain control module cannot be as critical as before.

  [0102] In some cases, gain control functionality can be suppressed when MaxGainCorrAmpExp is set to zero. In these cases, the associated sideband information HOAGainCollectionData () cannot be written to any HOA frame for each of the above tables indicating “HOAFframe syntax”. For configurations where MaxGainCorrAmpExp is set to 0, the techniques described in this disclosure may not signal HOAGainCollectionData. Further, in such a scenario, the inverse gain control module can be bypassed even when reducing decoder complexity by about 0.05 MOPS per transport channel without the impact of any negative aspects.

  [0103] In this regard, the present technique provides for higher-order ambisonic audio data so that the bitstream 21 does not include gain correction information when gain correction is suppressed during compression of the higher-order ambisonic audio data. Audio encoding device 20 may be configured to specify a bitstream 21 that represents a compressed version.

  [0104] In these and other cases, the audio encoding device 20 compresses the higher order ambisonic audio data according to a vector-based decomposition compression scheme to generate a compressed version of the higher order ambisonic audio data. Can be configured as follows. An example of a decomposition compression scheme is a singular value decomposition (or equivalent to that described in more detail above) to higher order ambisonic audio data to produce a compressed version of higher order ambisonic audio data. Can include applications.

  [0105] In these and other cases, the audio encoding device 20 may be configured to specify the MaxGainCorAmbExp syntax element in the bitstream 21 to zero to indicate that gain correction is suppressed. In some cases, the audio encoding device 20 is configured to specify a bitstream 21 such that when the gain correction is suppressed, the bitstream 21 does not include a HOAGainCollection data field that stores the gain correction data. obtain. In other words, the audio encoding device 20 specifies the MaxGainCorrAmbExp syntax element in the bitstream 21 to zero to indicate that the gain correction is suppressed and that the HOAGainCollection data field that stores the gain correction data is not included in the bitstream. Can be configured as follows.

  [0106] In these and other cases, the audio encoding device 20 may perform gain correction if the compression of the higher order ambisonic audio data includes speech acoustic integrated coding (USAC) for the higher order ambisonic audio data. It can be configured to suppress.

  [0107] The potential optimization described above for the signaling of various information in the bitstream 21 may be adapted or otherwise updated in a manner described in more detail below. Updates may be used to update only the various aspects discussed below, applied in conjunction with other updates, or described above. Thus, as described above, including the single update application described below for the optimization described above or any specific combination of applications described below for the optimization described above. Each potential combination of updates to optimization is considered.

  [0108] To specify a matrix in the bitstream, specify ID_CONFIG_EXT_HOA_MATRIX in mpegh3daConfigExtension () of the bitstream 21 as shown below in bold and highlighted in the table below. The following table shows the syntax for specifying part of mpegh3daConfigExtension () of bitstream 21:

ID_CONFIG_EXT_HOA_MATRIX in the above table provides a container that specifies a rendering matrix, which is denoted as “HoaRenderingMatrixSet ()”.

  [0109] The contents of the HoaRenderingMatrixSet () container may be defined according to the syntax described in the following table:

As shown directly in the table above, HoaRenderingMatrixSet () contains numHoaRenderingMatrixes, HoaRenderingMatrixId, CICPpeakerLayoutOutxx, HoaMatrixLenBits, and HoAMatrixLenBits.

  [0110] The numHoaRenderingMatrix syntax element may specify several HoaRenderingMatrixId definitions in the bitstream element. The HoaRenderingMatrixId syntax element may represent a field that uniquely defines an Id for a default HOA rendering matrix available at the decoder side or a transmitted HOA rendering matrix. In this regard, HoaRenderingMatrixId is an example of a signal value that includes two or more bits that define an index that indicates that the bitstream includes a matrix used to render spherical harmonics for multiple speaker feeds, or An example of a signal value including two or more bits defining an index associated with one of a plurality of matrices used to render spherical harmonics for a plurality of speaker feeds may be represented. The CICPpeakerLayoutIdx syntax element represents a value that describes the output loudspeaker layout for a given HOA rendering matrix and may correspond to the ChannelConfiguration element defined in ISO / IEC 23000 1-8. The HoaMatrixLenBits (also referred to as “HoaRenderingMatrixLenBits”) syntax element may specify the length of the following bitstream elements (eg, HoaRenderingMatrix () container) in bits.

[0111] The HoaRenderingMatrix () container includes NumOfHoaCoeffs followed by an outputConfig () container and an outputCount () container. The outputConfig () container may contain a channel configuration vector that specifies information about each loudspeaker. Bitstream generation unit 42 may assume this loudspeaker information, known from the channel configuration of the output layout. Each entry, outputConfig [i], represents the data structure with the following members:
Azimuth Angle (can indicate the absolute value of the speaker azimuth angle);
AzimuthDirection (can indicate azimuth direction using 0 for left and 1 for right as one example);
Elevation Angle (can indicate the absolute value of the speaker elevation);
ElevationDirection (which may indicate an elevation direction using 1 for down 0 for up as an example); and
isLFE (can indicate whether the speaker is a low frequency effect (LFE) speaker).
Bitstream generation unit 42 may call a helper function, in some cases, denoted as “findSymmetricSpeakers”, which may be further specified below:
pairType (which may store the value of SYMMETRIC, CENTER, or ASYMMETRIC (in some examples means a symmetric pair of two speakers));
symbolicpair-> originalPosition (for the SYMMETRI group only, the position of the second speaker in the group (eg, right) in the original channel configuration may be indicated).
The outputCount () container may specify a number of loudspeakers on which the HOA rendering matrix is defined.

  [0112] The bitstream generation unit 42 may specify a HoaRenderingMatrix () container according to the syntax described in the following table:

As shown directly in the table above, the numPairs syntax element is set to the value output from calling the findSymmetricSpeakers helper function using outputCount and outputConfig, using hasLfeRendering as input. Thus, numPairs may indicate the number of symmetric loudspeaker pairs identified in the output loudspeaker setup that can be considered for efficient symmetric coding. The precisionLevel syntax element in the above table may indicate the accuracy used for uniform quantization of the gain according to the following table:

  [0113] Describes the syntax of HoaRenderingMatrix () The gainLimitPerHoaOrder syntax element shown in the table above is a flag that indicates whether maxGain and minGain are specified here for orders lacking maxGain and minGain or for the full HOA rendering matrix Can be represented. The maxGain [i] syntax element may specify, as one example, the maximum actual gain in the matrix for the coefficient for the HOA order i expressed in decibels (dB). The minGain [i] syntax element may specify, as an example, the minimum actual gain in the matrix for the coefficient of HOA degree i expressed in dB. The isFullMatrix syntax element may represent a flag that indicates whether the HOA rendering matrix is sparse or sufficient. The firstSparseOrder syntax element may specify the first HOA order that is sparsely encoded if the HOA rendering matrix is specified as sparse for each isFullMatrix syntax element. The isHoaCoefSparse syntax element may represent a bitmask vector derived from the firstSparseOrder syntax element. The lfeExists syntax element may represent a flag that indicates whether one or more LFEs are present in the outputConfig. The hasLfeRendering syntax element indicates whether the rendering matrix includes non-zero elements for one or more LFE channels. The zeroOrderAlwaysPositive syntax element may represent a flag indicating whether the zeroth-order HOA order has only a positive value.

  [0114] The isAllValueSymmetric syntax element may represent a flag indicating whether all symmetric loudspeaker pairs have equal absolute values in the HOA rendering matrix. The isAnyValueSymmetric syntax element represents a flag that indicates whether some of the symmetric loudspeaker pairs have equal absolute values in the HOA rendering matrix, for example when false. The valueSymmetricPairs syntax element may represent a bitmask of length numPairs indicating a pair of loudspeakers with value symmetries. The isValueSymmetric syntax element may represent a bitmask derived in the manner shown in Table 3 from the valueSymmetricPairs syntax element. The isAllSignSymmetric syntax element may indicate whether all new metric loudspeaker pairs have at least number sign symmetries when there is no value symmetry in the matrix. The isAnySignSymmetric syntax element may represent a flag that indicates whether there are at least some symmetric loudspeaker pairs with number sign symmetry. The signSymmetricPairs syntax element may represent a bitmask of length numPairs indicating a loudspeaker pair with code symmetry. The isSignSymmetric variable may represent a bitmask derived from the signSymmetricPairs syntax element in the manner shown above in the table describing the syntax of HoaRenderingMatrix (). The hasVerticalCoef syntax element may represent a flag indicating whether the matrix is a horizontal only HOA rendering matrix. The bootVal syntax element may represent a variable used in the decoding loop.

  [0115] In other words, the bitstream generation unit 42 may use any one or more of the above value symmetry information (eg, isAllValueSymmetric syntax element, isAnyValueSymmetric syntax element, valueSymmetricPairs syntax element, isValueSyntax syntax element, isValueSyntax syntax element, Audio renderer 1 may be analyzed to generate (or any combination of one or more of) SymmetricPairs syntax elements) or otherwise to obtain value symmetric information. The bitstream generation unit 42 may specify the audio renderer information 2 in the bitstream 21 in the manner shown above such that the audio renderer information 2 includes value code symmetry information.

  [0116] In addition, the bitstream generation unit 42 may also include any one or more of the above code symmetry information (eg, isAllSignSymmetric syntax element, isAnySignalSymmetric syntax element, signSymmetricPairs syntax element, isSign Symmetric syntax element, Audio renderer 1 may be analyzed to generate (or any combination of one or more of the signSymmetricPairs syntax elements) or otherwise to obtain code symmetric information. The bitstream generation unit 42 may specify the audio renderer information 2 in the bitstream 21 in the manner shown above so that the audio renderer information 2 includes audio code symmetry information.

[0117] When determining value symmetry information and code symmetry information, the bitstream generation unit 42 may analyze various values of the audio renderer 1, specified as a matrix. The rendering matrix can be formulated as a pseudo inverse of the matrix R. In other words, to render an (N + 1) ² HOA channel (denoted as Z below) for L loudspeaker signals (denoted by a column vector of L loudspeaker signals, p), The equation of can be given:
Z = R * p
To arrive at a rendering matrix that outputs L loudspeaker signals, the inverse of the R matrix is multiplied by Z HOAch, as shown in the following equation:
p = R ^-1 * Z
If the number L of loudspeakers is not the same as the number of Z HOA channels, the matrix R will not be square and a perfect inverse cannot be determined. As a result, the pseudoinverse can be used instead as defined below:
pinv (R) = R ^T (R * RT) ⁻¹
Here, R ^T indicates transposition of the R matrix. Replacing R ^-1 in the above equation, the solution for the L loudspeaker signals indicated by the column vector p can be expressed mathematically as follows:
p = pinv (R) * Z = R ^T (R * RT) ⁻¹ * Z

[0118] The entry of the R matrix may be the value of the spherical harmonic for the loudspeaker position with (N + 1) ² rows and L columns for different spherical harmonics for the speaker. Bitstream generation unit 42 may determine a speaker pair based on the value for the loudspeaker. Analyzing the spherical harmonic values for the loudspeaker positions, the bitstream generation unit 42 determines which loudspeaker positions are paired (eg, pairs are similar, nearly the same, or the same value but opposite signs) based on the values. Can be determined).

  [0119] After identifying the pairs, the bitstream generation unit 42 may determine for each pair whether the pairs have the same or nearly the same value. When all of the pairs have the same value, the bitstream generation unit 42 may set the isAllValueSymmetric syntax element to 1. When all of the pairs do not have the same value, the bitstream generation unit 42 may set the isAllValueSymmetric syntax element to 0. Bitstream generation unit 42 may set the isAnyValueSymmetric syntax element to 1 when one, but not all, of the pairs have the same value. Bitstream generation unit 42 may set the isAnyValueSymmetric syntax element to 0 when none of the pairs have the same value. For pairs with symmetric values, the bitstream generation unit 42 only specifies one value rather than two separate values for the speaker pair, so that the audio rendering information 2 in the bitstream 21 is Reduce the number of bits used to represent (eg, the matrix in this example).

  [0120] When there is no value symmetry between the pair, the bitstream generation unit 42 also signifies that the speaker pair (meaning that one speaker has a negative value while the other speaker has a positive value). Can be determined for each pair. When all of the pairs have code symmetry, the bitstream generation unit 42 may set the isAllSignSymmetric syntax element to 1. When all of the pairs do not have code symmetry, the bitstream generation unit 42 may set the isAllSignSymmetric syntax element to 0. The bitstream generation unit 42 may set the isAnySignSymmetric syntax element to 1 when not all but one or more pairs have code symmetry. Bitstream generation unit 42 may set the isAnySignSymmetric syntax element to 0 when none of the pairs has code symmetry. For pairs with symmetric codes, the bitstream generation unit 42 may specify only one or no sign rather than two separate codes for the speaker pair, thereby providing audio rendering information in the bitstream 21. The number of bits used to represent (the matrix in this example) may be reduced.

  [0121] The bitstream generation unit 42 may specify the DecodeHoaMatrixData () container shown in the table that describes the syntax of HoaRenderingMatrix () according to the syntax shown in the following table:

  [0122] The hasValue syntax element in the preceding table describing the syntax of DecodeHoaMatrixData may represent a flag indicating whether the matrix element is sparsely encoded. The signMatrix syntax element may represent a matrix with the code values of the HOA rendering matrix in linearized vector form as one example. The hoaMatrix syntax element may represent a HOA rendering matrix value in a linearized vector format as one example. Bitstream generation unit 42 may specify a DecodeHoaGainValue () container shown in a table that describes the syntax of DecodeHoaMatrixData according to the syntax shown in the following table:

  [0123] The bitstream generation unit 42 may specify the readRange () container shown in the table that describes the syntax of DecodeHoaGainValue according to the syntax specified in the following table:

  [0124] Although not shown in the example of FIG. 3, audio encoding device 20 also encodes whether the current frame should be encoded using directional-based combining or vector-based combining. A bitstream output unit that switches a bitstream output from the audio encoding device 20 (eg, between a directivity-based bitstream 21 and a vector-based bitstream 21) based on what should be done Can be included. The bitstream output unit is either a directional-based synthesis performed (as a result of detecting that the HOA coefficient 11 was generated from the synthesized audio object) or a vector (as a result of detecting that the HOA coefficient was recorded). The switching can be performed based on a syntax element output by the content analysis unit 26 that indicates whether base composition has been performed. The bitstream output unit can specify the correct header syntax to indicate the switch or current encoding used for the current frame with each of the bitstreams 21.

[0125] Moreover, as described above, the sound field analysis unit 44 can identify a BG _TOT environmental HOA coefficient 47 that can vary from frame to frame (but sometimes BG _TOT is more than one Can remain constant or the same over adjacent (temporal) frames). Changes in BG _TOT can result in changes to the coefficients represented in the reduced foreground V [k] vector 55. Changes in BG _TOT can result in background HOA coefficients (sometimes referred to as “environmental HOA coefficients”) that change from frame to frame (although again, sometimes BG _TOT has more than one (in time) ) Can remain constant or the same across adjacent frames). This change is represented by the addition or removal of additional environmental HOA coefficients and the corresponding removal of coefficients from the reduced foreground V [k] vector 55 or addition of coefficients thereto. This often causes energy changes.

  [0126] As a result, the sound field analysis unit 44 further determines when the environmental HOA coefficient changes from frame to frame and changes to the environmental HOA coefficient with respect to being used to represent the environmental components of the sound field. Or other syntax elements may be generated (where this change may also be referred to as an environmental HOA coefficient “transition” or an environmental HOA coefficient “transition”). Specifically, the coefficient reduction unit 46 generates a flag (which may be indicated as an AmbCoeffTransition flag or an AmbCoeffIdxTransition flag), which may be included in the bitstream 21 (possibly as part of the side channel information). As such, the flag can be provided to the bitstream generation unit 42.

[0127] In addition to specifying the environmental coefficient transition flag, the coefficient reduction unit 46 may also modify the manner in which the reduced foreground V [k] vector 55 is generated. In one example, if one of the environmental HOA environmental coefficients is determined to be transitioning during the current frame, coefficient reduction unit 46 may reduce the reduced foreground V [k] vector corresponding to the environmental HOA coefficient being transitioned. For each of the 55 V vectors, a vector coefficient (which may also be referred to as a “vector element” or “element”) may be specified. Again, the transitional environmental HOA coefficients may add or remove the total number of background coefficients from the BG _TOT . Thus, the change that occurs in the total number of background coefficients indicates that the environmental HOA coefficients are included or not included in the bitstream and that the corresponding elements of the V vector are the second configuration mode described above and It affects whether it is included for the V vector specified in the bitstream in the third configuration mode. More information regarding how the coefficient reduction unit 46 can specify a reduced foreground V [k] vector 55 to overcome changes in energy was filed on Jan. 12, 2015. U.S. Patent Application No. 14 / 594,533 entitled "TRANSITIONING OF AMBIENT HIGH_ORDER AMBISONIC COEFFICIENTS".

  [0128] FIG. 4 is a block diagram illustrating the audio decoding device 24 of FIG. 2 in more detail. As shown in the example of FIG. 4, the audio decoding device 24 may include an extraction unit 72, a renderer reconstruction unit 81, a directivity-based reconstruction unit 90, and a vector-based reconstruction unit 92. As described below, more information regarding the audio decoding device 24 and various aspects of decompressing or otherwise decoding the HOA coefficients can be found in "INTERPOLATION FOR DECOMPOSED REPREENTATIONS OF A" filed May 29, 2014. It is available in International Patent Application Publication No. WO 2014/194099 entitled “SOUND FIELD”.

  [0129] The extraction unit 72 receives the bitstream 21, receives the audio rendering information 2 and various encoded versions of the HOA coefficients 11 (eg, directivity-based encoded version or vector-based encoding). The unit configured to extract the In other words, a higher order ambisonic (HOA) rendering matrix may be transmitted by the audio encoding device 20 that allows control over the HOA rendering process in the audio playback system 16. Transmission may be facilitated by means of mpegh3daConfigExtension of type ID_CONFIG_EXT_HOA_MATRIX indicated above. mpegh3daConfigExtension may include several HOA rendering matrices for different loudspeaker playback configurations. When the HOA rendering matrix is transmitted, the audio encoding device 20 signals for each HOA rendering matrix an associated target loudspeaker layout that determines the dimensions of the rendering matrix along with the dimensions of the HoaOrder.

  [0130] The transmission of the unique HoaRenderingMatrixId allows to refer to the default HOA rendering matrix available in the audio playback system 16 or the HOA rendering matrix transmitted from outside the audio bitstream 21. In some cases, every HOA rendering matrix is assumed to be standardized in N3D and follows the ordering of HOA coefficients as defined in bitstream 21.

[0131] As described above, the function findSymmetricSpeakers, as one example, determines the number and position of all loudspeakers within a provided loudspeaker setup that is symmetric with respect to the listener's midplane as a so-called "sweet spot". Show. This helper function can be defined as: int findSymmetricSpeakers (int outputCount, SpeakerInformation * outputConfig, int hasLfeRendering); the extraction unit 72 can then be used to generate a matrix associated with the symmetric loudspeaker. The function createSymSigns may be called to compute a vector of 1.0 and -1.0 values. This createSymSigns function can be defined as follows:
void createSymSigns (int * symSigns, int hoaOrder)
{
int n, m, k = 0;
for (n = 0; n <= hoaOrder; ++ n) {
for (m = -n; m <= n; ++ m)
symSigns [k ++] = ((m> = 0) * 2) -1;
}
}

[0132] The extraction unit 72 may call the function create2dBitmask to generate a bitmask to identify the HOA coefficients that are simply used for the horizontal plane. The create2dBitmask function can be defined as follows:
void create2dBitmask (int * bitmask, int hoaOrder)
{
int n, m, k = 0;
bitmask [k ++] = 0;
for (n = 1; n <= hoaOrder; ++ n) {
ffor (m = -n; m <= n; ++ m)
bitmask [k ++] = abs (m)! = n;
}
}

  [0133] To decode the HOA rendering matrix coefficients, the extraction unit 72 may first extract the syntax element HoaRenderingMatrixSet (), which achieves HOA rendering in the desired loudspeaker layout, as described above. May include one or more HOA rendering matrices that may be applied to. In some cases, a given bitstream may not contain more than one instance of HoaRenderingMatrixSet (). The syntax element HoaRenderingMatrix () may include HOA rendering matrix information (which may be shown as renderer information 2 in the example of FIG. 4). The extraction unit 72 can first be read in the configuration information. It may guide the decoding process. Thereafter, the extraction unit 72 may read the matrix elements accordingly.

  [0134] In some cases, the extraction unit 72 first reads the fields precisionLevel and gainLimitPerOrder. If the flag gainLimitPerOrder is set, the extraction unit 72 reads and decodes the maxGain and minGain fields separately for each HOA order. If the flag gainLimitPerOrder is not set, the extraction unit 72 reads and decodes the fields maxGain and minGain once and applies these fields to all HOA orders during the decoding process. In some cases, the minGain value must be between 0 db and -69 dB. In some cases, the maxGain value must be between 1 dB and 111 dB, which is less than the minGain value. FIG. 9 is a diagram illustrating an example of the minimum gain and the maximum gain depending on the HOA order in the HOA rendering matrix.

  [0135] Extraction unit 72 may then read the flag isFullMatrix. It may signal whether the matrix is defined as full or partially sparse. If the matrix is defined as partially sparse, the extraction unit 72 reads the next field (eg, firstSparseOrder syntax element). It specifies the HOA order in which the HOA rendering matrix is sparsely encoded. The HOA rendering matrix is often dense for the low order and can be sparse for the high order, depending on the loudspeaker playback setup. FIG. 10 is a diagram illustrating a partially sparse 6th order HOA rendering matrix for 22 loudspeakers. The sparseness of the matrix shown in FIG. 10 starts with the 26th HOA coefficient (HOA order 5).

  [0136] Depending on whether a low frequency effects (LFE) channel is present in the loudspeaker playback setup (indicated by the lfeExists syntax element), the extraction unit 72 may read the field hasLfeRendering. If hasLfeRendering is not set, the extraction unit 72 is configured to assume that the matrix elements for the LFE channel are digital zeros. The next field read by the extraction unit 72 is the flag zeroOrderAlwaysPositive. It signals whether the matrix element associated with the zeroth order coefficient is positive. If zeroOrderAlwaysPositive indicates that the zeroth order HOA coefficient is positive, the extraction unit 72 determines that the number code is not coded for the rendering matrix coefficients corresponding to the zeroth order HOA coefficient.

[0137] In the following, the properties of the HOA rendering matrix may be signaled for loudspeaker pair symmetry with respect to the median plane. In some cases, there are two symmetry properties for a) value symmetry and b) code symmetry. In the case of value symmetry, the matrix element of the left loudspeaker of the symmetry loudspeaker pair is not coded, but rather the extraction unit 72 decodes the right loudspeaker by using the helper function createSymSigns, which performs the following: Derive these elements from the given matrix elements:
pairIdx, outputConfig [j]. symbolicpair->originalPosition;
hoaMatrix [i * outputCount + j] = hoaMatrix [i * outputCount + pairIdx;];
signMatrix [i * outputCount + j] = symSigns [i] * signMatrix [i * outputCount + pairIdx].

[0138] When the loudspeaker pairs are not value symmetric, then the matrix elements can be symmetric with respect to their number codes. When the loudspeaker pair is code symmetric, the number code of the matrix element of the left loudspeaker of the symmetric loudspeaker pair is not coded and the extraction unit 72 uses the helper function createSymSigns to perform the following: Derive these number codes from the number codes of the matrix elements associated with the right loudspeaker:
pairIdx = outputConfig [j]. symbolicpair->originalPosition;
signMatrix [i * outputCount + j] = symSigns [i] * signMatrix [i * outputCount + pairIdx];

  [0139] FIG. 11 is a diagram for explaining symmetry property signaling. A loudspeaker pair may be defined as a value symmetric and a code symmetric at the same time. The final decoding flag has a flag hasVerticalCoef that specifies whether the matrix elements associated with the circular (ie 2D) HOA coefficients are coded. If hasVerticalCoef is not set, the matrix element associated with the HOA coefficient defined by the helper function create2dBitmask is set to digital zero.

  [0140] That is, the extraction unit 72 may extract the audio rendering information 2 according to the process described in FIG. The extraction unit 72 may first read the isAllValueSymmetric syntax element from the bitstream 21 (300). When the isAllValueSymmetric syntax element is set to 1 (or in other words, Boolean true), the extraction unit 72 will effectively set the valueSymmetricPairs array to a value of 1 (indicating that all of the speaker pairs are value syntax). A syntax element may be set and repeated through the value of the numPairs syntax element (302).

  [0141] When the isAllValueSymmetric syntax element is set to 0 (or in other words, Boolean false), the extraction unit 72 may then read the isAnyValueSymmetric syntax element (304). When the isAnyValueSymmetric syntax element is set to 1 (or in other words, Boolean true), the extraction unit 72 sets the valueSymmetricPairs array syntax element to the bits that are read sequentially from the bitstream 21, and the numPairs syntax syntax element. It can be repeated through the value of the element (306). Extraction unit 72 may also obtain the isAnySignSymmetric syntax element for any of the pairs having a valueSymmetricPairs syntax element set set to zero. Thereafter, the extraction unit 72 repeats again through the number of pairs and sets the signSymmetricPairs bit to the value read from the bitstream 21 when valueSymmetricPairs is equal to 0.

  [0142] The extraction unit 72 may read the isAllSignSymmetric syntax element from the bitstream 21 when the isAnyValueSymmetric syntax element is set to 0 (or in other words, Boolean false). When the isAllSignSymmetric syntax element is set to a value of 1 (or in other words Boolean true), the extraction unit 72 is set to a value of 1 (effectively indicating that all of the speaker pairs are sign new metrics). The signSymmetricPairs array syntax element can be set and iterated through the values of the numPairs syntax element.

  [0143] When the isAllSignSymmetric syntax element is set to 0 (or in other words Boolean false), the extraction unit 72 may read the isAnySignSymmetric syntax element from the bitstream 21. Extraction unit 72 may set the signSymmetricPairs array syntax element to the bits read sequentially from bitstream 21 and iterate through the value of the numPairs syntax element. Bitstream generation unit 42 may perform an inverse process to that described above with respect to extraction unit 72 to specify value symmetry information, code symmetry information, or a combination of both value and code symmetry information.

  [0144] The renderer reconstruction unit 81 represents a unit configuration d to reconstruct the renderer based on the audio rendering information 2. That is, using the properties described above, the renderer reconstruction unit 81 can read a series of matrix element gain values. To read the absolute gain value, the renderer reconstruction unit 81, function DecodeGainValue () may be called. The renderer reconstruction unit 81 may call the alphabet index function ReadRange () to uniformly decode the gain values. When the decoded gain value is not digital zero, the renderer reconstruction unit 81 may additionally read the number code value (as per the table below). When a matrix element is associated with a sparsely signaled HOA coefficient (via isHoaCoefSparse), the hasValue flag is placed before the gainValueIndex (see Table b). When the hasValue flag is 0, this element is set to digital 0 and the gainValueIndex and sign are not signaled.

  [0145] With the specified symmetry property for the loudspeaker pair, the renderer reconstruction unit 81 may derive matrix elements associated with the left loudspeaker from the right loudspeaker. In this case, in order to decode the matrix elements for the left loudspeaker, the audio rendering information 2 in the bitstream 21 is reduced or omitted accordingly.

  [0146] Thus, the audio decoding device 24 may determine symmetry information to reduce the size of the specified audio rendering information. In some cases, the audio decoding device 24 may determine symmetry information to reduce the size of the specified audio rendering information and derive at least a portion of the audio renderer based on the symmetry information.

  [0147] In these and other cases, audio decoding device 24 may determine value symmetry information to reduce the size of the specified audio rendering information. In these and other cases, audio decoding device 24 may derive at least a portion of the audio renderer based on the value symmetry information.

  [0148] In these and other cases, audio decoding device 24 may determine code symmetry information to reduce the size of the specified audio rendering information. In these and other cases, audio decoding device 24 may derive at least a portion of the audio renderer based on the code symmetry information.

  [0149] In these and other cases, the audio decoding device 24 may determine sparseness information that indicates the sparseness of the matrix used to render the spherical harmonics into multiple speaker feeds.

  [0150] In these and other cases, audio decoding device 24 may determine the speaker layout in which the matrix is to be used to render spherical harmonics into multiple speaker feeds.

  [0151] Thereafter, at this point, the audio decoding device 24 may determine the audio rendering information 2 specified in the bitstream. Based on the signal values included in the audio rendering information 2, the audio playback system 16 may render multiple speaker feeds 25 using one of the audio renderers 22. The speaker feed can derive the speaker 3. As described above, the signal values are in some cases used to render spherical harmonic coefficients in multiple speaker feeds (decoded and provided as one of the audio renderers 22). Can be included. In this case, the audio playback system 16 may use this one of the audio renderers 22 to configure one of the audio renderers 22 in a matrix to render the speaker feed 25 based on the matrix. .

  [0152] Various encoded versions of the HOA coefficient 11 are extracted and then decoded so that the HOA coefficient 11 can be rendered using the acquired audio renderer 22 , The extraction unit 72 can determine from the syntax elements described above that indicate whether the HOA coefficients 11 were encoded via various direction-based versions or vector-based versions. . When directivity-based encoding is performed, the extraction unit 72 uses the directivity-based version of the HOA coefficient 11 and the syntax elements associated with the encoded version (directivity-based information 91 in the example of FIG. 4). And the directivity base information 91 can be passed to the directivity base reconstruction unit 90. Directivity base reconstruction unit 90 may represent a unit configured to reconstruct HOA coefficients in the form of HOA coefficients 11 ′ based on directivity base information 91.

  [0153] When the syntax element indicates that the HOA coefficient 11 has been encoded using vector-based decomposition, the extraction unit 72 may use the coded foreground V [k] vector 57 (coded weights 57 and / or Or an index 63 or scalar quantized V-vector), the encoded environmental HOA coefficients 59, and the corresponding audio object 61 (also referred to as an encoded nFG signal 61) can be extracted. . Each audio object 61 corresponds to one of the vectors 57. The extraction unit 72 passes the coded foreground V [k] vector 57 to the V vector reconstruction unit 74 and passes the encoded environmental HOA coefficient 59 along with the encoded nFG signal 61 to the psychoacoustic decoding unit 80. Can do.

  [0154] V vector reconstruction unit 74 may represent a unit configured to reconstruct a V vector from the encoded foreground V [k] vector 57. The V vector reconstruction unit 74 can operate in a manner opposite to that of the quantization unit 52.

  [0155] The psychoacoustic decoding unit 80 decodes the encoded environmental HOA coefficient 59 and the encoded nFG signal 61, thereby energy-compensated environmental HOA coefficient 47 'and interpolated nFG signal 49'. Can be operated in the opposite manner to the psychoacoustic audio coder unit 40 shown in the example of FIG. 3 to generate (interpolated nFG audio object 49 ′). The psychoacoustic decoding unit 80 can pass the energy-compensated environmental HOA coefficient 47 ′ to the fade unit 770 and the nFG signal 49 ′ to the foreground organization unit 78.

[0156] The spatiotemporal interpolation unit 76 may operate in a manner similar to that described above with respect to the spatiotemporal interpolation unit 50. Spatiotemporal interpolation unit 76 receives the reduced foreground V [k] vector 55 _k, also in order to generate the interpolated foreground V [k] vector 55 k _'', foreground V [k] vector Spatiotemporal interpolation can be performed on 55 _k and reduced foreground V [k−1] vector 55 _k−1 . The spatiotemporal interpolation unit 76 can forward the interpolated foreground V [k] vector 55 _k ″ to the fade unit 770.

[0157] The extraction unit 72 may also output a signal 757 to the fade unit 770 indicating when one of the environmental HOA coefficients is in transition, and the fade unit 770 then outputs the SCH _BG 47 '( Here, SCH _BG 47 ′ may also be referred to as “environmental HOA channel 47 ′” or “environmental HOA coefficient 47 ′”) and any of the interpolated foreground V [k] vector 55 _k ″ elements. It can be determined whether a fade-in or fade-out should be performed. In some examples, the fade unit 770 can operate in the opposite manner with respect to each of the elements of the environmental HOA coefficient 47 ′ and the interpolated foreground V [k] vector 55 _k ″. That is, fade unit 770 can perform a fade-in or fade-out, or both fade-in or fade-out, with respect to a corresponding one of environmental HOA coefficients 47 ', while interpolated foreground V [k A fade-in or fade-out or both fade-in and fade-out can be performed on the corresponding one of the elements of the vector 55 _k ″. The fade unit 770 can output the adjusted environmental HOA coefficient 47 ″ to the HOA coefficient knitting unit 82 and output the adjusted foreground V [k] vector 55 _k ″ ″ to the foreground knitting unit 78. In this regard, the fade unit 770 may perform a fade operation with respect to various aspects of the HOA coefficients or derivatives thereof, eg, in the form of elements of an environmental HOA coefficient 47 ′ and an interpolated foreground V [k] vector 55 _k ″. Represents a unit configured to perform

[0158] The foreground organization unit 78 is configured to perform matrix multiplication on the adjusted foreground V [k] vector 55 _k '''and the interpolated nFG signal 49' to generate the foreground HOA coefficient 65. Unit may represent In this regard, the foreground knitting unit 78 is another way to show the vector 55k ′ ″ and (interpolated nFG49 ′) to reconstruct the dominant aspect of the foreground or in other words the HOA coefficient 11 ′. ) Audio object 49 'can be combined. Foreground knitting unit 78 may perform matrix multiplication of the adjusted foreground V [k] vector 55 k _'' 'NFG signal 49 interpolated by'.

  [0159] The HOA coefficient organization unit 82 may represent a unit configured to combine the foreground HOA coefficient 65 with the adjusted environmental HOA coefficient 47 "to obtain the HOA coefficient 11 '. The prime notation reflects that the HOA coefficient 11 'may be similar to the HOA coefficient 11 but not the same. The difference between the HOA coefficient 11 and the HOA coefficient 11 'may be due to loss due to transmission over a lossy transmission medium, quantization, or other lossy operations.

  [0160] Furthermore, the extraction unit 72 and the audio decoding device 24 are also potentially optimized in the manner described above with respect to not including various syntax elements or data fields in more general cases. May be configured to operate in accordance with various aspects of the techniques described in this disclosure to obtain a particular bitstream 21.

  [0161] In some cases, the audio decoding device 24 compresses the higher order ambisonic audio data when decompressing the higher order ambisonic audio data compressed using the first compression scheme. Can be configured to obtain a bitstream 21 representing a compressed version of higher-order ambisonic audio data that does not include bits corresponding to a second compression scheme that is used in The first compression scheme comprises a vector-based compression scheme and is defined in the resulting vector spherical harmonic domain and sent via the bitstream 21. Vector-based decomposition compression schemes include, in some cases, compression schemes that include applications for singular value decomposition for higher-order ambisonic audio data (or equivalents as detailed in connection with the example of FIG. 3). Including.

  [0162] The audio decoding device 24 may be configured to obtain a bitstream 21 that does not include bits corresponding to at least one syntax element used to perform the second type of compression scheme. As described above, the second compression scheme comprises a directivity-based compression scheme. More particularly, the audio decoding device 24 may be configured to obtain a bitstream 21 that does not include bits corresponding to the HOAPreductionInfo syntax element of the second compression scheme. In other words, when the second compression scheme comprises a directivity-based compression scheme, the audio decoding device 24 is configured to obtain a bitstream 21 that does not include bits corresponding to the HOAPreductionInfo syntax element of the directivity-based compression scheme. Can be done. As described above, the HOAPPredictionInfo syntax element may indicate a prediction between two or more direction-based signals.

  [0163] In some cases, as an alternative to or in conjunction with the above examples, in some cases, when audio decoding device 24 is suppressed during compression of higher order ambisonic audio data , May be configured to obtain a bitstream 21 representing a compressed version of higher-order ambisonic audio data that does not include gain correction data. In these cases, audio decoding device 24 may be configured to decompress higher-order ambisonic audio data according to a vector-based synthesis decompression scheme. The compressed version of higher order ambisonic data is generated through the application of singular value decomposition (or equivalent described in more detail with respect to the example of FIG. 3 above) for higher order ambisonic audio data. When SVD is applied to or equivalent to HOA audio data, the audio encoding device 20 specifies at least one of the resulting vectors or bits representing them in the bitstream 21. Here, the vector describes the spatial characteristics of the corresponding foreground audio object (eg, the width, position, volume, etc. of the corresponding foreground audio object).

  [0164] In more detail, audio decoding device 24 may be configured to obtain a bitstream 21MaxGainCorAmbExp syntax element having a value set to 0 to indicate that gain correction is suppressed. That is, the audio decoding device 24 may be configured to obtain a bitstream that does not include a HOAGainCollection data field in which the bitstream stores the gain correction when the gain correction is suppressed. Bitstream 21 may include a MaxGainCorrAmbExp syntax element having a value of 0 indicating that gain correction is suppressed and may not include a HOAGainCollection data field that stores gain correction data. The compression of higher order ambisonic audio data may occur when it includes speech acoustic integrated coding (USAC) applications for higher order ambisonic audio data.

  [0165] FIG. 5 is an exemplary operation of an audio encoding device, such as the audio encoding device 20 shown in the example of FIG. 3, in performing various aspects of the vector-based synthesis techniques described in this disclosure. It is a flowchart which shows. Initially, the audio encoding device 20 receives the HOA coefficient 11 (106). The audio encoding device 20 can call the LIT unit 30, which can convert the transformed HOA coefficients (eg, in the case of SVD, the transformed HOA coefficients are the US [k] vector 33 and the V [k] vector. LIT can be applied (107) with respect to the HOA coefficients.

  [0166] Audio encoding device 20 may then use US [k] vector 33, US [k-1] vector 33, V [k] vector, and V [k] vector to identify various parameters in the manner described above. The parameter calculation unit 32 can be invoked to perform the analysis described above for any combination of V / [k-1] vectors 35. That is, parameter calculation unit 32 may determine at least one parameter based on an analysis of the converted HOA coefficients 33/35 (108).

[0167] Audio encoding device 20 may then invoke reordering unit 34, which reordered transformed HOA coefficients 33 '/ 35' (as described above). Or in other words, to generate a US [k] vector 33 ′ and a V [k] vector 35 ′), the transformed HOA coefficients (again in the context of SVD, US [k] Vector 33 and V [k] vector 35 can be reordered (109). The audio encoding device 20 may also call the sound field analysis unit 44 during any of the aforementioned operations or subsequent operations. The sound field analysis unit 44, as explained above, is the total number of foreground channels (nFG) 45, the order of the background sound field (N _BG ), the number of additional BG HOA channels to be sent (nBGa) and Perform a sound field analysis on the HOA coefficient 11 and / or the transformed HOA coefficient 33/35 to determine the index (i) (which may be collectively shown as background channel information 43 in the example of FIG. 3). (109).

  [0168] The audio encoding device 20 may also invoke the background selection unit 48. Background selection unit 48 may determine background or environmental HOA coefficient 47 based on background channel information 43 (110). The audio encoding device 20 may further invoke a foreground selection unit 36, which may be based on nFG 45 (which may represent one or more indices identifying the foreground vector). Alternatively, a sorted US [k] vector 33 ′ and a sorted V [k] vector 35 ′ that represent distinct components can be selected (112).

  [0169] The audio encoding device 20 may invoke the energy compensation unit 38. The energy compensation unit 38 performs energy compensation (114) on the environmental HOA coefficient 47 to compensate for energy loss due to removal of various of the HOA coefficients by the background selection unit 48, thereby being energy compensated. An environmental HOA coefficient 47 'can be generated.

  [0170] The audio encoding device 20 may also invoke the spatiotemporal interpolation unit 50. The spatiotemporal interpolation unit 50 is also referred to as an interpolated foreground signal 49 ′ (also referred to as “interpolated nFG signal 49 ′”) and the remaining foreground directivity information 53 (also referred to as “V [k] vector 53”). In order to obtain, a spatiotemporal interpolation may be performed on the reordered transformed HOA coefficients 33 '/ 35' (116). The audio encoding device 20 can then invoke the coefficient reduction unit 46. Coefficient reduction unit 46 obtains reduced foreground directivity information 55 (which may also be referred to as reduced foreground V [k] vector 55) based on background channel information 43 to provide the remaining foreground V [k. A coefficient reduction may be performed on the vector 53 (118).

  [0171] The audio encoding device 20 then compresses the reduced foreground V [k] vector 55 and generates a coded foreground V [k] vector 57 in the manner described above. Can be invoked (120).

  [0172] The audio encoding device 20 may also call the psychoacoustic audio coder unit 40. The psychoacoustic audio coder unit 40 generates an encoded environmental HOA coefficient 59 and an encoded nFG signal 61 for each of the energy compensated environmental HOA coefficient 47 'and the interpolated nFG signal 49'. The vector can be psychoacoustically coded. The audio encoding device can then invoke the bitstream generation unit 42. The bitstream generation unit 42 generates the bitstream 21 based on the coded foreground directivity information 57, the coded environment HOA coefficient 59, the coded nFG signal 61, and the background channel information 43. be able to.

  [0173] FIG. 6 is a flowchart illustrating an example operation of an audio decoding device, such as the audio decoding device 24 shown in FIG. 4, in performing various aspects of the techniques described in this disclosure. Initially, audio decoding device 24 may receive bitstream 21 (130). Upon receipt of the bitstream, audio decoding device 24 may invoke extraction unit 72. For purposes of explanation, assuming that the bitstream 21 indicates that a vector-based reconstruction should be performed, the extraction device 72 parses the bitstream to retrieve the information described above and converts the information into a vector. It can be passed to the base reconstruction unit 92.

  [0174] In other words, the extraction unit 72 includes coded foreground directivity information 57 (also referred to as coded foreground V [k] vector 57), coded environment HOA coefficients 59, and A coded foreground signal (which may also be referred to as coded foreground nFG signal 59 or coded foreground audio object 59) may be extracted from bitstream 21 in the manner described above (132).

[0175] The audio decoding device 24 may further invoke an inverse quantization unit 74. Inverse quantization unit 74 may entropy decode and inverse quantize the coded foreground directivity information 57 to obtain reduced foreground directivity information 55 _k (136). Audio decoding device 24 may also call psychoacoustic decoding unit 80. The psychoacoustic audio decoding unit 80 uses the encoded environmental HOA coefficient 59 and the encoded foreground signal 61 to obtain the energy-compensated environmental HOA coefficient 47 'and the interpolated foreground signal 49'. It can be decoded (138). The psychoacoustic decoding unit 80 may pass the energy-compensated environmental HOA coefficient 47 ′ to the fade unit 770 and pass the nFG signal 49 ′ to the foreground organization unit 78.

[0176] The audio decoding device 24 may then invoke a spatiotemporal interpolation unit 76. The spatiotemporal interpolation unit 76 receives the reordered foreground directivity information 55 _k ′ and reduces the foreground directivity information 55 to generate interpolated foreground directivity information 55 _k ″. _A spatiotemporal interpolation may be performed for _k / 55 _k−1 (140). The spatiotemporal interpolation unit 76 may forward the interpolated foreground V [k] vector 55 _k ″ to the fade unit 770.

[0177] The audio decoding device 24 may invoke the fade unit 770. Fade unit 770 receives or otherwise obtains a syntax element (eg, from AmbCoeffTransition syntax element) that indicates when the energy compensated environmental HOA coefficient 47 'is in transition (eg, from extraction unit 72). obtain. Fade unit 770 fades in or out energy compensated environmental HOA coefficient 47 'based on the transition syntax element and the maintained transition state information, and adjusts adjusted environmental HOA coefficient 47''to HOA coefficient organization. Can be output to unit 82. The fade unit 770 also fades out or fades in the corresponding element or elements of the interpolated foreground V [k] vector 55 _k ″ based on the syntax elements and the maintained transition state information. Then, the adjusted foreground V [k] vector 55 _k ′ ″ may be output to the foreground organization unit 78 (142).

[0178] The audio decoding device 24 may invoke the foreground organization unit 78. Foreground organization unit 78 may perform matrix multiplication nFG signal 49 ′ with adjusted foreground directivity information 55 _k ″ ″ to obtain foreground HOA coefficient 65 (144). Audio decoding device 24 may also invoke HOA coefficient organization unit 82. The HOA coefficient knitting unit 82 may add the foreground HOA coefficient 65 to the adjusted environmental HOA coefficient 47 ″ to obtain the HOA coefficient 11 ′ (146).

  [0179] FIG. 7 is a flowchart illustrating an exemplary operation of a system, such as the system 10 shown in the example of FIG. 2, in performing various aspects of the techniques described in this disclosure. As discussed above, the content creator device 12 may use an audio editing system to create or edit captured or generated audio content (which is shown as the HOA coefficient 11 in the example of FIG. 2). 18 may be used. Content creator 12 may then render HOA coefficient 11 using audio renderer 1 on the generated multi-channel speaker feed, as discussed in more detail above (200). The content creator 22 may then play these speaker feeds using an audio playback system and, by way of example, determine whether further adjustments or edits are required to capture the desired artistic intent. (202). When further adjustment is desired (“YES” 202), the content creator 12 may remix the HOA coefficients (204), render the HOA coefficients (200), and determine whether further adjustment is required (202). ). If further adjustment is not desired (“NO” 202), the audio encoding device 20 may generate the bitstream 21 in the manner described above in connection with the example of FIG. 5 (206). Audio encoding device 20 may also generate and specify audio rendering information 2 in bitstream 21 as described in more detail above (208).

  [0180] The content consumer device 14 may then obtain audio rendering information 2 from the bitstream 21 (210). Decoding device 24 then decodes bitstream 21 to obtain audio content (which is shown as HOA coefficient 11 'in the example of FIG. 2) in the manner described above in connection with the example of FIG. obtain. The audio playback system 16 may then render the HOA coefficient 11 'based on the audio rendering information 2 in the manner described above (212) and play the audio content rendered via the loudspeaker 3 (214). .

  [0181] The techniques described in this disclosure may thus allow a device that generates a bitstream representing multi-channel audio content as a first example to specify audio rendering information. The device, in this first example, includes means for specifying audio rendering information including signal values that identify an audio renderer used when generating multi-channel audio content.

  [0182] The first example device, wherein the signal values include a matrix that is used to render the spherical harmonic coefficients into a plurality of speaker feeds.

  [0183] In the second example, the device of the first example, wherein the signal value indicates that the bitstream includes a matrix that is used to render spherical harmonics into multiple speaker feeds. Contains two or more bits that define an index.

  [0184] The device of the second example, wherein the audio rendering information further includes two or more bits that define the number of rows of the matrix included in the bitstream and the number of columns of the matrix included in the bitstream. A second example device comprising two or more bits defining.

  [0185] The first example device, wherein the signal value specifies a rendering algorithm that is used to render the audio object into a plurality of speaker feeds.

  [0186] In a first example device, the signal value specifies a rendering algorithm used to render spherical harmonic coefficients into a plurality of speaker feeds.

  [0187] The device of the first example, wherein the signal value is two or more bits defining an index associated with one of a plurality of matrices used to render the spherical harmonics into the plurality of speaker feeds. A device of a first example comprising:

  [0188] The device of the first example, wherein the signal value is two or more bits defining an index associated with one of a plurality of rendering algorithms used to render the audio object into a plurality of speaker feeds. A device of a first example comprising:

  [0189] The device of the first example, wherein the signal value defines two or more indices defining an index associated with one of a plurality of rendering algorithms used to render the spherical harmonic coefficients into a plurality of speaker feeds. The device of the first example, including bits.

  [0190] In the first example device, the means for specifying audio rendering information comprises means for specifying audio rendering information for each audio frame in the bitstream.

  [0191] The device of the first example, wherein the means for specifying audio rendering information comprises means for specifying audio rendering information once in the bitstream.

  [0192] In a third example, a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to specify the state of audio rendering in a bitstream. The information is a non-transitory computer readable storage medium that identifies an audio renderer that is used when generating multi-channel audio content.

  [0193] In a fourth example, a device for rendering multi-channel audio content from a bitstream, the device comprising a signal value identifying an audio renderer used when generating multi-channel audio content. A device comprising: means for determining audio rendering information to include; and means for rendering a plurality of speaker feeds based on audio rendering information specified in the bitstream.

  [0194] In a fourth example device, the signal value includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds is a matrix. A device of a fourth example comprising means for rendering a plurality of speaker feeds based on

  [0195] In a fifth example, the device of the fourth example, wherein the signal value is an index indicating that the bitstream includes a matrix used to render spherical harmonic coefficients into multiple speaker feeds Wherein the device further comprises means for analyzing the matrix from the bitstream in response to the index, the means for rendering the plurality of speaker feeds in the analyzed matrix A fourth example device comprising means for rendering a plurality of speaker feeds based thereon.

  [0196] The device of the fifth example, wherein the signal value further defines two or more bits defining the number of rows of the matrix included in the bitstream and the number of columns of the matrix included in the bitstream. Means for analyzing the matrix from the bitstream is responsive to the index, the two or more bits defining the number of rows and the two or more bits defining the number of columns A fifth example device comprising means for analyzing a matrix from a bitstream based on

  [0197] In a fourth example device, the signal value specifies a rendering algorithm used to render the audio object into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds comprises: The device of the fourth example comprising means for rendering a plurality of speaker feeds from an audio object using a specified rendering algorithm.

  [0198] In a fourth example device, the signal value specifies a rendering algorithm used to render the spherical harmonic coefficients into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds is A fourth example device comprising means for rendering a plurality of speaker feeds from spherical harmonics using a specified rendering algorithm.

  [0199] The device of the fourth example, wherein the signal value is two or more bits defining an index associated with one of a plurality of matrices used to render the spherical harmonics into the plurality of speaker feeds. And the means for rendering the plurality of speaker feeds comprises means for rendering the plurality of speaker feeds from the spherical harmonics using one of the plurality of matrices associated with the index. Devices.

  [0200] A fourth example device, wherein the signal value is two or more bits defining an index associated with one of a plurality of rendering algorithms used to render the audio object into a plurality of speaker feeds. And means for rendering a plurality of speaker feeds comprises means for rendering a plurality of speaker feeds from an audio object using one of a plurality of rendering algorithms associated with the index. Devices.

  [0201] The device of the fourth example, wherein the signal value defines two or more indices defining an index associated with one of a plurality of rendering algorithms used to render the spherical harmonic coefficients into a plurality of speaker feeds. The means for rendering a plurality of speaker feeds comprising bits comprises means for rendering the plurality of speaker feeds from the spherical harmonics using one of a plurality of rendering algorithms associated with the index. Example device.

  [0202] The device of the fourth example, wherein the means for determining audio rendering information includes means for determining audio rendering information for each audio frame from the bitstream.

  [0203] The device of the fourth example, wherein the means for determining the audio rendering information means includes determining the audio rendering information once from the bitstream.

  [0204] In a sixth example, when executed, causes one or more processors to determine audio rendering information including a signal value that identifies an audio renderer used when generating multi-channel audio content; A non-transitory computer readable storage medium having stored thereon instructions for rendering a plurality of speaker feeds based on audio rendering information specified in the bitstream.

  [0205] FIGS. 8A-8D are diagrams illustrating bitstreams 21A-21D formed in accordance with the techniques described in this disclosure. In the example of FIG. 8A, the bitstream 21A may represent one example of the bitstream 21 shown in FIGS. 2-4 above. The bitstream 21A includes audio rendering information 2A that includes one or more bits that define a signal value 554. This signal value 554 may represent any combination of the types of information described below. Bitstream 21A also includes audio content 558, which may represent one example of audio content 7/9.

  [0206] In the example of FIG. 8B, the bitstream 21B may be similar to the bitstream 21A, where the signal value 554 of the audio rendering information 2B defines an index 554A, the row size 554B of the signaled matrix 1 One or more bits, one or more bits defining a column size 554C of the signaled matrix, and a matrix coefficient 554D. Index 554A is defined using 2 to 5 bits, but each row size 554B and column size 554C may be defined using 2 to 16 bits.

  [0207] Extraction unit 72 may extract index 554A and determine whether the matrix is included in bitstream 21 or signaled (where an index value, such as 0000 or 1111, is determined by whether the matrix is a bit Signaling that it is explicitly specified in stream 21B). In the example of FIG. 8B, the bitstream 21B includes an index 554A that signals that a matrix is explicitly specified in the bitstream 21B. As a result, extraction unit 72 may extract row size 554B and column size 554C. The extraction unit 72 analyzes the row size 554B, the column size 554C and the number of bits to analyze representing the matrix coefficient as a function of the signaled (not shown in FIG. 8A) or implicit bit size of each matrix coefficient. May be configured to calculate Using the determined number of bits, extraction unit 72 may extract matrix coefficients 554D that may be used by audio playback system 16 to configure one of audio renderers 22 as described above. Although shown as signaling a single time of audio rendering information 2B in bitstream 21B, audio rendering information 2B is at least partially or completely in the bitstream or (in some cases optional data). Can be signaled multiple times in separate out-of-band channels.

  [0208] In the example of FIG. 8C, the bitstream 21C may represent one example of the bitstream 21 shown in FIGS. 2-4 above. The bitstream 21C includes audio rendering information 2C that includes a signal value 554 that specifies the algorithm index 554E in this example. The bitstream 21C also includes audio content 558. As described above, algorithm index 554E may be defined using 2-5 bits, and this algorithm index 554E is used when rendering audio content 558.

  [0209] Extraction unit 72 may extract algorithm index 550E and determine whether algorithm index 554E signals that a matrix is included in bitstream 21C (where an index such as 0000 or 1111). The value may signal that the matrix is explicitly specified in the bitstream 21). In the example of FIG. 8C, the bitstream 21C includes an algorithm index 554E that signals that a matrix is not explicitly specified in the bitstream 21C. As a result, extraction unit 72 may forward algorithm index 554E to audio playback device 16, which corresponds to the rendering algorithm (if available) (shown as renderer 22 in the examples of FIGS. 2-4). Select. Although shown as signaling audio rendering information 2C once in the bitstream 21C, in FIG. 8C, the audio rendering information 2C is at least partially in the bitstream 21C or (in some cases as optional data). Can be signaled multiple times in separate out-of-band channels.

  [0210] In the example of FIG. 8D, bitstream 21D may represent one example of bitstream 21 shown in FIGS. 2-4 above. Bitstream 21D includes audio rendering information 2D that includes signal value 554, which in this example specifies 55 matrix index 554F. The bitstream 21D also includes audio content 558. As described above, the matrix index 554F is defined using 2-5 bits, and this matrix index 554F may identify the rendering algorithm used when rendering the audio content 558.

  [0211] Extraction unit 72 extracts matrix index 550F and matrix index 554F signals that the matrix is included in bitstream 21D (where the matrix is explicitly specified in bitstream 21C). Can be determined whether an index value such as 0000 or 1111 can be signaled). In the example of FIG. 8D, the bitstream 21D includes a matrix index 554F that signals that the matrix is not explicitly specified in the bitstream 21D. As a result, the extraction unit 72 forwards the matrix index 554F to the audio playback device, which selects the corresponding one of the renderer 22 (if available). Although shown as signaling audio rendering information 2D once in bitstream 21D, audio rendering information 2D is at least partially or in bitstream 21D or (in some cases as optional data) or It can be signaled multiple times in a completely separate out-of-band channel.

  [0212] FIGS. 8E-8G are diagrams illustrating a portion of bitstream or side channel information that may specify compressed spatial components in more detail. FIG. 8E shows a first example of the frame 249 </ b> A ′ of the bitstream 21. In the example of FIG. 8E, the frame 249A 'includes a ChannelSideInfoData (CSID) field 154A-154C, a HOAGainCollectionData (HOAGCD) field, and VVectorData fields 156A and 156B. CSID field 154A includes unitC 267, bb 266, and ba265 along with ChannelType 269. Each of them is set to the corresponding value 01, 1, 0, and 01 shown in the example of FIG. 8E. The CSID field 154B includes unitC 267, bb 266, and ba265 along with ChannelType 269. Each of them is set to the corresponding value 01, 1, 0, and 01 shown in the example of FIG. 8E. CSID field 154C includes a ChannelType field 269 having a value of 3. Each of the CSID fields 154A-154C corresponds to a respective one of transport channels 1, 2, and 3. In effect, each CSID field 154A-154C has a corresponding payload 156A and 156B that is a direction-based signal (when the corresponding ChannelType is equal to 0) or a vector-based signal (when the corresponding ChannelType is equal to 1), Indicates an additional environmental HOA factor (when the corresponding ChannelType is equal to 2) or empty (when the ChannelType is equal to 3).

  [0213] In the example of FIG. 8E, frame 249A is provided with two vector-based signals (given ChannelType syntax element 269 equal to 1 in CSID fields 154A and 154B) and ChannelType 269 (given 3 in CSID field 154C). Included) and empty. Based on the above-described HOAconfig portion (not shown for ease of explanation), audio decoding device 24 may determine all 16V vector elements. Thus, each of VVectorData 156A and 156B includes all 16 vector elements, each of which is uniformly quantized with 8 bits.

  [0214] As further shown in the example of FIG. 8E, frame 249A 'does not include a HOAPPredictionInfo field. The HOAP predictionInfo field may represent a field corresponding to a direction-based compression scheme that may be removed according to the techniques described in this disclosure when the vector-based compression scheme compresses HOA audio data.

  [0215] FIG. 8F is a diagram illustrating a frame 249 "that is substantially similar to frame 249A, except that HOAGainCollectionData has been removed from each transport channel stored in field 249". The HOAGainCollectionData field may be removed from frame 249 '' when gain correction is suppressed in accordance with various aspects of the techniques described above.

  [0216] FIG. 8G may be similar to frame 249A "" that is similar to frame 249A "except that the HOAPPredictionInfo field is removed. Frame 249A "" represents one example where it can be applied to a combination to remove various fields that are not needed in an environment where both aspects of the techniques are present.

  [0217] The above techniques may be performed for any number of different situations and audio ecosystems. Some example situations are described below, but the technique should not be limited to those example situations. One exemplary audio ecosystem includes audio content, movie studios, music studios, gaming audio studios, channel-based audio content, coding engines, game audio stems, game audio coding / rendering engines, Distribution system.

  [0218] Movie studios, music studios, and gaming audio studios may receive audio content. In some examples, the audio content may represent an output of the acquisition. Movie studios can output channel-based audio content (eg, 2.0, 5.1, and 7.1), such as by using a digital audio workstation (DAW). The music studio can output channel-based audio content (eg, 2.0 and 5.1), such as by using a DAW. In any case, the coding engine uses one or more channel-based audio content-based codecs (eg, AAC, AC3, Dolby True HD, Dolby Digital Plus, and DTS Master Audio) for output by the distribution system. Can be received and encoded. A gaming audio studio can output one or more gaming audio stems, such as by using a DAW. The game audio coding / rendering engine can code and / or render the audio stem into channel-based audio content for output by the distribution system. Another exemplary situation in which the technique may be implemented includes broadcast recording audio objects, professional audio systems, consumer on-device capture, HOA audio formats, on-device rendering, consumer audio, and TV. And an audio ecosystem that may include accessories and a car audio system.

  [0219] Broadcast recording audio objects, professional audio systems, and consumer on-device captures can all use their HOA audio format to code their output. In this way, audio content is coded using the HOA audio format into a single representation that can be played using on-device rendering, consumer audio, TV and accessories, and car audio systems. obtain. In other words, a single representation of audio content is in contrast to requiring a specific configuration such as 5.1, 7.1, etc. in a general audio playback system, such as audio playback system 16. ) Can be played.

  [0220] Another example of a situation in which this technique may be implemented is an audio ecosystem that may include an acquisition element and a playback element. Acquisition elements may include wired and / or wireless acquisition devices (eg, Eigen microphones), on-device surround sound capture, and mobile devices (eg, smartphones and tablets). In some examples, the wired and / or wireless acquisition device may be coupled to the mobile device via a wired and / or wireless communication channel.

  [0221] In accordance with one or more techniques of this disclosure, a mobile device may be used to acquire a sound field. For example, a mobile device can acquire a sound field via wired and / or wireless acquisition devices and / or on-device surround sound capture (eg, multiple microphones integrated with the mobile device). The mobile device can then code the acquired sound field into HOA coefficients for playback by one or more of the playback elements. For example, a mobile device user can record a live event (eg, a meeting, conference, play, concert, etc.) (acquire its sound field) and code the recording into a HOA coefficient.

  [0222] The mobile device may also utilize one or more of the playback elements to play the HOA coded sound field. For example, a mobile device can decode a HOA-coded sound field and output a signal to one or more of the playback elements, which causes the sound field to be transmitted to one or more of the playback elements. Let it be recreated. As an example, a mobile device can utilize wireless and / or wireless communication channels to output signals to one or more speakers (eg, speaker arrays, sound bars, etc.). As another example, a mobile device outputs a signal to one or more docking stations and / or one or more docked speakers (eg, a sound system in a smart car and / or smart home). In addition, a docking solution can be used. As another example, a mobile device can utilize headphone rendering to output a signal to a set of headphones, eg, to create a realistic binaural sound.

  [0223] In some examples, a particular mobile device can both acquire a 3D sound field and play the same 3D sound field at a later time. In some examples, the mobile device acquires a 3D sound field, encodes the 3D sound field into a HOA, and encodes the 3D sound field for playback on one or more other devices (eg, Other mobile devices and / or other non-mobile devices).

  [0224] Another situation in which the present techniques may be performed includes an audio ecosystem that may include audio content, game studios, coded audio content, rendering engines, and distribution systems. In some examples, the game studio may include one or more DAWs that may support editing of the HOA signal. For example, the one or more DAWs may include HOA plug-ins and / or tools that may be configured to operate (eg, function) with one or more gaming audio systems. In some examples, the game studio can output a new stem format that supports HOA. In either case, the game studio can output the coded audio content to a rendering engine that can render the sound field for playback by the distribution system.

  [0225] The techniques may also be performed for an exemplary audio acquisition device. For example, the techniques may be performed on an Eigen microphone that may include multiple microphones configured together to record a 3D sound field. In some examples, multiple microphones of an Eigen microphone can be placed on the surface of a substantially spherical sphere with a radius of about 4 cm. In some examples, the audio encoding device 20 may be integrated into an Eigen microphone to output a bitstream 21 directly from the microphone.

  [0226] Another exemplary audio acquisition situation may include a production track that may be configured to receive signals from one or more microphones, such as one or more Eigen microphones. The production track may also include an audio encoder, such as the audio encoder 20 of FIG.

  [0227] The mobile device may also include a plurality of microphones that are configured together to record a 3D sound field in some cases. In other words, the plurality of microphones may have X, Y, Z diversity. In some examples, the mobile device may include a microphone that can be rotated to provide X, Y, Z diversity with respect to one or more other microphones of the mobile device. The mobile device may also include an audio encoder, such as audio encoder 20 of FIG.

  [0228] The impact resistant video capture device may be further configured to record a 3D sound field. In some examples, an impact resistant video capture device may be attached to a user's helmet involved in certain activities. For example, an impact resistant video capture device may be attached to a user's helmet that is torrenting. In this way, the impact-resistant video capture device captures a 3D sound field that represents all activities around the user (eg, water squeezing behind the user, another occupant talking in front of the user, etc.) can do.

  [0229] The techniques may also be performed on accessory-enhanced mobile devices that may be configured to record 3D sound fields. In some examples, the mobile device may be similar to the mobile device described above, but with one or more accessories added. For example, an Eigen microphone can be attached to the mobile device described above to form an accessory enhanced mobile device. In this way, an accessory-enhanced mobile device can capture a higher quality version of the 3D sound field than just using a sound capture component that is integral with the accessory-enhanced mobile device.

  [0230] Exemplary audio playback devices that can perform various aspects of the techniques described in this disclosure are further described below. According to one or more techniques of this disclosure, the speakers and / or soundbar may be arranged in any arbitrary configuration, while still playing a 3D sound field. Moreover, in some examples, a headphone playback device may be coupled to the decoder 24 via either a wired connection or a wireless connection. In accordance with one or more techniques of this disclosure, a single generic representation of a sound field may be utilized to render the sound field with any combination of speakers, sound bars, and headphone playback devices.

  [0231] Several different exemplary audio playback environments may also be suitable for performing various aspects of the techniques described in this disclosure. For example, 5.1 speaker playback environment, 2.0 (eg, stereo) speaker playback environment, 9.1 speaker playback environment with full height front loudspeaker, 22.2 speaker playback environment, 16.0 speaker playback environment, car speaker A playback environment, and a mobile device with an earbud playback environment may be a suitable environment for performing various aspects of the techniques described in this disclosure.

  [0232] According to one or more techniques of this disclosure, a single generic representation of a sound field may be utilized to render the sound field in any of the playback environments described above. In addition, the techniques of this disclosure allow a render to render a sound field from a generic representation for playback in playback environments other than those described above. For example, if design considerations prevent proper placement of speakers according to a 7.1 speaker playback environment (eg, where it is not possible to place right surround speakers), Allows the render to compensate with the other 6 speakers so that playback can be achieved in a 6.1 speaker playback environment.

  [0233] In addition, the user can watch sports matches while wearing headphones. In accordance with one or more techniques of this disclosure, a 3D sound field of a sports game may be obtained (eg, one or more Eigen microphones may be placed in and / or around a baseball field), 3D HOA coefficients corresponding to the sound field can be obtained and transmitted to the decoder, which can reconstruct the 3D sound field based on the HOA coefficients and output the reconstructed 3D sound field to the renderer; The renderer can obtain instructions about the type of playback environment (eg, headphones) and render the reconstructed 3D sound field into a signal that causes the headphones to output a 3D sound field representation of the sports game.

  [0234] In each of the various cases described above, the audio encoding device 20 performs a method, or else each step of the method that the audio encoding device 20 is configured to perform. It should be understood that means for performing can be provided. In some cases, these means may comprise one or more processors. In some cases, the one or more processors may represent a dedicated processor configured with instructions stored on a non-transitory computer readable storage medium. In other words, various aspects of the techniques in each of the example set of encodings, when executed, perform one or more processors in a method that the audio encoding device 20 is configured to execute. A non-transitory computer readable storage medium storing instructions to be stored may be provided.

  [0235] In one or more examples, the functions described above may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer readable medium may include a computer readable storage medium corresponding to a tangible medium such as a data storage medium. A data storage medium may be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. It can be a possible medium. The computer program product may include a computer readable medium.

  [0236] Similarly, in each of the various cases described above, the audio decoding device 24 performs a method, or else each of the methods that the audio decoding device 24 is configured to perform. It should be understood that means may be provided for performing the steps. In some cases, these means may comprise one or more processors. In some cases, the one or more processors may represent a dedicated processor configured with instructions stored on a non-transitory computer readable storage medium. In other words, various aspects of the techniques in each of the example set of encodings, when executed, cause one or more processors to perform a method that the audio decoding device 24 is configured to perform. A non-transitory computer readable storage medium storing instructions may be provided.

  [0237] By way of example, and not limitation, such computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage device, flash memory Or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. However, it should be understood that computer-readable storage media and data storage media are directed to non-transitory tangible storage media, rather than including connections, carrier waves, signals, or other temporary media. As used herein, a disk and a disc are a compact disc (CD), a laser disc (registered trademark) (disc), an optical disc (disc), a digital versatile disc (DVD). ), Floppy disk, and Blu-ray disk, where the disk typically reproduces data magnetically, while the disk ) Reproduce the data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media.

  [0238] The instructions may be one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated circuits or discrete logic circuits. Can be executed by one or more processors such as. Thus, as used herein, the term “processor” can refer to either the structure described above or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functions described herein may be configured in a dedicated hardware module and / or software module that is configured for encoding and decoding or embedded in a composite codec. Can be provided. Also, the techniques may be fully implemented with one or more circuits or logic elements.

  [0239] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (eg, a chip set). Although this disclosure describes various components, modules, or units to highlight functional aspects of a device configured to perform the disclosed techniques, those components, modules, or units Are not necessarily realized by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit, including one or more processors as described above, or interworking hardware, with suitable software and / or firmware. It can be given by a set of units.

[0240] Various aspects of the disclosure have been described. These and other aspects of the technique fall within the scope of the following claims.
Hereinafter, the invention described in the scope of claims of the present application will be appended.
[C1]
A device configured to render higher order ambisonic coefficients,
One or more processors configured to obtain code symmetry information indicative of code symmetry of a matrix used to render the higher order ambisonic coefficients to generate a plurality of speaker feeds;
A memory configured to store the sparseness information;
A device comprising:
[C2]
The one or more processors are used to determine value symmetry information indicative of value symmetry of the matrix and to represent the matrix based on the value symmetry information and the code symmetry information. Further configured to determine the number of bits obtained,
The device according to C1.
[C3]
The one or more processors are used to determine sparseness information indicative of sparseness of the matrix and to represent the matrix based on the sparseness information and the code symmetry information. Further configured to determine the number of bits obtained,
The device according to C1.
[C4]
The one or more processors are further configured to determine a speaker layout in which the matrix is used to render the plurality of speaker feeds from the higher order ambisonic coefficients.
The device according to C1.
[C5]
A speaker configured to reproduce a sound field represented by the higher order ambisonic coefficient based on the plurality of speaker feeds;
The device according to C1.
[C6]
The one or more processors obtain audio rendering information indicating signal values that identify audio renderers used when generating the plurality of speaker feeds; and the plurality of processors based on the audio rendering information Further configured to render a speaker feed;
The device according to C1.
[C7]
The signal values include the matrix used to render the higher order ambisonic coefficients in the plurality of speaker feeds;
The one or more processors are configured to render the plurality of speaker feeds based on the matrix included in the signal values.
The device according to C6.
[C8]
A method for rendering higher order ambisonic coefficients,
Obtaining code symmetry information indicative of code symmetry of a matrix used to render the higher order ambisonic coefficients to generate a plurality of speaker feeds;
Method.
[C9]
Determining value symmetry information indicative of value symmetry of the matrix;
Determining a reduced number of bits used to represent the matrix based on the value symmetry information and the code symmetry information;
The method of C8, further comprising:
[C10]
Determining sparseness information indicative of sparseness of the matrix;
Determining a reduced number of bits used to represent the matrix based on the sparseness information and the code symmetry information;
The method of C8, further comprising:
[C11]
Further comprising determining a speaker layout used by the matrix to render the multi-channel audio data from the higher order ambisonic coefficients.
The method according to C8.
[C12]
Replaying a sound field represented by the higher order ambisonic coefficient based on the plurality of speaker feeds;
The method according to C8.
[C13]
Obtaining audio rendering information indicative of a signal value identifying an audio renderer used when generating the plurality of speaker feeds;
Rendering the plurality of speaker feeds based on the audio rendering information;
The method of C8, further comprising:
[C14]
The signal value includes the matrix used to render the higher order ambisonic coefficients to generate the plurality of speaker feeds, and the method is based on the matrix included in the signal value. Rendering the plurality of speaker feeds;
The method according to C13.
[C15]
A device configured to create a bitstream,
A memory configured to store a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds;
One or more processors configured in code symmetry information indicating code symmetry of the matrix;
A device comprising:
[C16]
The one or more processors determine value symmetry information indicating value symmetry of the matrix, and reduce the number of bits indicating the matrix based on the value symmetry information and the code symmetry information; Further configured to do the
The device according to C15.
[C17]
The one or more processors determine code symmetry information indicating code symmetry of the matrix; and reducing the number of bits indicating the matrix based on the code symmetry information and the code symmetry information;
The device of C15, further configured to:
[C18]
The one or more processors are further configured to determine a speaker layout in which the matrix is used to render the plurality of speaker feeds from the higher order ambisonic coefficients.
The device according to C15.
[C19]
Further comprising a microphone configured to capture a sound field represented by the higher order ambisonic coefficient;
The device according to C15.
[C20]
A method of creating a bitstream,
Obtaining sparseness information indicative of sparseness of a matrix used to render higher order ambisonic coefficients that generate multiple speaker feeds;
Method.
[C21]
Determining value symmetry information indicative of value symmetry of the matrix;
Reducing the number of bits indicating the matrix based on the value symmetry information and the code symmetry information;
The method of C20, further comprising.
[C22]
Determining code symmetry information indicating code symmetry of the matrix, and reducing the number of bits indicating the matrix based on the code symmetry information and the code symmetry information;
The method of C20, further comprising:
[C23]
The matrix further comprising: determining a speaker layout used to render the plurality of multi-channel audio data from the higher order ambisonic coefficients;
The method according to C20.
[C24]
Further comprising capturing a sound field represented by the higher order ambisonic coefficients.
The method according to C20.

Claims (15)

A device configured to render higher order ambisonic coefficients,
Sparseness information indicating the sparseness of the matrix used to render the higher order ambisonic coefficients to generate a plurality of speaker feeds from a bitstream containing an encoded version of the higher order ambisonic coefficients Getting,
Obtaining code symmetry information indicating code symmetry of the matrix from the bitstream;
Obtaining a reduced number of bits used to represent the matrix from the bitstream;
One or more processors configured to: reconstruct the matrix based on the sparse information, the code symmetry information, and the reduced number of bits;
And a memory coupled to the one or more processors and configured to store the sparse information. The one or more processors are further configured to determine a speaker layout in which the matrix is used to render the plurality of speaker feeds from the higher order ambisonic coefficients.
The device of claim 1. A speaker configured to reproduce a sound field represented by the higher order ambisonic coefficient based on the plurality of speaker feeds;
The device of claim 1. The one or more processors obtain audio rendering information from the bitstream indicating signal values identifying audio renderers used when generating the plurality of speaker feeds, and based on the audio rendering information Further configured to render the plurality of speaker feeds;
The device of claim 1. The signal value includes an index associated with the matrix used to render the higher order ambisonic coefficients in the plurality of speaker feeds;
The one or more processors are configured to render the plurality of speaker feeds based on the matrix associated with the index included in the signal value.
The device of claim 4. A method for rendering higher order ambisonic coefficients,
Sparseness information indicating the sparseness of the matrix used to render the higher order ambisonic coefficients to generate a plurality of speaker feeds from a bitstream containing an encoded version of the higher order ambisonic coefficients Getting,
Obtaining code symmetry information indicating code symmetry of the matrix from the bitstream;
Obtaining a reduced number of bits used to represent the matrix from the bitstream;
Reconstructing the matrix based on the sparseness information, the code symmetry information, and the reduced number of bits. The matrix further comprising: determining a speaker layout used to render multi-channel audio data from the higher order ambisonic coefficients;
The method of claim 6. Replaying a sound field represented by the higher order ambisonic coefficient based on the plurality of speaker feeds;
The method of claim 6. Obtaining audio rendering information from the bitstream indicating a signal value identifying an audio renderer used when generating the plurality of speaker feeds;
The method of claim 6, further comprising rendering the plurality of speaker feeds based on the audio rendering information. The signal value includes an index associated with the matrix used to render the higher order ambisonic coefficients to generate the plurality of speaker feeds;
The method further comprises rendering the plurality of speaker feeds based on the matrix associated with the index included in the signal value.
The method of claim 9. A device configured to create a bitstream,
A memory configured to store a matrix used to render higher order ambisonic coefficients to generate a plurality of speaker feeds;
Coupled to the memory,
Obtaining code symmetry information indicating code symmetry of the matrix;
Obtaining sparse information indicating the sparseness of the matrix;
Determining a reduced number of bits used to represent the matrix based on the code symmetry information and the sparseness information;
Generating the bitstream to include a coded version of the higher order ambisonic coefficients, the code symmetry information, the sparseness information, and the reduced number of bits 1 A device comprising one or more processors. The one or more processors are further configured to determine a speaker layout in which the matrix is used to render the plurality of speaker feeds from the higher order ambisonic coefficients.
The device of claim 11. Further comprising a microphone configured to capture a sound field represented by the higher order ambisonic coefficient;
The device of claim 11. A method of creating a bitstream,
Obtaining sparseness information indicating the sparseness of the matrix used to render the higher-order ambisonic coefficients that generate multiple speaker feeds;
Obtaining code symmetry information indicating code symmetry of the matrix;
Determining a reduced number of bits used to represent the matrix based on the code symmetry information and the sparseness information;
Generating the bitstream to include a coded version of the higher order ambisonic coefficients, the code symmetry information, the sparseness information, and the reduced number of bits. The matrix further comprising: determining a speaker layout used to render multi-channel audio data from the higher order ambisonic coefficients;
The method according to claim 14.