KR20170015898A - Obtaining symmetry information for higher order ambisonic audio renderers - Google Patents

Abstract

Generally, techniques for obtaining audio rendering information in a bitstream are described. A device configured to render higher order ambience coefficients, including a processor and a memory, may perform techniques. The processor may be configured to obtain sign symmetry information indicative of the sign symmetry of the matrix used to render the high order ambience coefficients to produce a plurality of speaker feeds. The memory may be configured to store scarcity information.

Description

OBTAINING SYMMETRY INFORMATION FOR HIGHER ORDER AMBISONIC AUDIO RENDERERS [0001]

This application claims the benefit of U.S. Provisional Application No. 62 / 023,662 , filed July 11, 2014, entitled " SIGNALING AUDIO RENDERING INFORMATION IN A BITSTREAM & U.S. Provisional Application No. 62 / 005,829 , filed May 30, 2014, the entire contents of each of the foregoing U.S. Provisional Patent Applications are hereby incorporated herein by reference in their entirety as if each of these were all expressly set forth herein.

The present disclosure relates to rendering information, and more particularly, to rendering information about higher-order ambisonic (HOA) audio data.

During the generation of the audio content, the sound engineer may render the audio content using a particular renderer in an attempt to match the audio content to the target configurations of the speakers used to play the audio content. In other words, the sound engineer may use the speakers arranged in the targeted configuration to render the audio content and play the rendered audio content. The sound engineer can then use the speakers arranged in the targeted configuration to remix various aspects of the audio content, render the remixed audio content, and replay the rendered, remixed audio content. The sound engineer may repeat this way until a given artistic intention is provided by the audio content. In this way, the sound engineer may provide some artistic intention during playback (e.g., to accompany the video content played with the audio content), or may generate audio content that provides a predetermined sound field It is possible.

Generally, techniques for specifying audio rendering information in a bitstream representing audio data are described. In other words, these techniques may provide a way to signal audio rendering information used during audio content generation to a playback device, which may then render the audio content using audio rendering information. Providing the rendering information in this manner enables the playback device to render the audio content in a manner intended by the sound engineer, thereby enabling the proper reproduction of the audio content to be potentially understood by the listener . In other words, the rendering information used during rendering by the sound engineer is provided in accordance with the techniques described in this disclosure so that the audio reproduction device utilizes the rendering information to render the audio content in a manner intended by the sound engineer So as to ensure a more consistent experience for both creation and playback of audio content as compared to systems that do not provide this audio rendering information.

In one aspect, a device configured to render high order ambsonic coefficients comprises one or more devices configured to obtain sparseness information indicative of sparseness of a matrix used to render high order ambience coefficients into a plurality of speaker feeds, Processors, and memory configured to store scarcity information.

In another aspect, a method of rendering higher order ambience coefficients comprises obtaining scarcity information indicative of the scarcity of a matrix used to render higher order ambience coefficients to produce a plurality of speaker feeds.

In another aspect, a device configured to generate a bitstream comprises a memory configured to store a matrix, and one configured to obtain scarcity information indicative of the scarcity of a matrix used to render high order ambience coefficients to generate a plurality of speaker feeds, ≪ / RTI >

In another aspect, a method of generating a bitstream includes obtaining scarcity information indicative of the scarcity of a matrix used to render high order ambience coefficients to produce a plurality of speaker feeds.

In another aspect, a device configured to render higher order ambience coefficients comprises one or more processors configured to obtain sign symmetry information indicative of a sign symmetry of a matrix used to render higher order ambience coefficients to generate a plurality of speaker feeds, And a memory configured to store scarcity information.

In another aspect, a method of rendering higher order ambience coefficients includes obtaining sign symmetry information indicative of the sign symmetry of a matrix used to render higher order ambience coefficients to produce a plurality of speaker feeds.

In another aspect, a device configured to generate a bitstream comprises a memory configured to store a matrix used to render a high order ambience coefficient to generate a plurality of speaker feeds, and a memory configured to store the code symmetry information indicative of the symmetry of the matrix One or more processors.

In another aspect, a method of generating a bitstream includes obtaining scarcity information indicative of the scarcity of a matrix used to render high order ambience coefficients to produce a plurality of speaker feeds.

Details of one or more aspects of the techniques are set forth in the following description and the accompanying drawings. Other features, objects, and advantages of the techniques will be apparent from the description and drawings, and from the claims.

Figure 1 is a diagram illustrating spherical harmonic basis functions of various orders and sub-orders.
Figure 2 is a diagram illustrating a system that may perform various aspects of the techniques described in this disclosure.
Figure 3 is a block diagram illustrating in more detail one example of an audio encoding device shown in the example of Figure 2, which may perform various aspects of the techniques described in this disclosure.
4 is a block diagram illustrating the audio decoding device of FIG. 2 in greater detail.
5 is a flow chart illustrating an exemplary operation of an audio encoding device in performing various aspects of vector-based synthesis techniques described in this disclosure.
6 is a flow chart illustrating an exemplary operation of an audio decoding device in performing various aspects of the techniques described in this disclosure.
Figure 7 is a flow chart illustrating exemplary operation of a system, such as the one system shown in the example of Figure 2, in performing various aspects of the techniques described in this disclosure.
8A-8D are diagrams illustrating bitstreams formed in accordance with the techniques described in this disclosure.
8E-8G are diagrams illustrating portions of bitstream or side channel information that may specify compressed spatial components in more detail.
9 is a diagram illustrating an example of HOA order-dependent minimum and maximum gains in a higher order ambience sonic (HOA) rendering matrix.
10 is a diagram illustrating a partial scarce sixth order HOA rendering matrix for twenty-two loudspeakers.
11 is a flow diagram illustrating signaling of symmetric attributes.

Nowadays, the evolution of surround sound has made many output formats available for entertainment. Examples of such consumer surround sound formats are mostly 'channel based' in that they implicitly specify feeds to loudspeakers at certain geometric coordinates. Consumer surround sound formats include the popular 5.1 format (the following six channels: front left (FL), front right (FR), center or front center, rear left or surround left, rear right or surround right, and low frequency effects low frequency effects (LFE)), a growing 7.1 format, height speakers such as 22.2 format and 7.1.4 format (e.g., for use as an ultra high definition television standard) . Non-consumer formats may span any number of speakers, often referred to as " surround arrays " (in symmetric and asymmetric geometries). One example of such an array includes 32 loudspeakers positioned at the coordinates on the corners of the truncated icosahedron.

Inputs to future MPEG encoders are optionally one of three possible formats: (i) traditional channel-based audio (as discussed above) intended to be played through loudspeakers at pre-specified positions; (ii) object-based audio with discrete pulse-code-modulation (PCM) data for single audio objects with associated metadata including positional coordinates (among other information); And (iii) coefficients of spherical harmonic basis functions (also referred to as "spherical harmonic coefficients" or SHC, "Higher-order Ambisonics" or HOA, and "HOA Quot; coefficients ") of a scene-based audio. Future MPEG encoders will be published in Geneva, Switzerland, in January 2013, and available at http://mpeg.chiariglione.org/sites/default/files/files/standards/parts/docs/w13411.zip . May be further described in the document entitled " Call for Proposals for 3D Audio " by the International Electrotechnical Commission (ISO) / (IEC) JTC1 / SC29 / WG11 / N13411.

There are various 'surround-sound' channel-based formats in the market. These range from, for example, the 5.1 home theater system (which has been most successful in terms of moving beyond the stereo to the living room) to the 22.2 system developed by NHK (Nippon Hoso Kyokai or Japanese broadcaster). Content creators (for example, Hollywood studios) want to make a soundtrack for a movie once and do not try to remix it for each speaker configuration. Recently, standards development organizations have provided adaptive and unrestricted subsequent decoding for speaker geometry (and number) and acoustic conditions at the location of encoding and playback (with a renderer) into a standardized bitstream I was considering ways to do that.

To provide this flexibility for content creators, a hierarchical set of elements may be used to represent the sound field. A hierarchical set of elements may refer to a set of elements from which elements are ordered such that a basic set of lower order elements provides an overall representation of the modeled sound field. As the set is expanded to include higher order elements, the representation is further refined to increase resolution.

One example of a hierarchical set of elements is a set of spherical harmonic coefficients (SHC). The following equation uses SHC to describe the sound field description or representation:

에서의 압력

가 SHC,

에 의해 고유하게 표현될 수 있음을 나타낸다. 여기서,

이고, c 는 사운드의 속도 (~343 m/s) 이고,

은 참조의 포인트 (또는 관측 포인트) 이고,

은 차수 (order) n 의 구면 베셀 함수이며,

은 차수 n 및 하위차수 (sub-order) m 의 구면 조화 기저 함수들이다. 대괄호들에서의 용어는 이산 푸리에 변환 (DFT), 이산 코사인 변환 (DCT), 또는 웨이블릿 변환과 같은 다양한 시간-주파수 변환들에 의해 근사화될 수 있는 신호의 주파수-도메인 표현 (즉,

) 인 것이 인지될 수 있다. 계층적 세트들의 다른 예들은 웨이블릿 변환 계수들의 세트들 및 다해상도 기저 함수들의 계수들의 다른 세트들을 포함한다.This equation indicates that at any point in the sound field at time t

Pressure in

SHC,

≪ / RTI > here,

, C is the speed of sound (~ 343 m / s)

Is the point of reference (or observation point)

Is a spherical Bessel function of order n ,

Are the spherical harmonic basis functions of order n and sub-order m . The terms in brackets denote the frequency-domain representation of the signal that can be approximated by various time-frequency transforms such as discrete Fourier transform (DFT), discrete cosine transform (DCT), or wavelet transform

). ≪ / RTI > Other examples of hierarchical sets include sets of wavelet transform coefficients and other sets of coefficients of multi-resolution basis functions.

1 is a diagram illustrating spherical harmonic basis functions from a zero order ( n = 0) to a fourth order ( n = 4). As can be seen, for each order, there is an extension of the lower orders m , which is shown in the example of FIG. 1 but is not explicitly mentioned for ease of illustration purposes.

는 다양한 마이크로폰 어레이 구성들에 의해 물리적으로 포착 (예를 들어, 레코딩) 될 수 있거나 또는, 대안적으로, 음장의 채널-기반 또는 오브젝트-기반 디스크립션들로부터 도출될 수 있다. SHC 는 장면-기반 오디오를 표현하는데, 여기서 SHC 는 더 효율적인 송신 또는 저장을 촉진할 수도 있는 인코딩된 SHC 를 획득하기 위해 오디오 인코더에 입력될 수도 있다. 예를 들어, (1+4)² (25, 그리고 그에 따라 제 4 차수) 계수들을 수반하는 제 4 차수 표현이 이용될 수도 있다.SHC

May be physically captured (e.g., recorded) by various microphone array configurations, or alternatively, may be derived from channel-based or object-based descriptions of the sound field. The SHC represents scene-based audio, where the SHC may be input to an audio encoder to obtain an encoded SHC that may facilitate more efficient transmission or storage. For example, a fourth order expression involving (1 + 4) ² (25, and hence fourth order) coefficients may be used.

As noted above, the SHC may be derived from microphone recording using a microphone array. Various examples of how SHCs may be derived from microphone arrays are described in Poletti, M., "Three-Dimensional Surround Sound Systems Based on Spherical Harmonics ", J. Audio Eng. Soc., Vol. 53, No. 11, November 2005, pp. 1004-1025. &Quot;

는 다음과 같이 표현될 수도 있고:To illustrate how SHCs may be derived from an object-based description, consider the following equations. The coefficients for the sound field corresponding to the individual audio object

May be expressed as: < RTI ID = 0.0 >

이고,

은 차수 n 의 (제 2 종의) 구면 핸켈 함수이고,

는 오브젝트의 위치이다. (예를 들어, PCM 스트림에 대해 고속 푸리에 변환을 수행하는 것과 같은 시간-주파수 분석 기법들을 이용하여) 주파수의 함수로서 오브젝트 소스 에너지

를 아는 것은 각각의 PCM 오브젝트 및 대응하는 위치를 SHC,

로 컨버팅하게 한다. 추가로, 이것은 (상기가 선형 및 직교 분해이기 때문에) 각각의 오브젝트에 대한

계수들이 가산적임을 나타낼 수 있다. 이러한 방식으로, 다수의 PCM 오브젝트들은 (예를 들어, 개별 오브젝트들에 대한 계수 벡터들의 합으로서)

계수들에 의해 표현될 수 있다. 본질적으로, 계수들은 음장에 관한 정보 (3D 좌표들의 함수로서의 압력) 를 포함하고, 상기는, 관측 포인트

의 부근에서, 개별 오브젝트들로부터 전체 음장의 표현으로의 변환을 표현한다. 나머지 도면들은 오브젝트-기반 및 SHC-기반 오디오 코딩의 맥락에서 아래에 설명된다.Where i is

ego,

Is a spherical Hankel function of degree n (second kind)

Is the position of the object. (E. G., Using time-frequency analysis techniques such as performing a fast Fourier transform on the PCM stream)

Knowing that each PCM object and its corresponding location is SHC,

. In addition, it is possible for each object (since it is linear and orthogonal decomposition)

The coefficients may be additive. In this way, multiple PCM objects (e.g., as the sum of the coefficient vectors for individual objects)

Can be expressed by coefficients. In essence, the coefficients comprise information about the sound field (pressure as a function of 3D coordinates), which,

, The conversion from individual objects to the representation of the entire sound field is expressed. The remaining figures are described below in the context of object-based and SHC-based audio coding.

FIG. 2 is a diagram illustrating a system 10 that may perform various aspects of the techniques described in this disclosure. As shown in the example of FIG. 2, the system 10 includes a content creator device 12 and a content consumer device 14. Although illustrated in the context of the content creator device 12 and the content consumer device 14, the techniques may be implemented in SHCs (which may also be referred to as HOA coefficients) or any other hierarchical representation of the sound field Or may be implemented in any context where it is encoded to form a bitstream. Moreover, the content creator device 12 may be any form of device capable of implementing the techniques described in this disclosure, including a handset (or cellular phone), a tablet computer, a smartphone, or a desktop computer, May represent a computing device. Likewise, content consumer device 14 may be capable of implementing the techniques described in this disclosure, including a handset (or cellular phone), a tablet computer, a smartphone, a set-top box, Or may represent any type of computing device.

Content creator device 12 may be operated by a movie studio or other entity that may generate multi-channel audio content for consumption by operators of content consumer devices, such as content consumer device 14. [ In some instances, the content creator device 12 may be operated by an individual user who wishes to compress the HOA coefficients 11. Often, content creators generate audio content along with video content. The content consumer device 14 may be operated by an individual. Content consumer device 14 may include an audio playback system 16 that may refer to any form of audio playback system capable of rendering SHC for playback as multi-channel audio content.

The content creator device 12 includes an audio editing system 18. The content creator device 12 includes audio objects 9 that the content creator device 12 may edit using the audio editing system 18 and live recordings of various formats (including directly as HOA coefficients) 7). The microphone 5 may capture live recordings 7. The content creator can also render the HOA coefficients 11 from the audio objects 9 during the editing process and listen to the rendered speaker feeds in an attempt to identify various aspects of the sound field requiring further editing have. Thereafter, the content creator device 12 generates HOA coefficients 11 (potentially indirectly through the manipulation of different audio objects among the audio objects 9 that may cause the source HOA coefficients to be derived in the manner described above) You can edit it. The content creator device 12 may employ the audio editing system 18 to generate the HOA coefficients 11. The audio editing system 18 represents any system capable of editing audio data and outputting audio data as one or more source spherical harmonic coefficients.

When the editing process is completed, the content creator device 12 may generate the bitstream 21 based on the HOA coefficients 11. That is, the content creator device 12 is configured to represent a device configured to encode or otherwise compress the HOA coefficients 11 in accordance with various aspects of the techniques described in this disclosure for generating the bitstream 21 And an audio encoding device (20). The audio encoding device 20 may generate a bitstream 21 for transmission over a transmission channel, which may be, for example, a data storage device, a wired or wireless channel. The bitstream 21 may represent an encoded version of the HOA coefficients 11 and may include a primary bitstream or other side bitstream, which may be referred to as side channel information.

Although depicted in Figure 2 as being directly transmitted to the content consumer device 14, the content creator device 12 includes a bitstream 21 for the intermediate device positioned between the content creator device 12 and the content consumer device 14 Output. The intermediate device may store the bitstream 21 for later delivery to the content consumer device 14 which may request the bitstream. The intermediate device includes any other device capable of storing the bitstream 21 for future retrieval by a file server, web server, desktop computer, laptop computer, tablet computer, mobile phone, smart phone, or audio decoder It is possible. The intermediate device is capable of streaming the bitstream 21 to subscribers, such as the content consumer device 14 requesting the bitstream 21 (and possibly with the transmission of the corresponding video data bitstream) ) May reside in a content delivery network.

Alternatively, the content creator device 12 may store the bitstream 21 in a storage medium such as a compact disk, a digital video disk, a high definition video disk, or other storage media, many of which are read by a computer And thus may be referred to as computer-readable storage media or non-volatile computer-readable storage media. In this context, the transmission channel may refer to channels (and may include retail stores and other store-based delivery mechanisms) that cause content stored on these media to be transmitted. Accordingly, in any event, the techniques of this disclosure should not be limited to the example of FIG. 2 in this regard.

As further shown in the example of FIG. 2, the content consumer device 14 includes an audio playback system 16. The audio reproduction system 16 may represent any audio reproduction system capable of reproducing multi-channel audio data. The audio playback system 16 may include a number of different renderers 22. Renderers 22 may each provide a different type of rendering, where rendering of different types may be performed by one or more of various methods of performing vector-based amplitude panning (VBAP) and / And may include one or more of various methods of performing sound field synthesis. As used herein, "A and / or B" means "A or B ", or" A and B ".

The audio reproduction system 16 may further comprise an audio decoding device 24. [ The audio decoding device 24 may represent a device configured to decode the HOA coefficients 11 'from the bitstream 21, wherein the HOA coefficients 11' are similar to the HOA coefficients 11, (E. G., Quantization) and / or transmission over a transmission channel. The audio playback system 16 may decode the bitstream 21 and then obtain the HOA coefficients 11 'and render the HOA coefficients 11' to output the loudspeaker feeds 25. Loudspeaker feeds 25 may drive one or more loudspeakers (not shown in the example of FIG. 2 for ease of illustration purposes).

In order to select an appropriate renderer or, in some cases, to create an appropriate renderer, the audio playback system 16 may obtain loudspeaker information 13 indicating the number of loudspeakers and / or the spatial geometry of the loudspeakers have. In some cases, the audio playback system 16 may drive the loudspeakers in a manner that dynamically determines the loudspeaker information and obtain the loudspeaker information 13 using the reference microphone. In other cases or with dynamic determination of the loudspeaker information 13, the audio playback system 16 may interact with the audio playback system 16 and prompt the user to enter the loudspeaker information 13.

The audio playback system 16 may then select one of the audio renderers 22 based on the loudspeaker information 13. In some cases, the audio playback system 16 is configured so that none of the audio renderers 22 is within some criticality similarity measure (from the loudspeaker geometry point of view) to the loudspeaker geometry specified in the loudspeaker information 13 , It may generate one of the audio renderers 22 based on the loudspeaker information 13. The audio playback system 16 may in some cases be able to select one of the audio renderers 22 based on the loudspeaker information 13 without first attempting to select an existing one of the audio renderers 22. [ You can also create one. The one or more speakers 3 may then reproduce the rendered loudspeaker feeds 25.

In some cases, the audio playback system 16 may select an audio renderer of any one of the audio renderers 22 and may select a source (e.g., a DVD player, a Blu- (E.g., a video player, a video player, a video player, a ray player, a smartphone, a tablet computer, a gaming system, and a television). Although any one of the audio renderers 22 may be selected, the audio renderer used when creating the content is often one of the audio renderers, i.e., the audio renderer 5 in the example of FIG. 3, (And possibly the best) form of rendering due to the fact that the content has been created by the content creator 12 with the help of the content creator. Selecting one of the same or at least approximate audio renderers 22 (in terms of rendering style) may provide a better representation of the sound field and may result in a better surround sound experience for the content consumer 14 It is possible.

According to the techniques described in this disclosure, the audio encoding device 20 may generate the bitstream 21 to include audio rendering information 2 ("render information 2"). The audio rendering information 2 may include a signal value that identifies the audio renderer used to create the multi-channel audio content, i.e., the audio renderer 1 in the example of FIG. In some cases, the signal value includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In some cases, the signal value includes two or more bits defining an index indicating that the bitstream includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds. In some cases, when an index is used, the signal value may include two or more bits defining the number of rows of the matrix included in the bitstream and the number of columns of the matrix included in the bitstream Lt; RTI ID = 0.0 > and / or < / RTI > Given this information and given that each coefficient of a two-dimensional matrix is normally defined by a 32-bit floating-point number, the size in terms of the bits of the matrix is determined by the number of rows, the number of columns, May be computed as a function of the size of the floating-point numbers that define the coefficient, i. E. In this example, as 32-bits.

In some cases, the signal value specifies a rendering algorithm used to render the spherical harmonic coefficients into a plurality of speaker feeds. The rendering algorithm may include a matrix known to both the audio encoding device 20 and the decoding device 24. That is, the rendering algorithm may include the application of the matrix in addition to other rendering steps such as panning (e.g., VBAP, DBAP or simple panning) or NFC filtering. In some cases, the signal value includes two or more bits defining an index associated with one of the plurality of matrices used to render the spherical harmonic coefficients into a plurality of speaker feeds. Again, both the audio encoding device 20 and the decoding device 24 are configured with information indicating the order and the plurality of matrices of the plurality of matrices such that the index may uniquely identify one of the plurality of matrices It is possible. Alternatively, the audio encoding device 20 may be configured to decode the bitstream 21, which defines the order and / or the plurality of matrices of the plurality of matrices such that the index may uniquely identify a particular one of the plurality of matrices May be specified.

In some cases, the signal value includes two or more bits that define an index associated with one of a plurality of rendering algorithms used to render the spherical harmonic coefficients into a plurality of speaker feeds. Again, both the audio encoding device 20 and the decoding device 24 may use the information indicating the order of a plurality of rendering algorithms and a plurality of rendering algorithms so that the index may uniquely identify a particular one of the plurality of matrices . Alternatively, the audio encoding device 20 may be configured to decode the bitstream 21, which defines the order and / or the plurality of matrices of the plurality of matrices such that the index may uniquely identify a particular one of the plurality of matrices May be specified.

In some cases, the audio encoding device 20 specifies audio rendering information (2) on an audio frame basis in the bitstream. In other cases, the audio encoding device 20 specifies audio rendering information 2 in the bitstream a single time.

The decoding device 24 may then determine the audio rendering information (2) specified in the bitstream. Based on the signal values contained in the audio rendering information 2, the audio playback system 16 may render a plurality of speaker feeds 25 based on the audio rendering information 2. As mentioned above, the signal value may include a matrix used in some cases to render the spherical harmonic coefficients into a plurality of speaker feeds. In this case, the audio playback system 16 may configure one of the audio renderers 22 as a matrix to provide the audio renderer 22 with one of the audio renderers 22 to render the speaker feeds 25 based on the matrix. May be used.

In some cases, the signal value includes two or more bits defining an index indicating that the bitstream includes a matrix used to render the HOA coefficients 11 'into the speaker feeds 25. The decoding device 24 may parse the matrix from the bitstream in response to the index such that the audio playback system 16 constructs an audio renderer of one of the audio renderers 22 with a parsed matrix, Or one of the audio renderers 22 to render the speaker feeds 25. When the signal value includes two or more bits defining the number of rows of the matrix included in the bitstream and two or more bits defining the number of columns of the matrix included in the bitstream, It may parse the matrix from the bit stream based on two or more bits that define the number of rows and two or more bits that define the number of rows and the number of columns and in response to the index in the manner described above.

In some cases, the signal value specifies the rendering algorithm used to render the HOA coefficients 11 'into the speaker feeds 25. In these cases, some or all of the audio renderers 22 may perform these rendering algorithms. The audio playback device 16 may then utilize one of the specified rendering algorithms, for example, the audio renderers 22, to render the speaker feeds 25 from the HOA coefficients 11 ' .

When the signal value includes two or more bits defining an index associated with one of the plurality of matrices used to render the HOA coefficients 11 'into the speaker feeds 25, one of the audio renderers 22 Some or all of these may represent the plurality of matrices. Thus, the audio playback system 16 may render the speaker feeds 25 from the HOA coefficients 11 'using one of the audio renderers 22 associated with the index.

When the signal values include two or more bits defining an index associated with one of a plurality of rendering algorithms used to render the HOA coefficients 11 'into the speaker feeds 25, the audio renderers 34, Some or all of which may represent these rendering algorithms. Thus, the audio playback system 16 may render the speaker feeds 25 from the spherical harmonic coefficients 11 'using one of the audio renderers 22 associated with the index.

Depending on how often this audio rendering information is specified in the bitstream, the decoding device 24 may audio-frame-based or single-decode the audio rendering information (2).

By specifying audio rendering information 3 in this manner, techniques can potentially result in better playback of multi-channel audio content, and the content creator 12 may be able to adapt to the manner in which the multi- . As a result, techniques may provide a more immersive surround sound or multi-channel audio experience.

In other words, and as noted above, the higher order ambiance (HOA) may represent a way to describe the directional information of the sound field based on the spatial Fourier transform. Typically, the higher the ambsonic order N, the higher the spatial resolution, the larger the number of spherical harmonics (SH) coefficients (N + 1) ^ 2, and the larger the bandwidth required to transmit and store data.

A potential benefit of this description is the possibility to reproduce this sound field in most random loudspeaker setups (e.g., 5.1, 7.1 22.2, etc.). Converting from the sound field description to M loudspeaker signals may be done through a static rendering matrix with (N + 1) ² inputs and M outputs. As a result, all loudspeaker setups may require a dedicated rendering matrix. There may be several algorithms for computing a rendering matrix for desired loudspeakers that may be optimized for some objective or subjective scale, e.g., Gerzon criteria. For irregular loudspeaker setups, the algorithms may be complicated by repetitive numerical optimization procedures, such as convex optimization. To compute the rendering matrix for irregular loudspeaker layouts without waiting time, it may be advantageous to have enough computational resources available. Irregular loudspeaker setups may be common in home living environments due to architectural limitations and aesthetic preferences. Thus, for best sound field reproduction, a rendering matrix optimized for this scenario may be preferred in that it may enable more accurate reproduction of the sound field.

Because audio decoders typically do not require many computational resources, the device may not be able to compute an irregular rendering matrix at consumer friendly times. Various aspects of the techniques described in this disclosure may be provided for use in a cloud-based computing approach as follows:

1. The audio decoder may send loudspeaker coordinates (and in some cases also SPL measurements obtained with a calibration microphone) to the server over an Internet connection;

2. The cloud-based server may calculate a rendering matrix (and possibly several different versions, which the customer may later select from these different versions); And

3. The server may then send the rendering matrix (or different versions) back to the audio decoder over the Internet connection.

This approach allows manufacturers to reduce the manufacturing costs of audio decoders (since a robust processor may not be required to compute these irregular rendering matrices), but also allows for the use of rendering matrices that are typically designed for regular speaker configurations or geometries It may be possible to facilitate more optimal audio reproduction. The algorithm for computing the rendering matrix may also be optimized after the audio decoder is shipped, potentially reducing costs for hardware changes or even recall. The techniques may also collect, in some cases, a lot of information about different loudspeaker setups of consumer products that may be beneficial to future product developments.

In some cases, the system shown in FIG. 3 may not signal audio rendering information 2 in bitstream 21, as described above, but instead, as metadata separated from bitstream 21, The audio rendering information 2 may be signaled. Alternatively or in conjunction with what has been described above, the system shown in FIG. 3 may signal part of the audio rendering information 2 in the bitstream 21, as described above, And may signal a part of the audio rendering information (3). In some instances, the audio encoding device 20 may output this metadata, which may then be uploaded to a server or other device. The audio decoding device 24 may then download or otherwise retrieve this metadata which is then used by the audio decoding device 24 to augment the audio rendering information extracted from the bitstream 21 . The bitstream 21 formed according to the rendering information aspects of the techniques is described below with respect to the examples of Figs. 8a-8d.

FIG. 3 is a block diagram illustrating in greater detail one example of the audio encoding device 20 shown in the example of FIG. 2, which may perform various aspects of the techniques described in this disclosure. The audio encoding device 20 includes a content analysis unit 26, a vector-based decomposition unit 27 and a directionality-based decomposition unit 28. As will be briefly described below, the various aspects of compressing or otherwise encoding HOA coefficients and more information about the audio encoding device 20 are described in "INTERPOLATION FOR DECOMPOSED REPRESENTATIONS A SOUND FIELD" International Patent Application Publication No. WO 2014/194099, filed on May 29th.

The content analyzing unit 26 represents a unit configured to analyze the content of the HOA coefficients 11 to identify whether the HOA coefficients 11 represent content generated from live recording or audio objects. The content analyzing unit 26 may determine whether the HOA coefficients 11 were generated from recording of the actual sound field or from an artificial audio object. In some cases, when the framed HOA coefficients 11 have been generated from the recording, the content analysis unit 26 passes the HOA coefficients 11 to the vector-based decomposition unit 27. In some cases, when the framed HOA coefficients 11 have been generated from the composite audio object, the content analyzing unit 26 passes the HOA coefficients 11 to the directional-based compositing unit 28. The directional-based synthesis unit 28 may represent a unit configured to perform directional-based synthesis of the HOA coefficients 11 to produce a directional-based bitstream 21.

3, the vector-based decomposition unit 27 includes a linear inverse transform (LIT) unit 30, a parameter calculation unit 32, a reorder unit 34, a foreground selection unit 36, A sound field analysis unit 44, a coefficient reduction unit 46, a background (BG) selection unit 48, a spatial-temporal analysis unit 44, A temporal interpolation unit 50, and a quantization unit 52. [

Linear inverse transform (LIT) units 30 receives the the HOA coefficient 11 in the form of HOA channels, each channel may be displayed as the spherical basis functions (HOA [k], where k is the number of samples A block or frame of coefficients associated with a given degree, a lower order, of the current frame or block). The matrix of HOA coefficients 11 may have dimensions D : M x ( N +1) ² .

The LIT unit 30 may represent a unit configured to perform a form of analysis referred to as singular value decomposition. Although described with respect to SVD, the techniques described in this disclosure may be performed with respect to any similar transformations or decompositions that provide for sets of linearly uncorrelated energy-intensive outputs. Also, the term "sets" in this disclosure is intended to refer generally to non-zero sets unless specifically stated to the contrary, and includes the classical mathematical definition of sets including so-called " ≪ / RTI > Alternative transformations may include principal component analysis, often referred to as "PCA ". In accordance with this context, the PCA can be implemented in several ways, such as the Karhunen-Loeve transform, the Hotelling transform, the proper orthogonal decomposition (POD) eigenvalue decomposition (EVD)). < / RTI > Attributes of these operations that serve the primary purpose of compressing audio data are the " energy concentration " and " uncorrelated "

In any case, assuming that the LIT unit 30 performs singular value decomposition (again, referred to as "SVD") for exemplary purposes, the LIT unit 30 sends the HOA coefficients 11 to the transformed Into two or more sets of HOA coefficients. The "sets" of transformed HOA coefficients may include vectors of transformed HOA coefficients. In the example of FIG. 3, the LIT unit 30 may perform SVD with respect to the HOA coefficients 11 to generate the so-called V matrix, S matrix, and U matrix. The SVD in linear algebra can be expressed as a factorial of a y-by-z real number or a complex matrix X (where X may represent multi-channel audio data such as HOA coefficients 11) Can be expressed in the following form:

X = USV ^*

U may represent a y-by-y real number or a complex unitary matrix, where the y columns of U are known as left-specific vectors of multi-channel audio data. S may represent a y-by-z rectangular diagonal matrix with diagonal non-negative real numbers, where diagonal values of S are known as singular values of multi-channel audio data. V ^* (which may represent the conjugate transposition of V) may represent a z-by-z real or complex unitary matrix, where the z columns of V ^* are known as right-specific vectors of multi-channel audio data .

In some examples, the V ^* matrix in the SVD equation referenced above is represented as the conjugate transpose of the V matrix to reflect that the SVD may be applied to matrices containing complex numbers. When applied to matrices containing only real numbers, the complex conjugate (or, in other words, the V ^* matrix) of the V matrix may be considered to be a transpose of the V matrix. Below, for ease of illustration purposes, the HOA coefficients 11 are assumed to contain real numbers as a result of the V matrix being output through the SVD rather than the V ^* matrix. Furthermore, although shown as a V matrix in this disclosure, it should be understood that the reference to the V matrix refers to the transpose of the V matrix, where appropriate. V matrix, the techniques may be applied in a similar manner to the HOA coefficients 11 with complex coefficients, where the output of the SVD is a V ^* matrix. Accordingly, techniques should not be limited to providing only the application of SVD to generate a V matrix in this regard, but may involve applying SVD to HOA coefficients 11 with complex components to generate a V ^* matrix It is possible.

라고 지칭될 수도 있다.In this way, LIT unit 30 by performing the SVD respect to HOA coefficient 11, the dimensions D: s x M (N +1) with US ² [k] vector of (33) (S Vector And V [ k ] vectors 35 with dimensions D: ( N + 1) ² x ( N + 1) ² . The discrete vector elements in the US [k] matrix may also be referred to as X _PS (k) , while the discrete vectors in the V [k] matrix are also

. ≪ / RTI >

에 의해 대신 표현될 수도 있다. 벡터들

각각의 개별 엘리먼트들은 연관된 오디오 오브젝트에 대한 음장의 형상 (폭을 포함함) 및 포지션을 설명하는 HOA 계수를 표현할 수도 있다. U 행렬 및 V 행렬에서의 벡터들 양쪽은 이들의 제곱-평균-제곱근 (root-mean-square) 에너지들이 1 과 동일하도록 정규화된다. 따라서, U 에서의 오디오 신호들의 에너지는 S 에서의 대각선 엘리먼트들에 의해 표현된다. 따라서, U 와 S 를 곱하여 (개별 벡터 엘리먼트들 X _PS (k) 를 갖는) US[k] 를 형성하는 것은 에너지들을 갖는 오디오 신호를 표현한다. (U 에서의) 오디오 시간-신호들, (S 에서의) 이들의 에너지들, 및 (V 에서의) 이들의 공간 특성들을 커플링해제하기 위한 SVD 분해의 능력은 본 개시물에서 설명되는 기법들의 다양한 양태들을 지원할 수도 있다. 추가로, 기본 HOA[k] 계수들 (X) 을 US[k] 와 V[k] 의 벡터 곱셈에 의해 합성하는 모델은 본 문헌 전반에 걸쳐 사용되는 용어 "벡터-기반 분해" 를 발생시킨다.The analysis of the U, S and V matrices may indicate that these matrices contain or represent the spatial and temporal properties of the fundamental field described above by X. Each of the N vectors in U (of length M samples) are normalized separated audio signals that are mutually orthogonal and uncoupled from any spatial properties (which may also be referred to as directional information) (M samples Lt; / RTI > (for a time period represented by < RTI ID = 0.0 > Room shape and position (r, theta, pi) of the spatial properties of the matrix V are expressed in the individual i-th vector (the length of each of the (N + 1) ²⁾

May be represented instead by. Vectors

Each individual element may represent a HOA coefficient describing the shape (including width) and position of the sound field for the associated audio object. Both vectors in the U matrix and V matrix are normalized such that their root-mean-square energies are equal to one. Thus, the energy of the audio signals at U is represented by the diagonal elements at S. Thus, multiplying U by S to form US [ k ] (with discrete vector elements X _PS (k) ) represents an audio signal with energies. The ability of the SVD decomposition to decouple the audio time-signals (at U), their energies (at S), and their spatial properties (at V) It may support various aspects. In addition, a model that synthesizes the basic HOA [ k ] coefficients X by vector multiplication of US [ k ] and V [ k ] generates the term "vector-based decomposition" used throughout this document.

The LIT unit 30 may apply a linear inverse transform to the derivatives of the HOA coefficients 11, although it is described as being performed directly with respect to the HOA coefficients 11. For example, the LIT unit 30 may apply the SVD with respect to the power spectral density matrix derived from the HOA coefficients 11. By performing the SVD with respect to the power spectral density (PSD) of the HOA coefficients rather than the coefficients themselves, the LIT unit 30 can reduce the computational complexity of performing SVD in terms of one or more of the processor cycles and storage space, May achieve the same source audio encoding efficiency as if it were directly applied to the HOA coefficients.

The parameter calculation unit 32 represents a unit configured to calculate various parameters such as correlation parameter R , directionality parameters ? , ? , R and energy attribute e . Each of the parameters for the current frame may be denoted as R [k], θ [k ], φ [k], r [k] and e [k]. The parameter calculation unit 32 may perform energy analysis and / or correlation (or so-called cross-correlation) on the US [ k ] vectors 33 to identify these parameters. The parameter calculation unit 32 also may determine the parameters for the previous frame, where the previous frame parameters based on the previous frame of the US [k -1] and a vector V [k -1] vector R [k -1] , it may be expressed as θ [k -1], φ [ k -1], r [k -1] and e [k -1]. The parameter calculation unit 32 may output the current parameters 37 and the previous parameters 39 to the reorder unit 34. [

로서 표시될 수도 있음) 및 리오더링된 V[k] 행렬 (35') (수학적으로

로서 표시될 수도 있음) 을 전경 사운드 (또는 우세 사운드 - PS) 선택 유닛 (36) ("전경 선택 유닛 (36)") 및 에너지 보상 유닛 (38) 에 출력할 수도 있다.The parameters computed by the parameter computation unit 32 may be used by the reorder unit 34 to reorder audio objects to represent their natural evaluation or continuity over time. The reorder unit 34 maps each of the parameters 37 from the first US [ k ] vectors 33 to the parameters 39 for the second US [ k -1] vectors 33, You can also compare for each. The reorder unit 34 calculates the various vectors in the US [ k ] matrix 33 and the V [ k ] matrix 35 based on the current parameters 37 and the previous parameters 39 (in one example, (Using a Hungarian algorithm) to generate a reordered US [ k ] matrix 33 '(mathematically

) And a reordered V [ k ] matrix 35 '(mathematically

To the foreground sound (or dominant sound-PS) selection unit 36 ("foreground selection unit 36") and the energy compensation unit 38. [

The sound field analysis unit 44 may represent a unit configured to perform sound field analysis with respect to the HOA coefficients 11 to potentially achieve a target bit rate 41. [ The sound field analysis unit 44 may determine the total number of psychoacoustic coder instantiations (which may be a function of the total number of surrounding or background channels BG _TOT ) and the number of psychoacoustic coder instantiations based on the analyzed and / May determine the number of foreground channels, or, in other words, the number of dominant channels. The total number of psychoacoustic coder instantiations can be displayed as numHOATransportChannels.

The sound field analysis unit 44 also includes a total number of foreground channels nFG 45 and a minimum degree N of the background (or, e. G., Surrounding) sound field to potentially achieve the target bit rate 41 again _BG, or, alternatively, MinAmbHOAorder), the number of which of the physical channels representing the minimum degree of the background sound corresponding (nBGa = (MinAmbHOAorder + 1) 2), and to send the index of the additional BG HOA channel (i) (Fig. (Which may be collectively displayed as background channel information 43 in the example of FIG. 3). Background channel information 42 may also be referred to as peripheral channel information 43. [ numHOATransportChannels - Each of the channels left from nBGa may be an "additional background / peripheral channel", "active vector-based dominant channel", "active direction-based dominant signal" or "completely inactive". In one aspect, the channel types may be represented by a syntax element (e.g., 00: directional based signal, 01: vector-based dominant signal, 10: ; 11: inactive signal). The total number of background or surrounding signals (nBGa) may be given by (MinAmbHOAorder + 1) ² + the number of times the index 00 appears in the bit stream for that frame as the channel type (in the example above).

The sound field analyzing unit 44 selects the number of background (or, moreover, peripheral) channels and the number of foreground (or, more dominant) channels based on the target bit rate 41, May select more background and / or foreground channels when the target bit rate 41 is relatively high (e.g., when the target bit rate 41 is greater than 512 Kbps). In one aspect, MinAmbHOAorder may be set to 1 in the header selection of the bitstream while numHOATransportChannels may be set to 8. In this scenario, in all frames, four channels may be dedicated to represent the background or surrounding portion of the sound field, while the other four channels may be frame based - for example, additional background / surround channel or foreground / It can be varied for the type of channel used as the dominant channel. The foreground / dominant signals may be one of vector-based or directional based signals, as described above.

In some cases, the total number of vector-based dominant signals for a frame may be given by the number of times the ChannelType index is 01 in the bitstream of that frame. In the above embodiment, for every additional background / perimeter channel (e.g., corresponding to a ChannelType of 10), the corresponding information of any HOA coefficient (beyond the first four) May be expressed. The information may be an index for indicating the HOA coefficients 5 to 25 for the fourth-order HOA contents. The first four neighboring HOA coefficients 1 through 4 may be transmitted at all times when minAmbHOAorder is set to one so that the audio encoding device can only transmit one of the additional neighboring HOA coefficients having an index of only 5 to 25 It may be necessary to indicate. Thus, this information may be transmitted using a 5-bit syntax element (for fourth-order content) that may be denoted as "CodedAmbCoeffIdx ". In any case, the sound field analysis unit 44 outputs the background channel information 43 and the HOA coefficients 11 to the background (BG) selection unit 36 and the background channel information 43 to the coefficient reduction unit 46 ) And the bitstream generating unit 42, and outputs the nFG 45 to the foreground selecting unit 36. [

The background selection unit 48 is a background channel information (e.g., additional BG indices of HOA channel (i) and the number (nBGa) and the background field (N _BG) send) the background or ambient HOA coefficient based on ( 47). ≪ / RTI > For example, when N _BG is equal to one, the background selection unit 48 may select the HOA coefficients 11 for each sample of the audio frame having a degree less than or equal to one. Thereafter, the background selection unit 48 may, in this example, select the HOA coefficients 11 with the index identified by one of the indices i as additional BG HOA coefficients, To be specified in the bitstream 21 to allow an audio decoding device such as the audio decoding device 24 shown in the example of Figures 2 and 4 to parse the background HOA coefficients 47 from the bitstream 21 And is provided to the bitstream generating unit 42. Thereafter, the background selection unit 48 may output the peripheral HOA coefficients 47 to the energy compensation unit 38. [ Peripheral HOA coefficient 47 is the dimension D: may have a _{M x [(N BG +1)} 2 + nBGa]. The neighboring HOA coefficients 47 may also be referred to as "neighboring HOA coefficients 47 ", wherein each of the neighboring HOA coefficients 47 is associated with a separate peripheral HOA channel 47 to be encoded by the psychoacoustic audio coder unit 40 (47).

(49) 로서 표시될 수도 있음) 을 심리음향 오디오 코더 유닛 (40) 에 출력할 수도 있고, 여기서 nFG 신호들 (49) 은 차원들 D: M x nFG 를 가질 수도 있고 각각은 모노-오디오 오브젝트들을 표현한다. 전경 선택 유닛 (36) 은 또한, 음장의 전경 성분들에 대응하는 리오더링된 V[k] 행렬 (35') (또는

(35')) 을 공간-시간 보간 유닛 (50) 에 출력할 수도 있고, 여기서 전경 성분들에 대응하는 리오더링된 V[k] 행렬 (35') 의 서브세트는 차원들 D: (N+1)² x nFG 를 갖는 전경 V[k] 행렬 (51 _k ) (수학적으로

로서 표시될 수도 있음) 로서 표시될 수도 있다.The foreground selection unit 36 includes a reordered V [ k ] matrix 35 'that represents the foreground or distinct components of the sound field based on the nFG 45 (which may represent one or more indices identifying foreground vectors) And a reordered US [ k ] matrix 33 '. The foreground selection unit 36 receives the nFG signals 49 (reordered US [ k ] _{1, ..., nFG} 49, FG _{1, ..., nfG} [ k ]

(Which may also be denoted as < RTI ID = 0.0 > 49), to the psychoacoustic audio coder unit 40 where the nFG signals 49 may have dimensions D: M x nFG, Express. The foreground selection unit 36 also includes a reordered V [ k ] matrix 35 'corresponding to the foreground components of the sound field (or

(35 ')) to a space-time interpolation unit 50, wherein a subset of the reordered V [ k ] matrix 35' corresponding to the foreground components comprises dimensions D: ( N + 1) a foreground V [ k ] matrix 51 _k with ² x nFG (mathematically

As shown in FIG.

The energy compensation unit 38 represents a unit configured to perform energy compensation with respect to the surrounding HOA coefficients 47 to compensate for the energy loss due to removal of the various HOA channels among the HOA channels by the background selection unit 48 It is possible. The energy compensation unit 38 is a reordering the US [k] matrix (33 '), the reordering of V [k] matrix (35'), nFG signal (49), foreground V [k] vector (51 _k ) And neighboring HOA coefficients 47, energy compensation may then be performed based on energy analysis to generate energy-compensated neighboring HOA coefficients 47 '. The energy compensation unit 38 may output the energy-compensated neighboring HOA coefficients 47 'to the psychoacoustic audio coder unit 40.

Space-time interpolation unit 50 in the foreground of the k-th frame, V [k] vector s (51 _k) and a previous frame view V [k -1] vector, for a (k-1 displayed accordingly) (51 _{k -1} ) and perform spatial-temporal interpolation to generate interpolated foreground V [ k ] vectors. The space-time interpolation unit 50 may reconstruct the reordered foreground HOA coefficients by recombining the nFG signals 49 with the foreground V [ k ] vectors 51 _k . Thereafter, the space-time interpolation unit 50 may generate the interpolated nFG signals 49 'by dividing the reordered foreground HOA coefficients by the interpolated V [ k ] vectors. Space-time interpolation unit 50 also, the interpolated foreground V [k] in the foreground that was used to produce the vector V [k] vector s (51 _k), the method may output audio such as an audio decoding device 24 The decoding device may generate the interpolated foreground V [ k ] vectors and thereby restore the foreground V [ k ] vectors _51k . The used to generate the interpolated foreground V [k] vector foreground V [k] of the vector (51 _k) is expressed as the remaining foreground V [k] vector (53). To ensure that the same V [k] and V [k-1] are used in the encoder and decoder (to produce the interpolated vectors V [k]), the quantized / Lt; / RTI > The spatial-temporal interpolation unit 50 outputs the interpolated nFG signals 49 'to the psychoacoustic audio coder unit 46 and outputs the interpolated foreground V [ k ] vectors 51 _k to the coefficient reduction unit 46, As shown in FIG.

The coefficient reducing unit 46 reduces the foreground V [k] vectors (55) based upon a on a quantization unit 52, background channel information 43 to output the remaining foreground V [k] vector 53 A unit configured to perform a coefficient reduction with respect to a coefficient. The reduced foreground V [ k ] vectors 55 may have dimensions D: [( N +1) ² - ( N _BG +1) ² -BG _TOT ] x nFG. The coefficient reduction unit 46 may, in this regard, represent a unit configured to reduce the number of coefficients in the remaining foreground V [ k ] vectors 53. In other words, the coefficient of the reduction unit 46 is not substantially have for directional information, the foreground (the remaining views V [k] vectors to form a (53)) V [k] vector with the unit configured to eliminate the coefficients in the expressed It is possible. In some instances, the distinct or, in other words, the coefficients of the foreground V [ k ] vectors corresponding to the first and the zero order basis functions (which may be denoted as N _BG ) provide little or no directional information, May be removed from the foreground V-vectors (via a process that may be referred to as "factor reduction"). In this example, not only the coefficients corresponding to N _BG but also additional HOA channels (which may be indicated by the variable TotalOfAddAmbHOAChan) from the set of [(N _BG +1) ² +1, (N + 1) ² ] May be provided with the greatest flexibility to identify.

The quantization unit 52 performs any form of quantization to compress the reduced foreground V [ k ] vectors 55 to produce coded foreground V [ k ] vectors 57 to produce a coded foreground V [ k ] vectors 57 to the bitstream generation unit 42. The bitstream generation unit 42 may be a unit configured to output the k- In operation, the quantization unit 52 may represent a unit configured to compress one or more of the spatial components of the sound field, i. E., The reduced foreground V [ k ] vectors 55 in this example. The quantization unit 52 may perform any of the following twelve quantization modes, as indicated by the quantization mode syntax element indicated by "NbitsQ"

NbitsQ value Types of quantization mode

0-3: Reserved

4: Vector quantization

5: Scalar quantization without Huffman coding

6: 6-bit scalar quantization using Huffman coding

7: 7-bit scalar quantization using Huffman coding

8: 8-bit scalar quantization using Huffman coding

... ...

16: 16-bit scalar quantization using Huffman coding

The quantization unit 52 may also perform predicted versions of any of the quantization modes of the types described above, wherein the elements of the V-vector of the previous frame (or the weight when vector quantization is performed) The difference determined between the elements of the V-vector of the current frame (or the weight when vector quantization is performed) is determined. The quantization unit 52 may then quantize the difference between the current frame and the elements or weights of the previous frame, rather than the value of the element of the V-vector of the current frame.

Quantization unit 52 to obtain reduced foreground V [k] number of the coded version of the vectors 55, a reduced view V by performing quantization of a plurality of types with respect to each [k] vector (55) It is possible. Quantization unit 52 may select one of the coded version of the reduced as a coded foreground V [k] vector 57 views V [k] vector (55). The quantization unit 52, in other words, a non-predicted vector-quantized V-vector for use as an output switched-quantized V-vector based on any combination of the criteria discussed in this disclosure, Vector-quantized V-vector, a non-Huffman-coded scalar-quantized V-vector, and a Huffman-coded scalar-quantized V-vector. In some instances, the quantization unit 52 selects a quantization mode from a set of quantization modes that include a vector quantization mode and one or more scalar quantization modes, and based on the selected mode (or in accordance with the selected mode) It can also be quantized. Thereafter, the quantization unit 52 generates a predicted vector-quantized V-vector (e.g., in terms of weighted values or bits representing it), a predicted vector-quantized V-vector , A non-Huffman-coded scalar-quantized V-vector, and a Huffman-coded scalar-quantized V-vector in the coded foreground V [ k ] vector To the bitstream generating unit 52 as the bitstreams 57. The quantization unit 52 may also provide syntax elements (e.g., NbitsQ syntax elements) representing the quantization mode and any other syntax elements used to de-quantize or otherwise reconstruct the V-vector have.

The psychoacoustic audio coder unit 40 included in the audio encoding device 20 may represent a plurality of instances of a psychoacoustic audio coder, each of which includes energy-compensated neighboring HOA coefficients 47 'and an interpolated nFG signal 47' Are used to encode the different audio objects or HOA channels of each of the frames 49 'to generate encoded neighboring HOA coefficients 59 and encoded nFG signals 61. The psychoacoustic audio coder unit 40 may output the encoded neighboring HOA coefficients 59 and the encoded nFG signals 61 to the bitstream generation unit 42. [

The bitstream generation unit 42 included in the audio encoding device 20 formats the data to comply with a known format (which may be referred to as a format known by the decoding device) Lt; / RTI > The bitstream 21, in other words, may represent encoded audio data that was encoded in the manner described above. The bitstream generation unit 42 includes in some examples coded foreground V [ k ] vectors 57, encoded neighboring HOA coefficients 59, encoded nFG signals 61 and background channel information 43 Lt; RTI ID = 0.0 > a < / RTI > The bitstream generating unit 42 then generates a bitstream based on the coded foreground V [ k ] vectors 57, the encoded neighboring HOA coefficients 59, the encoded nFG signals 61 and the background channel information 43 To generate the bit stream 21. In this way, the bitstream generation unit 42 may specify the vectors 57 in the bitstream 21 to thereby obtain the bitstream 21 therefrom. The bitstream 21 may comprise a primary or main bitstream and one or more side channel bitstreams.

Various aspects of the techniques may also enable the bitstream generation unit 46 to specify the audio rendering information 2 in the bitstream 21, as described above. Although the current version of the upcoming 3D video compression task draft provides for signaling specific downmix matrices in the bitstream 21, this working draft is based on the assumption that the renderer used in rendering the HOA coefficients 11 in the bitstream 21 Lt; / RTI > For HOA content, the equivalent of this downmix matrix is a rendering matrix that converts the HOA representation into the loudspeaker feeds desired. Various aspects of the techniques described in this disclosure allow the bitstream generation unit 46 to signal the HOA and the channel content (e. G., As audio rendering information 2) by signaling the HOA rendering matrices in the bitstream. It is proposed to further match feature sets.

One exemplary signaling solution based on the coding scheme of the downmix matrices and optimized for HOA is presented below. Similar to the transmission of downmix matrices, HOA rendering matrices may be signaled within mpegh3daConfigExtension () . Techniques may provide a new extension type ID_CONFIG_EXT_HOA_MATRIX as presented in the following tables (italics and boldface represent changes to existing tables).

Table - Syntax of mpegh3daConfigExtension () (Table 13 on the CD)

Table - Values of usacConfigExtType (Table 1 on the CD)

The bit field HOARenderingMatrixSet () may be the same in structure and functionality as DownmixMatrixSet () . Instead of inputCount (audioChannelLayout) , HOARenderingMatrixSet () may use the "equivalent" NumOfHoaCoeffs value computed in HOAConfig . In addition, since the ordering of HOA coefficients may be fixed within the HOA decoder (see, for example, Appendix G on CD), the HOARenderingMatrixSet does not need any equivalent for inputConfig (audioChannelLayout) .

Table 2 - Syntax of HOARenderingMatrixSet () (adopted from Table 15 on CD)

Various aspects of the techniques may also allow the bitstream generation unit 46 to generate HOA audio data (e. G., Audio data) using a first compression scheme (such as a decomposition compression scheme represented by a vector-based decomposition unit 27) (For example, the direction-based compression scheme represented by the direction-based decomposition unit 28 or the directional-based decompression unit 28 shown in FIG. 4, the HOA coefficients 11 in the example of FIG. directionality-based compression scheme) may be specified in the bitstream 21 so that the bitstream 21 is not included in the bitstream 21. For example, the bitstream generation unit 42 may generate a bitstream 21 such that it does not include HOAPredictionInfo syntax elements or fields that may be reserved for use in specifying prediction information between directional signals of a directional- May be generated. Examples of bit streams 21 generated in accordance with various aspects of the techniques described in this disclosure are illustrated in the examples of Figures 8E and 8F.

In other words, the prediction of the directional signals may be part of the dominant sound synthesis employed by the direction-based decomposition unit 28 and depends on the presence of ChannelType 0 (which may represent a direction-based signal). When no direction-based signal is present in the frame, no prediction of the directional signals may be performed. However, the associated sideband information HOAPredictionInfo () may be written to all frames independently of the presence of direction-based signals, even if they are not used. When no directional signal is present in the frame, the techniques described in this disclosure cause the bitstream generation unit 42 to signal HOAPredictionInfo to the sideband as presented in the next table (underlined italics indicate additions) It may be possible to reduce the size of the sidebands by not doing so:

Table: Syntax of HOAFrame

In this regard, techniques may be applied to devices, such as audio encoding device 20, for compressing higher order ambsonic audio data using a first compression scheme, It may be possible to specify a bitstream representing a compressed version of the higher order ambience sound data that does not include the bits corresponding to the scheme.

In some cases, the first compression scheme includes a vector-based decomposition compression scheme. In these and other cases, the vector-based decomposition compression scheme includes a compression scheme involving the application of singular value decomposition (or equivalents thereof as described in greater detail in this disclosure) to higher order ambsonic audio data.

In these and other cases, the audio encoding device 20 may be configured to specify a bitstream that does not include bits corresponding to at least one syntax element used to perform the second type of compression scheme. The second compression scheme, as mentioned above, may also include a directional-based compression scheme.

The audio encoding device 20 may also be configured to specify the bitstream 21 such that the bitstream 21 does not include the bits corresponding to the HOAPredictionInfo syntax element of the second compression scheme.

When the second compression scheme comprises a directional-based compression scheme, the audio encoding device 20 generates a bitstream 21 such that the bitstream 21 does not contain the bits corresponding to the HOAPredictionInfo syntax element of the directional- . ≪ / RTI > In other words, the audio encoding device 20 is configured to specify the bitstream 21 such that the bitstream 21 does not include the bits corresponding to the at least one syntax element used to perform the second type of compression schemes And the at least one syntax element represents a prediction between two or more directional-based signals. Again, again, when the second compression scheme includes a directional-based compression scheme, the audio encoding device 20 determines that the bitstream 21 does not contain bits corresponding to the HOAPredictionInfo syntax element of the directional-based compression scheme , Where the HOAPredictionInfo syntax element indicates a prediction between two or more directional-based signals.

Various aspects of the techniques may further enable the bitstream generation unit 46 to specify the bitstream 21 such that in some cases the bitstream 21 does not include the gain correction data . The bitstream generation unit 46 may specify the bitstream 21 so that when the gain correction is suppressed, the bitstream 21 does not include the gain correction data. Examples of the bit stream 21 generated according to various aspects of the techniques are shown in the examples of Figures 8E and 8F, as mentioned above.

In some cases, gain correction is applied when certain types of psychoacoustic encodings are performed, giving a relatively smaller dynamic range of certain types of these psychoacoustic encodings relative to other types of psychoacoustic encodings. For example, AAC has a relatively smaller dynamic range than unified speech and audio coding (USAC). When a compression scheme (such as a vector-based synthetic compression scheme or a directional-based compression scheme) involves USAC, the bitstream generation unit 46 generates a bitstream 21 (for example, a value of 0 in HOAConfig The bitstream 21 may be specified to not include the gain correction data ( by specifying the syntax element MaxGainCorrAmpExp in the HOAGainCorrectionData () field ) after signaling to the bitstream 21 that the gain correction has been suppressed.

In other words, (see table 71 in the CD) as a part of the bit field HOAConfig MaxGainCorrAmpExp may control the range automatic gain control module is affecting the transmission channel signal prior to the USAC core coding. In some cases, this module has been developed to improve the non-ideal dynamic range of RM0-enabled AAC encoder implementations. With the change from AAC to USAC corecoder during the integration phase, the dynamic range of the core encoder is improved so that the need for this gain control module may not be as important as before.

In some cases, the gain control functionality can be suppressed when MaxGainCorrAmpExp is set to zero. In these cases, the associated sideband information HOAGainCorrectionData () may not be written to every HOA frame per table above, which illustrates "Syntax of HOAFrame ". For configurations where MaxGainCorrAmpExp is set to zero, the techniques described in this disclosure may not signal HOAGainCorrectionData. Additionally, in such a scenario, the reverse gain control module may even be bypassed to reduce decoder complexity to about 0.05 MOPS per transport channel without any adverse side effects.

In this regard, techniques may be used to generate a bitstream (e. G., A bitstream) that represents a compressed version of high-order ambsonic audio data so that when the gain correction is suppressed during compression of higher- 21 of the audio encoding device 20. [

In these and other cases, the audio encoding device 20 may be configured to compress higher order ambsonic audio data according to a vector-based decomposition compression scheme to produce a compressed version of higher order ambsonic audio data. Examples of decomposition compression schemes may involve the application of singular value decomposition (or equivalents thereof as described in more detail above) to higher order ambience sound data to produce a compressed version of the higher order ambience sound data.

In these and other cases, the audio encoding device 20 may be configured to specify the MaxGainCorrAmbExp syntax element in the bitstream 21 as zero to indicate that the gain correction is suppressed. In some cases, the audio encoding device 20 may be configured to specify the bitstream 21 such that when the gain correction is suppressed, the bitstream 21 does not include a HOAGainCorrection data field that stores the gain correction data have. In other words, the audio encoding device 20 specifies the MaxGainCorrAmbExp syntax element in the bitstream 21 as 0 to indicate that the gain correction is suppressed and does not include the HOAGainCorrection data field storing the gain correction data in the bitstream .

In these and other cases, the audio encoding device 20 suppresses gain correction when the compression of higher order ambsonic audio data involves the application of a single audio audio and voice audio coding (USAC) for higher order ambsonic audio data. .

The aforementioned potential optimizations for the signaling of various information in the bitstream 21 may be adapted or otherwise updated in a manner to be described in more detail below. These updates may be applied together with other updates discussed below or may be used to update only the various aspects of the optimizations discussed above. As such, the potential of each of the updates to the above-described optimizations, including any specific combinations of updates described below for the above-described optimizations or the application of a single update described below for the above- Combinations are considered.

In order to specify the matrix in the bitstream, the bitstream generation unit 42 generates ID_CONFIG_EXT_HOA_MATRIX in mpegh3daConfigExtension () of the bitstream 21, for example as shown in the bold and highlighted table in the following table It can be specified. The following table represents the syntax for specifying the mpegh3daConfigExtension () portion of the bitstream 21:

Table - Syntax for mpegh3daConfigExtension ()

ID_CONFIG_EXT_HOA_MATRIX in the above table provides a container for specifying the rendering matrix, and the container is indicated as "HOARenderingMatrixSet () ".

The contents of the HOARenderingMatrixSet () container may be defined according to the syntax shown in the following table:

Table - Syntax of HOARenderingMatrixSet ()

As shown in the table immediately above, HOARenderingMatrixSet () includes a number of different syntax elements, including numHoaRenderingMatrices, HoaRendereringMatrixId, CICPspeakerLayoutIdx, HoaMatrixLenBits, and HoARenderingMatrix.

The numHoaRenderingMatrices syntax element may specify the number of HoaRendereringMatrixId definitions present in the bitstream element. The HoARenderingMatrixId syntax element may represent a field that uniquely defines Id for the default HOA rendering matrix or the transmitted HOA rendering matrix available on the decoder side. In this regard, the HoARenderingMatrixId may comprise a signal value or spherical harmonic coefficients, including two or more bits defining an index indicating that the bitstream includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds, May represent an example of a signal value that includes two or more bits that define an index associated with one of a plurality of matrices used to render into feeds. The CICPspeakerLayoutIdx syntax element may represent a value that describes the output loudspeaker layout for a given HOA rendering matrix, or may correspond to a ChannelConfiguration element defined in ISO / IEC 23000 1-8. The HoaMatrixLenBits syntax element (which may also be denoted as "HoARenderingMatrixLenBits") may specify the length of a subsequent bitstream element (e.g., a HoARenderingMatrix () container) in bits.

The HoARenderingMatrix () container contains NumOfHoaCoeffs followed by the outputConfig () and outputCount () containers. The outputConfig () container may contain channel configuration vectors that specify information about each loudspeaker. Bitstream generation unit 42 may assume that this loudspeaker information is known from the channel configurations of the output layout. Each entry outputConfig [i] may represent a data structure with the following members:

AzimuthAngle (may also represent the absolute value of the speaker azimuth);

AzimuthDirection (in one example, it may indicate the azimuth direction using 0 for the left and 1 for the right);

Elevation Angle (may also represent the absolute value of the speaker altitude angles);

ElevationDirection (in one example, it may indicate the altitude direction using 0 for the upper side and 1 for the lower side); And

isLFE (may also indicate whether the speaker is a low frequency effect (LFE) speaker).

Bitstream generation unit 42 may invoke, in some cases, a helper function indicated as "findSymmetricSpeakers ", which may further specify:

pairType (which may store the value of SYMMETRIC (which in some examples means a symmetric pair of two speakers), CENTER, or ASYMMETRIC); And

symmetricPair-> originalPosition (For SYMMETRIC groups only, it may indicate the position in the original channel configuration of the second (eg, right) speaker in the group).

The outputCount () container may specify the number of loudspeakers for which the HOA rendering matrix is defined.

The bitstream generation unit 42 may specify a HoARenderingMatrix () container according to the syntax presented in the following table:

Table - Syntax of HoARenderingMatrix ()

As shown in the table immediately above, the numPairs syntax element is set to the value output from calling the findSymmetricSpeakers helper function using outputCount, outputConfig, and hasLfeRendering as inputs. numPairs may thus indicate the number of symmetric loudspeaker pairs identified in the output loudspeaker setup that may be considered for efficient symmetric coding. The precisionLevel syntax element in the above table may also indicate the precision used for uniform quantization of the gains according to the following table:

Table - uniform quantization step size of hoaGain as a function of precisionLevel

The gainLimitPerHoaOrder syntax element shown in the above table that presents the syntax of HoARenderingMatrix () may represent a flag indicating whether maxGain and minGain are individually specified for each order or for the entire HOA rendering matrix. The maxGain [ i ] syntax elements, as an example, may specify the maximum real gain in the matrix for coefficients for the indicated HOA order i for decibels (dB). The minGain [ i ] syntax elements may again specify the minimum real gain in the matrix for the coefficients of the indicated HOA order i , in dB, as an example. The isFullMatrix syntax element may also represent a flag indicating whether the HOA rendering matrix is sparse or full. The firstSparseOrder syntax element may specify a rarely coded first HOA order if the HOA rendering matrix is specified to be rare for the isFullMatrix syntax element. The isHoaCoefSparse syntax element may represent a bit mask vector derived from the firstSparseOrder syntax element. The lfeExists syntax element may represent a flag indicating whether one or more LFEs are present in the outputConfig. The hasLfeRendering syntax element indicates whether the rendering matrix includes non-zero elements for one or more LFE channels. The zerothOrderAlwaysPositive syntax element may also represent a flag indicating whether the zeroth HOA order has only positive values.

The isAllValueSymmetric syntax element may represent a flag indicating whether all symmetric loudspeaker pairs have the same absolute values in the HOA rendering matrix. The isAnyValueSymmetric syntax element represents, for example, a flag indicating whether some of the symmetric loudspeaker pairs have the same absolute values in the HOA rendering matrix when, for example, it is false. The valueSymmetricPairs syntax element may represent a bit mask of length numPairs representing loudspeaker pairs with value symmetry. The isValueSymmetric syntax element may represent a bit mask derived from the valueSymmetricPairs syntax element in the manner shown in Table 3. The isAllSignSymmetric syntax element may indicate whether all symmetric loudspeaker pairs have at least number sign symmetries when there are no value symmetries in the matrix. The isAnySignSymmetric syntax element may represent a flag indicating whether there are at least some symmetric loudspeaker pairs with numeric symmetricities. The signSymmetricPairs syntax element may also represent a bit mask of length numPairs representing the pairs of loudspeakers symmetrically. The isSignSymmetric variable may represent a bit mask derived from the signSymmetricPairs syntax element in the manner shown above in a table that presents the syntax of HoARenderingMatrix (). The hasVerticalCoef syntax element may also represent a flag indicating whether the matrix is a horizontal dedicated HOA rendering matrix. The bootVal syntax element may represent a variable used in the decoding loop.

In other words, the bitstream generation unit 42 may generate a bitstream that is a combination of one or more syntax elements of the above value symmetry information (e.g., isAllValueSymmetric syntax element, isAnyValueSymmetric syntax element, valueSymmetricPairs syntax element, isValueSymmetric syntax element, and valueSymmetricPairs syntax element) ) Or otherwise analyze the audio renderer 1 to obtain the value symmetry information. Bitstream generation unit 42 may specify audio renderer information 2 in bitstream 21 in the manner shown above so that audio renderer information 2 includes value sign symmetry information.

Furthermore, the bitstream generation unit 42 may also generate the bitstream generation unit 42 in any combination of one or more syntax elements of the above sign symmetry information (e.g., isAllSignSymmetric syntax element, isAnySignSymmetric syntax element, signSymmetricPairs syntax element, isSignSymmetric syntax element, and signSymmetricPairs syntax element) ), Or otherwise analyze the audio renderer 1 to obtain sign symmetry information. Bitstream generation unit 42 may specify audio renderer information 2 in bitstream 21 in a manner shown above so that audio renderer information 2 includes audio code symmetry information.

When determining the value symmetry information and the code symmetry information, the bitstream generating unit 42 may analyze various values of the audio renderer 1, which may be specified as a matrix. The rendering matrix may be formulated as a pseudo-inverse of the matrix R. [ In other words, to render (N + 1) ² HOA channels (denoted as Z below) with L loudspeaker signals (denoted by column vector p of L loudspeaker signals), the following equation may be given :

Z = R * p.

To arrive at a rendering matrix that outputs L loudspeaker signals, the inverse of the R matrix is multiplied by Z HOA channels as shown in the following equation:

p = R- ¹ * Z.

As long as the number L of loudspeaker channels is not equal to the number (N + 1) ² of Z HOA channels, the matrix R may not be squared and a complete inverse may not be determined. As a result, a pseudo-inverse may be used instead, and this pseudo-inverse is defined as:

pinv (R) = R ^T (R * R ^T ) ^-1 ,

Where R ^T denotes the transpose of the R matrix. Substituting R ^-1 in the above equation, a solution to the L loudspeaker signals denoted by column vector p may be mathematically expressed as:

p = pinv (R) * Z = R ^T (R * R ^T ) ^-1 * Z.

The entries in the R matrix are the values of the spherical harmonics for the loudspeaker positions with (N + 1) ² rows for the different spherical harmonics and L columns for the speakers. The bitstream generation unit 42 may determine the loudspeaker pairs based on the values for the speakers. By analyzing the values of the spherical harmonics for the loudspeaker positions, the bitstream generation unit 42 determines which of the loudspeaker positions are pairs based on their values (e.g., the pairs are similar, May have the same value but have opposite signs).

After identifying the pairs, the bitstream generation unit 42 may, for each pair, determine whether the pairs have the same value or approximately the same value. When all of the pairs have the same value, the bitstream generating unit 42 may set the isAllValueSymmetric syntax element to one. Bitstream generation unit 42 may set the isAllValueSymmetric syntax element to zero when all of the pairs do not have the same value. Bitstream generating unit 42 may set the isAnyValueSymmetric syntax element to 1 when more than one but not all of the pairs have the same value. When none of the pairs have the same value, the bitstream generating unit 42 may set the isAnyValueSymmetric syntax element to zero. For pairs having symmetric values, the bitstream generation unit 42 only specifies one value rather than two distinct values for a pair of speakers, thereby causing the audio rendering information 2 ( For example, a matrix in this example).

When there are no value symmetries in the pairs, the bitstream generation unit 42 also determines for each pair that the pairs of speakers are symmetric (which means that one speaker has a negative value but the other speaker has a positive value Quot;) < / RTI > When both of the pairs have sign symmetry, the bitstream generating unit 42 may set the isAllSignSymmetric syntax element to one. When both of the pairs do not have sign symmetry, the bitstream generating unit 42 may set the isAllSignSymmetric syntax element to zero. Bitstream generation unit 42 may set the isAnySignSymmetric syntax element to 1 when more than one but not all of the pairs have sign symmetry. When none of the pairs has sign symmetry, the bitstream generation unit 42 may set the isAnySignSymmetric syntax element to zero. For pairs with symmetric codes, the bitstream generation unit 42 specifies only one sign or no code for the speaker pair, rather than two distinct codes, thereby identifying in the bitstream 21 The number of bits used to represent the audio rendering information 2 (e.g., a matrix in this example) may be reduced.

The bitstream generating unit 42 may specify the DecodeHoaMatrixData () container shown in the table that presents the syntax of HoARenderingMatrix () according to the syntax shown in the following table:

Table - Syntax for DecodeHoaMatrixData

The hasValue syntax element in the above table which presents the syntax of DecodeHoaMatrixData may represent a flag indicating whether the matrix element is rarely coded. The signMatrix syntax element, as an example, may represent a matrix having sign values of the HOA rendering matrix in a linearized vector-form. The hoaMatrix syntax element, as an example, may represent the HOA rendering matrix values in a linearized vector-form. The bitstream generating unit 42 may specify the DecodeHoaGainValue () container shown in the table that presents the syntax of DecodeHoaMatrixData according to the syntax shown in the following table:

Table - Syntax for DecodeHoaGainValue

The bitstream generation unit 42 may specify a readRange () container as indicated in the table presenting the syntax of DecodeHoaGainValue according to the syntax specified in the following table:

Table 7 - Syntax for ReadRange

Although not shown in the example of FIG. 3, the audio encoding device 20 is also configured to determine whether the current frame should be encoded using directional-based synthesis or vector-based synthesis (e.g., 21) and a vector-based bit stream (21)). The bit stream output unit may switch the bit stream output from the audio encoding device (20). The bitstream output unit determines whether directional-based compositing has been performed (as a detection result that the HOA coefficients 11 have been generated from the composite audio object) or whether vector-based compositing has been performed (as a result of detecting that the HOA coefficients have been recorded) Based on the syntax element output by the content analyzing unit 26, which represents the syntax element. The bitstream output unit may specify a correct header syntax for indicating the current encoding or switch used for the current frame, along with each one of the bitstreams 21.

Moreover, as mentioned above, the sound field analysis unit 44 may be configured to determine the BG _TOT (which may sometimes vary on a frame basis) (although the BG _TOT may remain constant or uniform over two or more (temporally) _May identify the HOA coefficients 47 around the _TOT . The change in BG _TOT may result in changes to the coefficients represented in the reduced foreground V [ k ] vectors 55. The change in BG _TOT (again, occasionally) may be due to background HOA coefficients (also referred to as "neighboring HOA coefficients ") that vary on a frame basis, although the BG _TOT may remain constant or identical over two or more (temporally) May also be referred to as " s "s). The changes are often made by adding or removing additional surrounding HOA coefficients and applying energy to aspects of the sound field represented by the corresponding elimination of coefficients from the reduced foreground V [ k ] vectors 55 or addition of coefficients thereto .

As a result, the sound field analysis unit 44 additionally determines when the surrounding HOA coefficients change between frames in terms of being used to represent the surrounding components of the sound field, and generates a flag or other syntax (Where the change may also be referred to as a "transition" of a surrounding HOA coefficient or a "transition" of a surrounding HOA coefficient). In particular, the coefficient reduction unit 46 generates a flag (which may be indicated as the AmbCoeffTransition flag or the AmbCoeffIdxTransition flag) and provides the flag to the bitstream generation unit 42 so that the flag (possibly as part of the side channel information) And may be included in the bitstream 21 as well.

The coefficient reduction unit 46 may also modify how the reduced foreground V [ k ] vectors 55 are generated, in addition to specifying the periphery coefficient transition flag. In one example, when determining that one of the neighboring HOA perimeter coefficients during the current frame is being transitioned, the coefficient reduction unit 46 calculates the reduced foreground V [ k ] vectors 55 corresponding to the surrounding HOA coefficients at transition Vectors (also referred to as "vector elements" or "elements") for each of the V- Again, the HOA coefficients around the transition may be added or removed from the BG _TOT total number of background factors. Thus, the resulting change in the total number of background coefficients is determined by whether the neighboring HOA coefficients are included or not in the bitstream, and whether the V-vectors specified in the bitstream in the second and third configuration modes described above Lt; RTI ID = 0.0 > V-vectors < / RTI > More information about how the coefficient reduction unit 46 may overcome changes in energy by specifying reduced foreground V [ k ] vectors 55 is described in US patent application Ser. "And is filed in U.S. Serial No. 14 / 594,533, filed January 12,2015.

FIG. 4 is a block diagram illustrating the audio decoding device 24 of FIG. 2 in more detail. 4, the audio decoding device 24 includes an extraction unit 72, a renderer reconstruction unit 81, a direction-based reconstruction unit 90 and a vector-based reconstruction unit 92 It is possible. Although described below, the various aspects of decompressing or otherwise decoding the HOA coefficients and more information about the audio decoding device 24 are described in "INTERPOLATION FOR DECOMPOSED REPRESENTATIONS OF A SOUND FIELD & International Patent Application Publication No. WO 2014/194099, filed on May 29th.

The extraction unit 72 receives the bitstream 21 and generates various encoded versions of the audio rendering information 2 and the HOA coefficients 11 (e.g., a directionally-based encoded version or a vector- Version) of the program. In other words, the higher order ambiance (HOA) rendering matrices may be transmitted by the audio encoding device 20 to enable control through the HOA rendering process in the audio reproduction system 16. Transmission may be facilitated by the mpegh3daConfigExtension of type ID_CONFIG_EXT_HOA_MATRIX shown above. The mpegh3daConfigExtension may include several HOA rendering matrices for different loudspeaker playback configurations. When the HOA rendering matrices are transmitted, the audio encoding device 20 signals, for each HOA rendering matrix signal, the associated target loudspeaker layout that together determine the dimensions of the rendering matrix with HoaOrder.

The transmission of the unique HoARenderingMatrixId makes it possible to reference the default HOA rendering matrix available in the audio reproduction system 16, or the HOA rendering matrix transmitted from outside the audio bitstream 21. In some cases, all of the HOA rendering matrices are assumed to be normalized in N3D and follow the ordering of HOA coefficients as defined in bitstream 21.

The function findSymmetricSpeakers may represent, as one example, the position and number of all loudspeaker pairs in a given loudspeaker setup that is symmetric with respect to the midpoint of the listener in the so-called "sweet spot ", as mentioned above. This helper function may be defined as:

The extraction unit 72 may call the function createSymSigns to compute vectors of 1.0 and -1.0 values, which may then be used to generate matrix elements associated with symmetric loudspeakers. This createSymSigns function can also be defined as:

The extraction unit 72 may call the function create2dBitmask to generate a bit mask for identifying the HOA coefficients that are only used in the horizontal plane. The create2dBitmask function can also be defined as:

To decode the HOA rendering matrix coefficients, the extraction unit 72 may first extract the syntax element HOARenderingMatrixSet (), which may be applied to achieve HOA rendering for the desired loudspeaker layout, as noted above Lt; RTI ID = 0.0 > HOA < / RTI > In some cases, a given bitstream may not contain more than one instance of HOARenderingMatrixSet (). The syntax element HoARenderingMatrix () includes the HOA rendering matrix information (which may be represented as the renderer information (2) in the example of FIG. 4). The extraction unit 72 may first read the config information, which may guide the decoding process. Thereafter, the extraction unit 72 reads the matrix elements accordingly.

In some cases, the extraction unit 72 reads the fields precisionLevel and gainLimitPerOrder at the starting point. When the flag gainLimitPerOrder is set, the extraction unit 72 reads and decodes the maxGain and minGain fields separately for each HOA order. When the flag gainLimitPerOrder is not set, the extraction unit 72 reads and decodes the fields maxGain and minGain once, and applies these fields to all HOA orders during the decoding process. In some cases, the minGain value should be between 0db and -69dB. In some cases, the maxGain value should be between 1dB and 111dB lower than the minGain value. Figure 9 is a diagram illustrating an example of HOA order dependent minimum and maximum gains in the HOA rendering matrix.

The extraction unit 72 may then read the flag isFullMatrix which may signal whether the matrix is defined as being full or partially sparse. When the matrix is defined as being partially sparse, the extraction unit 72 reads the next field (e.g., the firstSparseOrder syntax element) specifying the HOA order that causes the HOA rendering matrix to be rarely coded. The HOA rendering matrices are often dense for lower orders and may be scarce in higher orders, depending on the loudspeaker reproduction setup. 10 is a diagram illustrating a partial sparse 6th order HOA rendering matrix for 22 loudspeakers. The sparseness of the matrix shown in FIG. 10 starts at the 26th HOA coefficient (HOA order 5).

Depending on whether one or more low frequency effect (LFE) channels (represented by the lfeExists syntax element) are present in the loudspeaker reproduction setup, the extraction unit 72 may read the field hasLfeRendering. When hasLfeRendering is not set, the extraction unit 72 is configured to assume that the matrix elements associated with the LFE channels are digital zeros. The next field read by the extraction unit 72 is the flag zerothOrderAlwaysPositive which signals whether the matrix elements associated with the zeroth coefficient are positive. In this case, where the zerothOrderAlwaysPositive indicates that the zeroth order HOA coefficients are positive, the extraction unit 72 determines that the numeric codes are not coded for the render matrix coefficients corresponding to the zeroth order HOA coefficients.

Next, the attributes of the HOA rendering matrix may be signaled for pairs of loudspeakers that are symmetric with respect to the median plane. In some cases, there are two symmetric properties related to a) value symmetry and b) sign symmetry. In the case of value symmetry, the matrix elements of the left loudspeaker of the symmetric loudspeaker pair are not coded, but rather the extraction unit 72 forms the decoded matrix elements of the right loudspeaker by employing a helper function createSymSigns These elements are derived from:

If the loudspeaker pairs are not symmetric, then the matrix elements may be symmetrical with respect to their numerical signatures. When the loudspeaker pairs are symmetrical, the numeric signs of the matrix elements of the left loudspeaker of the symmetrical loudspeaker pair are not coded, and the extraction unit 72 can use the helper function createSymSigns to: Derive these numeric codes from the numeric codes of the matrix elements:

11 is a diagram illustrating signaling of symmetric attributes. Loudspeaker pairs can not be defined as simultaneously symmetrical and symmetrical. The final decoding flag hasVerticalCoef specifies whether only the matrix elements associated with circular (i.e., 2D) HOA coefficients are coded. If hasVerticalCoef is not set, the matrix elements associated with the HOA coefficients defined by the helper function create2dBitmask are set to digital zero.

That is, the extraction unit 72 may extract the audio rendering information 2 according to the process shown in FIG. The extraction unit 72 may first read the isAllValueSymmetric syntax element from the bitstream 21 (300). When the isAllValueSymmetric syntax element is set to 1 (or, in other words, Boolean true), the extraction unit 72 repeats the value of the numPairs syntax element and sets the valueSymmetricPairs array syntax element to a value of 1 Effectively indicating that both pairs are value symmetric).

When the isAllValueSymmetric syntax element is set to 0 (or, in other words, Boolean false), the extraction unit 72 may then read the isAnyValueSymmetric syntax element (304). When the isAnyValueSymmetric syntax element is set to 1 (or, in other words, not true), the extraction unit 72 iterates the value of the numPairs syntax element so that the valueSymmetricPairs array syntax element is replaced with a bit that is read out sequentially from the bitstream 21 (306). The extraction unit 72 may also obtain (308) an isAnySignSymmetric syntax element for any of the pairs having a valueSymmetricPairs syntax element set to zero. The extraction unit 72 then repeats the multiple pairs again and may set the signSymmetricPairs bit to the value read from the bitstream 21 when the valueSymmetricPairs equals zero.

The extraction unit 72 may read 312 the isAllSignSymmetric syntax element from the bitstream 21 when the isAnyValueSymmetric syntax element is set to zero (or, in other words, false). When the isAllSignSymmetric syntax element is set to a value of 1 (or, in other words, not true), the extraction unit 72 repeats the value of the numPairs syntax element and sets the valueSymmetricPairs array syntax element to a value of 1 Lt; / RTI > is effectively symmetric) (316).

The extraction unit 72 may read (316) the isAnySignSymmetric syntax element from the bitstream 21 when the isAllSignSymmetric syntax element is set to zero (or, in other words, false). The extraction unit 72 may repeat the value of the numPairs syntax element to set the signSymmetricPairs array syntax element to bits that are read out sequentially from the bitstream 21 (318). Bitstream generation unit 42 may perform a process that is contrary to that described above with respect to extraction unit 72 to specify a combination of value symmetry information, code symmetry information, or both a value and sign symmetry information.

The renderer reconstruction unit 81 may represent a unit configured to reconstruct the renderer based on the audio rendering information (2). That is, using the above-mentioned properties, the renderer reconstruction unit 81 may read a set of matrix element gain values. To read the absolute gain value, the renderer reconstruction unit 81 may call the function DecodeGainValue (). The renderer reconstruction unit 81 may call the function ReadRange () of the alphabet index to uniformly decode the gain value. When the decoded gain value is not digital zero, the renderer reconstruction unit 81 may additionally read the numerical sign value (per table below). When a matrix element is associated with a HOA coefficient signaled to be scarce (via isHoaCoefSparse), the hasValue flag precedes gainValueIndex (see table b). When the hasValue flag is 0, this element is set to digital zero and no gainValueIndex and sign are signaled.

Examples of bitstream syntax for decoding tables a and b - matrix elements

Depending on the symmetry properties specified for the loudspeaker pairs, the renderer reconstruction unit 81 may derive the matrix elements associated with the left loudspeaker from the right loudspeaker. In this case, the audio rendering information 2 in the bitstream 21 for decoding the matrix elements for the left loudspeaker is accordingly reduced or potentially omitted altogether.

In this way, the audio decoding device 24 may determine the symmetry information and reduce the size of the audio rendering information to be specified. In some cases, the audio decoding device 24 may determine the symmetry information to reduce the size of the audio rendering information to be specified and derive at least a portion of the audio renderer based on the symmetry information.

In these and other cases, the audio decoding device 24 may determine the value symmetry information to reduce the size of the audio rendering information to be specified. In these and other cases, the audio decoding device 24 may derive at least a portion of the audio renderer based on the value symmetry information.

In these and other cases, the audio decoding device 24 may determine the code symmetry information to reduce the size of the audio rendering information to be specified. In these and other cases, the audio decoding device 24 may derive at least a portion of the audio renderer based on the sign symmetry information.

In these and other cases, the audio decoding device 24 may determine scarcity information indicative of the scarcity of the matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In these and other cases, the audio decoding device 24 may determine the speaker layout in which the matrix should be used to render the spherical harmonic coefficients into a plurality of speaker feeds.

The audio decoding device 24 may, in this regard, determine audio rendering information (2) that is then specified in the bitstream. Based on the signal values contained in the audio rendering information 2, the audio playback system 16 may render a plurality of speaker feeds 25 using one of the audio renderers 22. The speaker feeds may also drive the speakers 3. As mentioned above, the signal value may include a matrix (which is decoded and provided as one of the audio renderers 22) used to render the spherical harmonic coefficients in a plurality of speaker feeds in some cases. In this case, the audio playback system 16 may configure one of the audio renderers 22 as a matrix to provide the audio renderer 22 with one of the audio renderers 22 to render the speaker feeds 25 based on the matrix. May be used.

The extraction unit 72 extracts the various encoded versions of the HOA coefficients 11 so as to enable the HOA coefficients 11 to be rendered using the obtained audio renderer 22, 11 may be determined from the above-mentioned syntax elements indicating whether they have been encoded through various direction-based or vector-based versions. When directional-based encoding has been performed, the extraction unit 72 determines whether the syntax elements associated with the encoded version (indicated as directional-based information 91 in the example of FIG. 4) and the directionality of the HOA coefficients 11 Based version 91 and forward the directional-based information 91 to the directional-based reconstruction unit 90. The direction- The directional-based reconstruction unit 90 may represent a unit configured to reconstruct HOA coefficients in the form of HOA coefficients 11 'based on the directional-based information 91.

When the syntax element indicates that the HOA coefficients 11 have been encoded using vector-based decomposition, the extraction unit 72 extracts the coded foreground V [ k ] vectors 57 (coded weights 57 and / Encoded indexes 63 or scalar quantized V-vectors), encoded neighboring HOA coefficients 59 and corresponding audio objects 61 (also referred to as encoded nFG signals 61) May be extracted. Each of the audio objects 61 corresponds to one of the vectors 57. The extraction unit 72 transfers the coded foreground V [ k ] vectors 57 to the V-vector reconstruction unit 74 and the encoded neighboring HOA coefficients 59 along with the encoded nFG signals 61 To the psychoacoustic decoding unit 80. [

The V-vector reconstruction unit 74 may represent a unit configured to reconstruct the V-vectors from the encoded foreground V [ k ] vectors 57. The V-vector reconstruction unit 74 may operate in a manner incompatible with that of the quantization unit 52.

The psychoacoustic decoding unit 80 decodes the encoded neighboring HOA coefficients 59 and the encoded nFG signals 61 thereby generating energy-compensated neighboring HOA coefficients 47 'and interpolated nFG signals 49 (Which may also be referred to as interpolated nFG audio objects 49 ') (also referred to as interpolated nFG audio objects 49'). Psychoacoustic decoding unit 80 may communicate energy compensated neighboring HOA coefficients 47 'to fade unit 770 and nFG signals 49' to foreground formulation unit 78.

The space-time interpolation unit 76 may operate in a manner similar to that described above with respect to the space-time interpolation unit 50. Space-time interpolation unit 76 decreases the foreground V [k] vector s (55 _k) for receiving and views V [k] vector s (55 _k) and the reduced view V [k -1] vector ( It may produce the perform time interpolation to the interpolated foreground V [k] vector _{(55 k '') - 55} k -1) with respect to space. The space-time interpolation unit 76 may forward the interpolated foreground V [ k ] vectors _55k " to the fade unit 770. [

Extraction unit 72 also may output a signal 757 that indicates when a transition of the peripheral HOA coefficient to fading unit 770, the fading unit Thereafter, the interpolated foreground V [k] vector s (55 _k '') elements, and SHC _BG (47 a ') (wherein SHC _BG (47') is also "peripheral HOA channel (47 'appear as)" or "peripheral HOA coefficient (47)." May be determined to be fade-in or fade-out. In some instances, the fade unit 770 may operate inversely with respect to each of the elements of the interpolated foreground V [ k ] vectors 55 _k " and the surrounding HOA coefficients 47 '. That is, the fade unit 770 fades-in or fade-out or fades-in and phase-out both for the corresponding one of the elements of the interpolated foreground V [ k ] vectors _55k " In or fade-out, or both fade-in or phase-out, with respect to a corresponding one of the neighboring HOA coefficients 47 ', while performing the same operation. The fade unit 770 outputs the adjusted foreground V [ k ] vectors _55k '''to the foreground formulating unit 78 ). In this regard, the fading unit 770 is, for example, the interpolated foreground V [k] vector s (55 _k '') of the elements and the peripheral HOA coefficients (47 in the form of a), the HOA coefficient or Expresses a unit configured to perform a fade operation with respect to various aspects of its derivatives.

The foreground formulation unit 78 performs the matrix multiplication with respect to the adjusted foreground V [k] vector (55 _k ''') and interpolated in nFG signal (49) to generate a foreground HOA coefficient 65 And may represent a configured unit. In this regard, the foreground formatter unit 78 combines the audio objects 49 '(which is another way to display the interpolated nFG signals 49') with vectors 55 _k ''' The foreground of the coefficients 11 ', or, in other words, dominant aspects. Foreground formulation unit 78 may perform matrix multiplication of 'nFG interpolated signal by a (49 in the adjusted foreground V [k] vector _(k 55'')").

The HOA coefficient formulation unit 82 may represent a unit configured to combine the foreground HOA coefficients 65 with the adjusted neighboring HOA coefficients 47 '' to obtain the HOA coefficients 11 '. The main notation reflects that HOA coefficients 11 'are similar to HOA coefficients 11 but may not be the same. Differences between HOA coefficients 11 and 11 'may result from loss due to transmissions through lossy transmission media, quantization or other lossy operations.

In addition, the extraction unit 72 and the audio decoding device 24 may more generally include, in certain cases, a potentially optimized bitstream May also be configured to operate in accordance with various aspects of the techniques described in this disclosure to acquire < RTI ID = 0.0 >

In some cases, the audio decoding device 24 may be configured to support a second compression scheme that is also used to compress higher order ambience sound data when decompressing higher order ambience sound data compressed using the first compression scheme To obtain a bitstream 21 that represents a compressed version of higher order ambience sound data that does not include the bits that are < / RTI > The first compression scheme may comprise a vector-based compression scheme, and the resulting vector is defined in the spherical harmonic domain and transmitted via bitstream 21. The vector-based decomposition compression scheme may, in some instances, include a compression scheme involving the application of singular value decomposition (or equivalents thereof as described in more detail with respect to the example of FIG. 3) to higher order ambience sound data have.

The audio decoding device 24 may be configured to obtain a bit stream 21 that does not include the bits corresponding to at least one syntax element used to perform the second type of compression scheme. As mentioned above, the second compression scheme includes a direction-based compression scheme. More specifically, the audio decoding device 24 may be configured to obtain a bitstream 21 that does not include the bits corresponding to the HOAPredictionInfo syntax elements of the second compression scheme. In other words, when the second compression scheme includes a directional-based compression scheme, the audio decoding device 24 is configured to obtain a bitstream 21 that does not contain the bits corresponding to the HOAPredictionInfo syntax element of the directional-based compression scheme . As noted above, the HOAPredictionInfo syntax element may represent a prediction between two or more directional-based signals.

In some cases, alternatively or in conjunction with the foregoing examples, the audio decoding device 24 may be configured to generate a high-order ambience sound data that does not include gain correction data when the gain correction is suppressed during compression of the high- Lt; RTI ID = 0.0 > 21 < / RTI > The audio decoding device 24, in these cases, may be configured to decompress high order ambience sound data according to a vector-based composite decompression scheme. A compressed version of higher order ambsonic data is generated through the application of singular value decomposition (or equivalents thereof described in greater detail with respect to the example of FIG. 3 above) to higher order ambsonic audio data. When the SVD or its equivalents are applied to the HOA audio data, the audio encoding device 20 specifies the resulting vectors or at least one of the bits representing it in the bitstream 21, where the vectors correspond to the corresponding foreground audio objects Spatial properties (e.g., width, location, and volume of corresponding foreground audio objects).

More specifically, the audio decoding device 24 may be configured to obtain from the bitstream 21 a MaxGainCorrAmbExp syntax element having a value set to zero to indicate that the gain correction is suppressed. That is, the audio decoding device 24 may be configured to obtain a bitstream such that when the gain correction is suppressed, the bitstream does not include a HOAGainCorrection data field that stores the gain correction data. Bitstream 21 may include a MaxGainCorrAmbExp syntax element having a value of zero to indicate that the gain correction is suppressed and does not include the HOAGainCorrection data field storing the gain correction data. The suppression of gain correction may occur when the compression of higher order ambsonic audio data involves the application of unified voice and audio and speech coding (USAC) to higher order ambsonic audio data.

5 is a flow chart illustrating an exemplary operation of an audio encoding device, such as the audio encoding device 20 shown in the example of FIG. 3, in performing various aspects of the vector-based synthesis techniques described in this disclosure . Initially, the audio encoding device 20 receives the HOA coefficients 11 (106). The audio encoding device 20 may invoke the LIT unit 30 and this LIT unit may apply the LIT with respect to the HOA coefficients to output the transformed HOA coefficients (e.g., in the case of SVD, The HOA coefficients may include US [ k ] vectors 33 and V [ k ] vectors 35).

The audio encoding device 20 then uses the US [ k ] vectors 33, US [ k- 1] vectors 33, V [ k ], and / or the like to identify various parameters in the manner described above May invoke the parameter calculation unit 32 to perform the analysis described above with respect to any combination of V [ k -1] vectors 35. That is, the parameter calculation unit 32 may determine at least one parameter based on the analysis of the transformed HOA coefficients 33/35 (108).

The audio encoding device 20 may then call the reorder unit 34 which in turn transforms the transformed HOA coefficients (again US [ k ] vectors 33 in the context of the SVD) (Or, in other words, a US [ k ] vector 35, which may be referred to as V [ k ] vectors 35) (33 ') and V [ k ] vectors 35') (109). The audio encoding device 20 may also call the sound field analyzing unit 44 during any of the above described operations or subsequent operations. The sound field analysis unit 44 performs sound field analysis on the HOA coefficients 11 and / or the transformed HOA coefficients 33/35 to determine the total of the foreground channels nFG 45 the number, the order of the indexes of the additional BG HOA channel transfer (may be displayed collectively in a background channel information 43. in the example of FIG. 3) (i) and the number (nBGa) and the background field (N _BG) (109).

The audio encoding device 20 may also invoke the background selection unit 48. Background selection unit 48 may determine background or neighbor HOA coefficients 47 based on background channel information 43 (110). The audio encoding device 20 may additionally call a foreground selection unit 36 which selects the foreground of the sound field based on the nFG 45 (which may represent one or more indices identifying foreground vectors) Or a reordered US [ k ] matrix 33 'that represents the distinguished components and a reordered V [ k ] matrix 35' and a reordered US [ k ] matrix 33 '.

The audio encoding device 20 may call the energy compensation unit 38. [ The energy compensation unit 38 performs energy compensation with respect to the neighboring HOA coefficients 47 to compensate for the energy loss due to the removal of various HOA coefficients by the background selection unit 48 (114) Energy-compensated neighboring HOA coefficients 47 '.

The audio encoding device 20 may also invoke the space-time interpolation unit 50. The spatial-temporal interpolation unit 50 performs spatial-temporal interpolation on the reoriented transformed HOA coefficients 33 '/ 35' to generate interpolated foreground signals 49 '(also referred to as "interpolated nFG signals (Which may also be referred to as "V [ k ] vectors 49") and the remaining foreground directional information 53 (which may also be referred to as "V [ k ] vectors 53"). The audio encoding device 20 may then call the coefficient reduction unit 46. [ The coefficient reduction unit 46 performs a coefficient reduction on the remaining foreground V [ k ] vectors 53 based on the background channel information 43 to obtain reduced foreground direction information 55 (also reduced foreground V [ k] May also be referred to as vectors 55 (118).

The audio encoding device 20 then calls the quantization unit 52 to compress the reduced foreground V [ k ] vectors 55 and the coded foreground V [ k ] vectors 57 in the manner described above, (120).

The audio encoding device 20 may also call the psychoacoustic audio coder unit 40. Psychoacoustic audio coder unit 40 psychoacoustically codes each vector of energy-compensated neighboring HOA coefficients 47 'and interpolated nFG signals 49' to produce encoded neighboring HOA coefficients 59 and encoding Lt; RTI ID = 0.0 > nFG < / RTI > The audio encoding device may then call the bitstream generation unit 42. [ Bitstream generation unit 42 generates bitstream 21 based on coded foreground directional information 57, coded peripheral HOA coefficients 59, coded nFG signals 61 and background channel information 43, May be generated.

FIG. 6 is a flow chart illustrating exemplary operation of an audio decoding device, such as the audio decoding device 24 shown in FIG. 4, in performing various aspects of the techniques described in this disclosure. Initially, the audio decoding device 40 may receive the bitstream 21 (130). Upon receipt of the bitstream, the audio decoding device 24 may call the extraction unit 72. Assuming for the purposes of the discussion that the bitstream 21 indicates that a vector-based reconstruction is to be performed, the extraction unit 72 parses the bitstream to extract the information referred to above, Unit 92 as shown in FIG.

In other words, the extracting unit 72 extracts the foreground directional information 57 (again, also referred to as coded foreground V [ k ] vectors 57) coded from the bit stream 21 in the manner described above, (Which may also be referred to as coded foreground nFG signals 59 or coded foreground audio objects 59) of coded foreground HAs coefficients 59 and coded foreground signals 59 (also referred to as coded foreground nFG signals 59 or coded foreground audio objects 59) .

The audio decoding device 24 may additionally call the dequantization unit 74. [ Quantization off unit 74 may obtain a coded foreground directional information (57) entropy decoding and reduced by turning off the quantization view direction information (55 _k) (136). The audio decoding device 24 may also call the psychoacoustic decoding unit 80. The psychoacoustic audio decoding unit 80 decodes the encoded neighboring HOA coefficients 59 and the encoded foreground signals 61 to produce energy compensated neighboring HOA coefficients 47 'and interpolated foreground signals 49' (138). Psychoacoustic decoding unit 80 may communicate energy compensated neighboring HOA coefficients 47 'to fade unit 770 and nFG signals 49' to foreground formulation unit 78.

The audio decoding device 24 may then invoke the space-time interpolation unit 76. [ The spatial-temporal interpolation unit 76 receives the reordered foreground directional information _55k 'and performs space-time interpolation on the reduced foreground directional information _55k / _{55k- 1} to generate interpolated foreground directional Information _55k '' (140). The space-time interpolation unit 76 may forward the interpolated foreground V [ k ] vectors _55k " to the fade unit 770. [

The audio decoding device 24 may call the fade unit 770. [ Fade unit 770 may receive (e.g., from extraction unit 72) syntax elements (e.g., AmbCoeffTransition syntax element) indicating when energy-compensated neighboring HOA coefficients 47 'are in transition Or otherwise acquire. The fade unit 770 fades in or fades out the energy-compensated neighboring HOA coefficients 47 'based on the transition syntax elements and the held transition state information to adjust the adjusted neighboring HOA coefficients 47'') To the HOA coefficient formulation unit 82. [ Fade unit 770 is also based on the syntax elements and the held transition state information and fades out or fades in corresponding elements of the interpolated foreground V [ k ] vectors _55k " And output the adjusted foreground V [ k ] vectors _55k '''to the foreground formulation unit 78 (142).

The audio decoding device 24 may invoke the foreground formulation unit 78. [ Foreground formulation unit 78 is to perform a matrix multiplication of '(nFG signals 49) by the adjusted view direction information (55 _k' ')' may obtain the foreground HOA coefficient (65, 144). The audio decoding device 24 may also invoke the HOA coefficient formulation unit 82. The HOA coefficient formulation unit 82 may obtain 146 the HOA coefficients 11 'by adding the foreground HOA coefficients 65 to the adjusted neighboring HOA coefficients 47''.

FIG. 7 is a flow chart illustrating exemplary operation of a system, such as the system 10 shown in the example of FIG. 2, in performing various aspects of the techniques described in this disclosure. As discussed above, the content creator device 12 employs an audio editing system 18 to create or edit the captured or generated audio content (shown as HOA coefficients 11 in the example of FIG. 2) You may. The content creator device 12 may then render 200 the HOA coefficients 11 using the audio renderer 1 for the generated multi-channel speaker feeds, as discussed in more detail above. . The content creator device 12 then uses these audio playback systems to play back these speaker feeds and may determine whether further adjustments or editing are required, as an example, to capture the desired artistic intention 202 ). The content creator device 12 remixes 204 the HOA coefficients 11, renders 200 the HOA coefficients 11, and then makes further adjustments (e.g., (202). ≪ / RTI > When no further adjustments are desired ("NO" 202), the audio encoding device 20 may encode (206) the audio content to produce the bitstream 21 in the manner described above with respect to the example of FIG. The audio encoding device 20 may also generate and specify 208 the audio rendering information 2 in the bitstream 21, as described in more detail above.

The content consumer device 14 may then obtain audio rendering information 2 from the bitstream 21 (210). The decoding device 24 may then decode the bitstream 21 to obtain audio content (shown as HOA coefficients 11 'in the example of FIG. 2) in the manner described above with respect to the example of FIG. 6 (211). The audio playback system 16 then renders the HOA coefficients 11 'based on the audio rendering information 2 in the manner described above 212 and plays the rendered audio content through the loudspeakers 3 (214).

The techniques described in this disclosure may accordingly enable, as a first example, a device that generates a bitstream representing multi-channel audio content to specify audio rendering information. The device may, in this first example, comprise means for specifying audio rendering information including signal values identifying an audio renderer used when generating multi-channel audio content.

In the device of the first example, the signal value includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In a second example, in the device of the first example, the signal value includes two or more bits defining an index indicating that the bitstream includes a matrix used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In the device of the second example, the audio rendering information further includes two or more bits defining the number of rows of the matrix included in the bitstream and two or more bits defining the number of columns of the matrix included in the bitstream do.

In the device of the first example, the signal value specifies a rendering algorithm used to render audio objects into a plurality of speaker feeds.

In the device of the first example, the signal value specifies a rendering algorithm used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In the device of the first example, the signal value includes two or more bits defining an index associated with one of the plurality of matrices used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In the device of the first example, the signal value comprises two or more bits defining an index associated with one of a plurality of rendering algorithms used to render audio objects into a plurality of speaker feeds.

In the device of the first example, the signal value comprises two or more bits defining an index associated with one of a plurality of rendering algorithms used to render the spherical harmonic coefficients into a plurality of speaker feeds.

In the device of the first example, the means for specifying audio rendering information includes means for specifying audio rendering information on an audio frame basis in the bitstream.

In the device of the first example, the means for specifying the audio rendering information includes means for specifying the audio rendering information in the bitstream a single time.

In a third example, when executed, one or more processors store instructions that cause audio processors to specify audio rendering information in a bitstream, wherein the audio rendering information is generated to generate multi-channel audio content Identifies the audio renderer to be used.

In a fourth example, a device for rendering multi-channel audio content from a bitstream comprises means for determining audio rendering information comprising a signal value identifying an audio renderer to be used in generating multi-channel audio content, And means for rendering a plurality of speaker feeds based on audio rendering information specified in the bitstream.

In a fourth example device, the signal value includes a matrix used to render spherical harmonic coefficients into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds comprises means for rendering a plurality of speaker feeds based on the matrix .

In a fifth example, in the device of the fourth example, the signal value includes two or more bits defining an index indicating that the bitstream includes a matrix used to render spherical harmonic coefficients into a plurality of speaker feeds, The device further comprises means for parsing a matrix from the bitstream in response to the index and the means for rendering the plurality of speaker feeds comprises means for rendering a plurality of speaker feeds based on the parsed matrix.

In the device of the fifth example, the signal value further comprises two or more bits defining the number of rows of the matrix included in the bitstream and two or more bits defining the number of columns of the matrix included in the bitstream , The means for parsing the matrix from the bitstream comprises parsing the matrix from the bitstream in response to the index and based on two or more bits defining two or more bits and the number of columns defining the number of rows Means.

In the device of the fourth example, the signal value specifies a rendering algorithm used to render audio objects into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds comprises a plurality Lt; RTI ID = 0.0 > speaker feeds. ≪ / RTI >

In the device of the fourth example, the signal value specifies a rendering algorithm used to render the spherical harmonic coefficients into a plurality of speaker feeds, and the means for rendering the plurality of speaker feeds uses spherical harmonic coefficients And means for rendering a plurality of speaker feeds.

In a fourth example device, the signal value includes two or more bits defining an index associated with one of a plurality of matrices used to render spherical harmonic coefficients into a plurality of speaker feeds, The means for rendering a plurality of speaker feeds from spherical harmonic coefficients using one of a plurality of matrices associated with the index.

In a fourth example device, the signal value comprises two or more bits defining an index associated with one of a plurality of rendering algorithms used to render audio objects into a plurality of speaker feeds, The means for rendering a plurality of speaker feeds from the audio objects using one of a plurality of rendering algorithms associated with the index.

In the device of the fourth example, the signal value comprises two or more bits defining an index associated with one of a plurality of rendering algorithms used to render spherical harmonic coefficients into a plurality of speaker feeds, The means for rendering includes means for rendering a plurality of speaker feeds from the spherical harmonic coefficients using one of a plurality of rendering algorithms associated with the index.

In the device of the fourth example, the means for determining audio rendering information includes means for determining audio rendering information from an audio stream based on an audio frame.

In the device of the fourth example, the means for determining audio rendering information comprises means for single-rounding audio rendering information from the bitstream.

In a sixth example, the non-transitory computer readable storage medium, when executed, causes one or more processors to determine audio rendering information including signal values identifying an audio renderer to be used in generating multi-channel audio content ; And to render a plurality of speaker feeds based on audio rendering information specified in the bitstream.

8A-8D are diagrams illustrating bitstreams 21A-21D formed in accordance with the techniques described in this disclosure. In the example of Fig. 8A, the bit stream 21A may represent one example of the bit stream 21 shown in Figs. 2 to 4 above. Bitstream 21A includes audio rendering information 2A that includes one or more bits that define a signal value 554. This signal value 554 may represent any combination of the types of information described below. Bitstream 21A also includes audio content 558, which may represent one example of audio content 7/9.

In the example of FIG. 8B, bitstream 21B may be similar to bitstream 21A, where the signal value 554 of audio rendering information 2B includes an index 554A, a row size 554B of the signaled matrix One or more bits defining a column size 554C of the signaled matrix, and matrix coefficients 554D. Index 554A may be defined using two of the five bits while row size 554B and column size 554C may each be defined using two to sixteen bits.

The extraction unit 72 may extract an index 554A and determine whether the index signals that the matrix is included in the bitstream 21B (where predetermined index values, e.g., 0000 or 1111, (I. E., ≪ / RTI > explicitly specified in step 21B). In the example of FIG. 8B, the bitstream 21B includes an index 554A that signals that the matrix is explicitly specified in the bitstream 21B. As a result, the extraction unit 72 may extract the row size 554B and the column size 554C. The extraction unit 72 is configured to determine the number of bits to parse representing the matrix coefficients as a function of row size 554B, column size 554C and signaled (not shown in FIG. 8A) or implied bit size of each matrix coefficient . Using the determined number of bits, the extraction unit 72 may extract the matrix coefficients 554D, while the audio playback system 16 may be used to construct one of the audio renderers 22 as described above It is possible. Audio rendering information 2B may be encoded in bit stream 21B multiple times or at least partially or entirely in a separate distinct out-of-band channel Lt; / RTI > may be signaled as optional data in some cases).

In the example of Fig. 8C, the bit stream 21C may represent one example of the bit stream 21 shown in Figs. 2 to 4 above. The bit stream 21C includes audio rendering information 2C that includes a signal value 554 that specifies an algorithm index 554E in this example. Bitstream 21C also includes audio content 558. [ Algorithm index 554E may be defined using two to five bits, as noted above, where algorithm index 554E identifies the rendering algorithm to be used when rendering audio content 558 You may.

The extraction unit 72 may extract the algorithm index 550E and determine whether the algorithm index 554E signals that the matrix is included in the bitstream 21C where certain index values, e.g., 0000 or 1111, The matrix may be signaled that it is explicitly specified in the bit stream 21C). In the example of FIG. 8C, the bitstream 21C includes an algorithm index 554E that signals that the matrix is not explicitly specified in the bitstream 21C. As a result, the extraction unit 72 forwards the algorithm index 554E to the audio reproduction system 16 which, if available (as the renderers 22 in the example of Figures 2 to 4) Selects a corresponding one of the rendering algorithms. 8C, the audio rendering information 2C may be generated in the bitstream 21C a number of times or at least partially or completely independent of the audio rendering information 2C in the bitstream 21C, And may be signaled in the out-of-band channel (in some cases as optional data).

In the example of Fig. 8D, the bit stream 21D may represent one example of the bit stream 21 shown in Figs. 2 to 4 above. The bit stream 21D includes audio rendering information 2D that includes a signal value 554 that specifies a matrix index 554F in this example. Bitstream 21D also includes audio content 558. [ The matrix index 554F may be defined using two to five bits as described above wherein the matrix index 554F identifies the rendering algorithm to be used when rendering the audio content 558 You may.

The extraction unit 72 may extract the matrix index 550F and determine whether the matrix index 554F signals that the matrix is included in the bitstream 21D where certain index values, e.g., 0000 or 1111, The matrix may be signaled that it is explicitly specified in the bit stream 21C). In the example of FIG. 8D, the bitstream 21D includes a matrix index 554F that signals that the matrix is not explicitly specified in the bitstream 21D. As a result, the extraction unit 72 forwards the matrix index 554F to the audio reproduction device, which selects the corresponding one of the renderers 22 (if available). 8D, the audio rendering information 2D may be encoded in the bitstream 21D a number of times or at least partially or completely independent of the audio rendering information 2D in the bitstream 21D, And may be signaled in the out-of-band channel (in some cases as optional data).

8E-8G are diagrams illustrating portions of bitstream or side channel information that may specify compressed spatial components in more detail. FIG. 8E illustrates a first example of a frame 249A 'of the bitstream 21. In the example of FIG. 8E, frame 249A 'includes ChannelSideInfoData (CSID) fields 154A through 154C, HOAGainCorrectionData (HOAGCD) fields, and VVectorData fields 156A and 156B. CSID field 154A includes unitC 267, bb 266 and ba 265 along with ChannelType 269, each of which corresponds to the corresponding values 01, 1, 0, and 01 shown in the example of FIG. Lt; / RTI > CSID field 154B includes unitC 267, bb 266 and ba 265 along with ChannelType 269, each of which corresponds to the corresponding values 01, 1, 0, and 01 shown in the example of FIG. Lt; / RTI > The CSID field 154C includes a ChannelType field 269 having a value of three. Each of the CSID fields 154A through 154C corresponds to one of each of the transport channels 1, 2, and 3. In fact, each of the CSID fields 154A through 154C indicates whether the corresponding payloads 156A and 156B are direction-based signals (when the corresponding ChannelType equals zero), vector-based signals (When the corresponding ChannelType is equal to 2) or empty (when the ChannelType is equal to 3).

In the example of FIG. 8E, frame 249A includes two vector-based signals (given a ChannelType 269 equal to 1 in CSID fields 154A and 154B) and an empty (ChannelType 269) CSID field 154C) is equal to 3). Based on the previous HOAconfig portion (not shown for ease of illustration purposes), the audio decoding device 24 may determine that all 16 V vector elements have been encoded. Accordingly, each of the VVectorData 156A and 156B includes all 16 vector elements, each of which is uniformly quantized into 8 bits.

As further shown in the example of FIG. 8E, frame 249A 'does not include a HOAPredictionInfo field. The HOAPredictionInfo field may represent a field corresponding to a second directional-based compression scheme that may be removed in accordance with the techniques described in this disclosure when a vector-based compression scheme is used to compress the HOA audio data.

8F is a diagram illustrating a frame 249A " substantially similar to frame 249A except that HOAGainCorrectionData is removed from each transport channel stored in frame 249A ". The HOAGainCorrectionData field may be removed from frame 249A " when gain correction is suppressed according to various aspects of the techniques described above.

FIG. 8G is a diagram illustrating a frame 249A '' 'that may be similar to frame 249A' 'except that the HOAPredictionInfo field is removed. Frame 249A '' 'represents one example in which both aspects of the techniques may be applied in connection with controlling various fields that may not be needed in certain situations.

The techniques described above may be performed with respect to any number of different contexts and audio echo systems. While a number of exemplary contexts are described below, techniques should be limited to those exemplary contexts. One exemplary audio echo system includes audio content, movie studios, music studios, gaming audio studios, channel based audio content, coding engines, game audio systems, game audio coding / rendering engines, .

Movie studios, music studios, and gaming audio studios may also receive audio content. In some instances, the audio content may represent the output of the acquisition. Movie studios may output channel based audio content (e.g., at 2.0, 5.1, and 7.1), for example, by using a digital audio workstation (DAW). Music studios may output channel based audio content (e.g., at 2.0, and 5.1), for example, by using a DAW. In any case, the coding engines may use one or more codecs (e.g., AAC, AC3, Dolby True HD, Dolby Digital Plus, and DTS Master Audio) for output by the delivery systems (DTS Master Audio)) to receive and encode channel based audio content. Gaming audio studios may output one or more game audio systems, for example, by using a DAW. Game audio coding / rendering engines may also code and / or render audio systems with channel based audio content for output by delivery systems. Other exemplary contexts in which the techniques may be implemented are broadcast recording audio objects, professional audio systems, consumer on-device capture, HOA audio format, on-device rendering, consumer audio, TV, and accessories, And an audio echo system that may include audio systems.

Broadcast recording audio objects, professional audio systems, and consumer on-device captures may both code their output using the HOA audio format. In this way, the audio content may be coded using the HOA audio format in a single representation that may be reproduced using on-device rendering, consumer audio, TV, and accessories, and car audio systems. In other words, a single representation of the audio content may be played in a generic audio playback system, such as audio playback system 16 (i.e., as opposed to requiring a particular configuration such as 5.1, 7.1, etc.).

Other examples of contexts in which techniques may be performed include capture elements, and audio echo systems that may include playback elements. Acquisition elements may include wired and / or wireless acquisition devices (e.g., eigenmicrophones), on-device surround sound capture, and mobile devices (e.g., smartphones and tablets). In some instances, the wired and / or wireless acquisition devices may be coupled to the mobile device via the wired and / or wireless communication channel (s).

According to one or more techniques of the present disclosure, the mobile device may be used to acquire a sound field. For example, the mobile device may capture the sound field through wired and / or wireless acquisition devices and / or on-device surround sound capture (e.g., a plurality of microphones integrated into the mobile device). The mobile device may then code the acquired sound field for playback by one or more of the playback elements with HOA coefficients. For example, a user of a mobile device may record a live event (e.g., a conference, a conference, a play, a concert, etc.) (capture the sound field of a live event) and code the recording into HOA coefficients.

The mobile device may also utilize one or more of the reproduction elements to reproduce the HOA coded sound field. For example, the mobile device may decode the HOA coded sound field and output one or more of the playback elements to one or more of the playback elements to cause a signal to regenerate the sound field. As one example, a mobile device may utilize wireless and / or wired communication channels to output signals to one or more speakers (e.g., speaker arrays, sound bars, etc.). As another example, a mobile device may utilize docking solutions to output signals to one or more docking stations and / or one or more docked speakers (e.g., soundcards in smart cars and / or homes) have. As another example, a mobile device may utilize headphone rendering to output a signal to a set of headphones, for example, to produce a realistic binaural sound.

In some instances, a particular mobile device may take both capturing the 3D sound field and playing back the same 3D sound field at a later time. In some examples, the mobile device captures the 3D sound field, encodes the 3D sound field to HOA, and transmits the encoded 3D sound field for playback to one or more other devices (e.g., other mobile devices and / Devices).

Another context in which techniques may be performed includes an audio echo system that may include audio content, game studios, coded audio content, rendering engines, and delivery systems. In some instances, game studios may include one or more DAWs that may support editing of HOA signals. For example, one or more DAWs may include HOA plug-ins and / or tools that may be configured to operate (e.g., operate) with one or more game audio systems. In some instances, game studios may output new stem formats that support HOA. In any case, game studios may output coded audio content with rendering engines that may render the sound field for playback by delivery systems.

The techniques may also be performed with respect to exemplary audio acquisition devices. Techniques may be performed with respect to an eigenmicrophone, which may, for example, comprise a plurality of microphones collectively configured to record a 3D sound field. In some instances, the plurality of microphones of the eigenmicrophone may be located on the surface of a substantially spherical ball having a radius of approximately 4 cm. In some instances, the audio encoding device 20 may be integrated into the eigenmicrophone to output the bitstream 21 directly from the microphone.

Other exemplary audio capture contexts may include a production truck that may be configured to receive signals from one or more microphones, such as one or more ear microphones. The manufacturing truck may also include an audio encoder, such as the audio encoder 20 of FIG.

The mobile device may also include, in some cases, a plurality of microphones that are collectively configured to record a 3D sound field. In other words, the plurality of microphones may have X, Y, Z diversity. In some instances, the mobile device may include a microphone, which may be rotated to provide X, Y, Z diversity for one or more other microphones of the mobile device. The mobile device may also include an audio encoder, such as the audio encoder 20 of FIG.

A ruggedized video capture device may also be configured to record a 3D sound field. In some instances, the trusted video capture device may be attached to the user ' s helmet involved in the activity. For example, a tagged video capture device may be attached to the helmet of a torrent rafting user. In this way, the latched video capture device can be used to capture 3D images of all the actions around the user (e.g., the water hitting in the back of the user, other ruffers speaking in front of the user, etc.) You can also capture the sound field.

The techniques may also be performed with respect to an accessory enhancement mobile device that may be configured to record a 3D sound field. In some instances, the mobile device may be similar to the mobile devices described above, to which one or more accessories have been added. For example, an eigenmicrophone may be attached to the above-mentioned mobile device to form an accessory enhancement mobile device. In this way, the accessory enhancement mobile device may capture a higher quality version of the 3D sound field than using only the integrated sound capture components in the accessory enhancement mobile device.

Exemplary audio playback devices that may perform various aspects of the techniques described in this disclosure are discussed further below. According to one or more techniques of the present disclosure, the speakers and / or sound bars may still be arranged in any arbitrary configuration while reproducing the 3D sound field. Moreover, in some instances, the headphone playback devices may be coupled to the decoder 24 via either a wired or wireless connection. According to one or more techniques of the present disclosure, a single general representation of a sound field may be utilized to render a sound field in any combination of speakers, sound bars, and headphone reproduction devices.

A number of different exemplary audio playback environments may also be suitable for performing various aspects of the techniques described in this disclosure. For example, a 5.1 speaker reproduction environment, a 2.0 (e.g., stereo) speaker reproduction environment, a 9.1 speaker reproduction environment with full height front loudspeakers, a 22.2 speaker reproduction environment, a 16.0 speaker reproduction environment, A mobile device having an ear bud playback environment may be suitable environments for performing various aspects of the techniques described in this disclosure.

According to one or more techniques of the present disclosure, a single general representation of the sound field may be utilized to render the sound field in any of the playback environments described above. Additionally, the techniques of the present disclosure enable a renderer to render a sound field from a regular expression for playback in playback environments other than those described above. For example, if design considerations prohibit proper placement of speakers in accordance with a 7.1 speaker playback environment (e.g., where it is not possible to place a right surround speaker), the techniques of the present disclosure will allow the lender, 6.1 It is possible to compensate with the other six speakers so that reproduction in the speaker reproduction environment may be achieved.

Moreover, the user may watch the sports game while wearing the headphones. According to one or more techniques of the present disclosure, a 3D sound field of a sports game may be captured (e.g., one or more child microphones may be placed in and / or around the baseball stadium), a 3D sound field corresponding to a 3D sound field HOA coefficients may be obtained and transmitted to the decoder and the decoder may reconstruct the 3D sound field based on the HOA coefficients and output the reconstructed 3D sound field to the renderer, Type and may render the reconstructed 3D sound field with signals that cause the headphones to output a representation of the 3D sound field of the sports game.

In each of the various cases described above, the audio encoding device 20 may comprise means for performing the method configured to perform the audio encoding device 20, or otherwise performing the respective steps of the method I have to understand. In some cases, the means may comprise one or more processors. In some cases, one or more processors may represent a special purpose processor configured by instructions stored in a non-volatile computer readable storage medium. In other words, various aspects of the techniques in each of the sets of encoding examples, when executed, may be implemented in a non-transitory computer readable storage (ROM) storage medium that stores instructions that cause one or more processors to perform the method Media may be provided.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code, or may be executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to type media, such as data storage media. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure have. The computer program product may comprise a computer readable medium.

Likewise, in each of the various cases described above, the audio decoding device 24 includes means for performing, or otherwise performing, the method (s) configured for the audio decoding device 24 to perform It should be understood that it is possible. In some cases, the means may comprise one or more processors. In some cases, one or more processors may represent a special purpose processor configured by instructions stored in a non-volatile computer readable storage medium. In other words, various aspects of the techniques in each of the sets of encoding examples, when executed, may be used to cause one or more processors to perform a non-volatile computer readable storage Media may be provided.

By way of example, and not limitation, such computer-readable storage media can be stored in RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, Or any other medium which can be used to store the desired program code and which can be accessed by a computer. However, it should be understood that the computer-readable storage mediums and data storage mediums do not include connections, carrier waves, signals, or other temporal media, but instead relate to non-transitory, type of storage media. A disk and a disc as used herein include a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc and a Blu-ray disc, discs usually reproduce data magnetically, while discs reproduce data optically with a laser. In addition, combinations of the above should also be included within the scope of computer readable media.

The instructions may include one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry Lt; / RTI > Accordingly, the term "processor" as used herein may refer to any of the structures described above, or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules included in a codec configured or combined for encoding and decoding. Techniques may also be fully implemented in one or more circuits or logic elements.

The techniques of the present disclosure may be implemented in a wide variety of devices or devices including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chipset). The various components, modules, or units are described in this disclosure to emphasize the functional aspects of the devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined into a codec hardware unit, or may be provided by a collection of interacting hardware units, including one or more of the processors described above, along with suitable software and / or firmware have.

Various aspects of the techniques have been described. These and other embodiments are within the scope of the following claims.

Claims (24)

A device configured to render higher order ambisonic coefficients,
One or more processors configured to obtain sign symmetry information indicative of sign symmetry of a matrix used to render the high order ambience coefficients to produce a plurality of speaker feeds; And
And a memory configured to store the sparseness information. The method according to claim 1,
Wherein the one or more processors are further configured to determine value symmetry information indicative of the value symmetry of the matrix and to determine a reduced number of bits used to represent the matrix based on the value symmetry information and the code symmetry information , And configured to render higher order ambience coefficients. The method according to claim 1,
Wherein the one or more processors are further configured to determine scarcity information representative of the scarcity of the matrix and to determine a reduced number of bits used to represent the matrix based on the scarcity information and the sign symmetry information, A device configured to render sonic coefficients. The method according to claim 1,
Wherein the one or more processors are further configured to render high order ambience coefficients, the matrix being configured to determine a speaker layout that should be used to render the plurality of speaker feeds from the higher order ambience coefficients. The method according to claim 1,
Further comprising a speaker configured to reproduce a sound field represented by the higher order ambience coefficients based on the plurality of speaker feeds. The method according to claim 1,
The one or more processors may also be configured to obtain audio rendering information indicative of a signal value identifying an audio renderer to be used in generating the plurality of speaker feeds, A device configured to render high-order ambience coefficients. The method according to claim 6,
Wherein the signal value comprises the matrix used to render the higher order ambience coefficients into the plurality of speaker feeds,
Wherein the one or more processors are configured to render the plurality of speaker feeds based on the matrix included in the signal value. CLAIMS 1. A method of rendering high order ambience coefficients,
And obtaining sign symmetry information indicative of the sign symmetry of the matrix used to render the higher order ambience coefficients to produce a plurality of speaker feeds. 9. The method of claim 8,
Determining a value symmetry information indicating a value symmetry of the matrix; And
Determining a number of reduced bits used to represent the matrix based on the value symmetry information and the code symmetry information,
≪ / RTI > further comprising the steps of: 9. The method of claim 8,
Determining scarcity information indicating the scarcity of the matrix; And
Determining a reduced number of bits used to represent the matrix based on the scarcity information and the symmetric information,
≪ / RTI > further comprising the steps of: 9. The method of claim 8,
Further comprising determining the speaker layout in which the matrix should be used to render multi-channel audio data from the higher order ambience coefficients. 9. The method of claim 8,
And reproducing a sound field represented by the higher order ambience coefficients based on the plurality of speaker feeds. 9. The method of claim 8,
Obtaining audio rendering information representing a signal value identifying an audio renderer used when generating the plurality of speaker feeds; And
Rendering the plurality of speaker feeds based on the audio rendering information
≪ / RTI > further comprising the steps of: 14. The method of claim 13,
Wherein the signal value comprises the matrix used to render the higher order ambience coefficients to produce the plurality of speaker feeds,
The method further comprises rendering the plurality of speaker feeds based on the matrix included in the signal value. A device configured to generate a bitstream,
A memory configured to store a matrix used to render high order ambience coefficients to generate a plurality of speaker feeds; And
One or more processors configured for sign symmetry information indicative of the sign symmetry of the matrix
And to generate a bitstream. 16. The method of claim 15,
Wherein the one or more processors are further configured to determine value symmetry information indicative of the value symmetry of the matrix and to reduce the number of bits representing the matrix based on the value symmetry information and the symmetry information. A device configured to create. 16. The method of claim 15,
Wherein the one or more processors are further configured to determine code symmetry information indicating the sign symmetry of the matrix and to reduce the number of bits representing the matrix based on the code symmetry information and the code symmetry information. A device configured to create. 16. The method of claim 15,
Wherein the one or more processors are further configured to determine a speaker layout in which the matrix should be used to render the plurality of speaker feeds from the higher order ambience coefficients. 16. The method of claim 15,
Further comprising a microphone configured to capture a sound field represented by the higher order ambience coefficients. A method of generating a bitstream,
And obtaining scarcity information indicative of the scarcity of a matrix used to render higher order ambience coefficients to produce a plurality of speaker feeds. 21. The method of claim 20,
Determining a value symmetry information indicating a value symmetry of the matrix; And
Decreasing the number of bits representing the matrix based on the value symmetry information and the code symmetry information
≪ / RTI > 21. The method of claim 20,
Determining sign symmetry information indicating sign symmetry of the matrix; And
Reducing the number of bits representing the matrix based on the code symmetry information and the code symmetry information
≪ / RTI > 21. The method of claim 20,
Further comprising determining the speaker layout in which the matrix should be used to render multi-channel audio data from the higher order ambience coefficients. 21. The method of claim 20,
Further comprising the step of capturing a sound field represented by the higher order ambience coefficients.

Images