EA043574B1 - MULTILEVEL DECODING OF COMPRESSED REPRESENTATIONS OF SOUND OR SOUND FIELD - Google Patents

Description

Cross reference to related applications

The present application claims priority under European Patent Application No.

15306590.9, filed October 8, 2015, and US Patent Application No. 62/361809, the contents of which are incorporated herein by reference in their entirety.

Field of technology to which the invention relates

This document relates to methods and apparatus for multi-layer audio coding. In particular, this document relates to methods and apparatus for layered audio encoding of compressed audio (or sound field) representations, such as Higher-Order Ambisonics (HOA) audio (or sound field) representations.

State of the art

For streaming audio (or sound field) over a transmission channel with time-varying conditions, multi-level coding is a means to adapt the quality of the audio representation to the transmission conditions and, in particular, to avoid unwanted signal dropouts.

For layered coding, the audio (or sound field) representation is typically divided into a high-priority base layer of relatively small size and additional enhancement layers of decreasing priorities and arbitrary sizes. Each enhancement layer is generally assumed to contain incremental information to complement all of the lower layers to improve the quality of the sound presentation (or sound field). The amount of error protection for transmission of individual layers is controlled based on their priority. In particular, the base layer is provided with high error protection, which is reasonable and acceptable due to its small size.

However, there is a need for multi-layer encoding schemes for (an extended version of) special types of compressed audio or sound field representations, such as, for example, compressed NOA audio or sound field representations.

This document addresses the issues mentioned above. In particular, methods and encoders/decoders for layered encoding of compressed audio or sound field representations are described.

The essence of the invention

In accordance with an aspect, a method for layered encoding of a compressed representation of an audio or sound field is described. The compressed audio representation may include a basic compressed audio representation that includes a plurality of components. Multiple components may be complementary components. The compressed audio representation may further include basic auxiliary information for decoding the basic compressed audio representation into a basic reconstructed audio or sound field representation. The compressed audio representation may further include enhancement auxiliary information including parameters for enhancing (eg, expanding) the underlying reconstructed audio representation. The method may include subdividing (eg, grouping) a plurality of components into a plurality of component groups. The method may further include assigning (eg, adding) each of the plurality of groups to a corresponding one of the plurality of hierarchical levels. An assignment may indicate a correspondence between corresponding groups and levels. Components assigned to a corresponding level can be said to be included in that level. The number of groups may correspond (for example, be equal) to the number of levels. The plurality of levels may include a base level and one or more hierarchical enhancement levels. The plurality of hierarchical levels can be ordered from a base level, through a first enhancement level, a second enhancement level, etc., up to a common highest enhancement layer (common top level). The method may further include adding basic auxiliary information to the base layer (eg, including basic auxiliary information in the base layer or distributing the basic auxiliary information to the base layer, for example, for transmission or storage purposes). The method may further include determining a plurality of pieces of improvement auxiliary information based on the improvement auxiliary information. The method may further include assigning (eg, adding) each of the plurality of pieces of enhancement auxiliary information to a corresponding one of the plurality of levels. Each piece of enhancement auxiliary information may include parameters for enhancing the reconstructed (eg, reconstructed) audio representation available from data included (eg, assigned or added) to the corresponding layer and any layers below the corresponding layer. Multi-level encoding may be performed for purposes of transmission over a transmission channel or for storage on a suitable storage medium such as, for example, a CD, DVD or Blu-ray Disc™.

Configured as mentioned above, the proposed method allows efficient application of multi-level coding to compressed audio representations containing multiple components as well as first and enhancement auxiliary information (e.g., independent base

- 1 043574 auxiliary information and improving auxiliary information) having the properties described above. In particular, the proposed method ensures that each layer includes suitable supporting information for reconstructing a reconstructed audio representation from components included in any layers up to the layer in question. It is assumed that the levels up to the considered level include, for example, a basic level, a first enhancement level, a second enhancement level, etc., up to the considered level. Thus, regardless of the actual highest applicable layer (eg, the layer below the lowest applicable layer that was not correctly received, and thus all layers below the highest applicable layer and the highest applicable layer itself being correctly received), the decoder is allowed to improve or expand the reconstructed audio representation, even though the reconstructed audio representation may differ from the complete audio representation. In particular, regardless of the actual highest applicable layer, it is sufficient for a decoder to decode the enhancement auxiliary information payload for only a single layer (i.e., the highest applicable layer) to improve or expand the reconstructed audio representation that is available based on all components included in levels up to the actual highest applicable level. Thus, for each time slot (eg, frame), only a single enhancement ancillary information payload must be decoded. On the other hand, the proposed method allows taking full advantage of the reduction in required bandwidth that can be achieved by using multi-level coding.

In embodiments, components of the underlying compressed audio representation may correspond to monaural signals (eg, transport signals or monaural transport signals). Monaural cues can represent either predominant audio cues or sequences of NOA presentation coefficients. Monaural signals can be quantized.

In embodiments, the basic support information may include information that specifies decoding (eg, decoding) of one or more of the plurality of components individually, independently of the other components. For example, the basic auxiliary information may represent auxiliary information related to individual monaural signals, independent of other monaural signals. Thus, the basic auxiliary information may be referred to as independent basic auxiliary information.

In embodiments, the enhancement auxiliary information may represent enhancement auxiliary information. The enhancing auxiliary information may include prediction parameters for the base compressed audio representation to improve (eg, enhance) the base reconstructed audio representation that are available from the base compressed audio representation and the base auxiliary information.

In embodiments, the method may further include generating a transport stream for transmitting data of multiple layers (eg, data assigned to or appended to, or otherwise included in, respective layers). The base layer may have the highest transmission priority, and hierarchical enhancement layers may have decreasing transmission priorities. Thus, the transmission priority may be decreased from the base layer to the first enhancement layer, from the first enhancement layer to the second enhancement layer, etc. The amount of error protection for data transmission of multiple layers can be controlled in accordance with the respective transmission priorities. In this way it can be ensured that at least some of the lower layers are transmitted reliably, on the other hand reducing the overall required bandwidth without applying excessive error protection to the higher layers.

In embodiments, the method may further include, for each of the plurality of layers, generating a transport layer packet including data of the corresponding layer. For example, for each time slot (eg, frame), a corresponding transport layer packet may be generated for each of a plurality of layers.

In embodiments, the compressed audio representation may further include additional base auxiliary information for decoding the base compressed audio representation into a base reconstructed audio representation. The additional basic support information may include information that specifies decoding of one or more of the plurality of components depending on the corresponding other components. The method may further include decomposing the additional basic auxiliary information into a plurality of pieces of the additional basic auxiliary information. The method may further include adding pieces of additional basic auxiliary information to the base layer (eg, including a piece of additional basic auxiliary information in the base layer or distributing pieces of additional basic auxiliary information to the base layer, for example, for transmission or storage purposes). Each piece of additional basic auxiliary information may be associated with a corresponding layer and may include

- 2 043574 formation that determines the decoding of one or more components assigned to the corresponding level depending (only) on the corresponding other components assigned to the corresponding level and any levels below the corresponding level. Thus, each piece of additional basic auxiliary information defines components at the corresponding level to which that piece of additional basic auxiliary information corresponds, regardless of any other components assigned to levels higher than the corresponding level.

Configured in this way, the proposed method avoids the fragmentation of additional basic auxiliary information by adding all the parts to the base layer. In other words, all pieces of additional basic auxiliary information are included in the base layer. Decomposition of the additional basic auxiliary information ensures that a portion of the additional basic auxiliary information is available for each level, which does not require knowledge of the components at higher levels. Thus, regardless of the actual highest applicable layer, it is sufficient for the decoder to decode the additional basic auxiliary information included in the layers up to the highest applicable layer.

In embodiments, the additional basic auxiliary information may include information that specifies decoding (eg, decoding) of one or more of the plurality of components depending on the other components. For example, additional basic auxiliary information may represent auxiliary information related to individual monaural signals depending on other monaural signals. Thus, the additional basic auxiliary information may be referred to as dependent basic auxiliary information.

In embodiments, the compressed audio representation may be processed for successive time slots, for example, time slots of equal size. Consecutive time slots may be frames. Thus, the method can operate on a frame-by-frame basis, ie, the compressed audio representation can be encoded frame by frame. A compressed audio representation may be available for each consecutive time interval (eg, each frame). Thus, the compression operation by which the compressed audio representation was obtained may operate on a frame-by-frame basis.

In embodiments, the method may further include generating configuration information that indicates, for each layer, components of the underlying compressed audio representation that are assigned to that layer. In this way, the decoder can easily access the information needed for decoding without unnecessarily analyzing the received data payloads.

In accordance with another aspect, a method for layered encoding of a compressed representation of an audio or sound field is described. The compressed audio representation may include a basic compressed audio representation that includes a plurality of components. The plurality of components may be complementary components. The compressed audio representation may further include base support information (eg, independent base support information) and third information (eg, dependent base support information) for decoding the base compressed audio representation into a base reconstructed audio or sound field representation. The basic support information may include information that specifies decoding of one or more of the plurality of components individually, independently of the other components. The additional basic support information may include information that specifies decoding of one or more of the plurality of components depending on the corresponding other components. The method may include subdividing (eg, grouping) a plurality of components into a plurality of component groups. The method may further include assigning (eg, adding) each of the plurality of groups to a corresponding one of the plurality of hierarchical levels. An assignment may indicate a correspondence between corresponding groups and levels. Components assigned to the corresponding level can be said to be included in that level. The number of groups may correspond (for example, be equal) to the number of levels. The plurality of levels may include a base level and one or more hierarchical enhancement levels. The method may further include adding basic auxiliary information to the base layer (eg, including the basic auxiliary information in the base layer or distributing the basic auxiliary information to the base layer, such as for transmission or storage purposes). The method may further include decomposing the additional basic auxiliary information into a plurality of additional basic auxiliary information pieces and adding the additional basic auxiliary information pieces to the base layer (e.g., including the additional basic auxiliary information pieces into the base layer or allocating the additional basic auxiliary information pieces to the base layer , for example, for transmission or storage purposes). Each piece of additional basic supporting information can be associated with a corresponding

- 3 043574 layer and include information that specifies the decoding of one or more components assigned to the corresponding layer depending on the corresponding other components assigned to the corresponding layer and any layers below the corresponding layer.

Thus configured, the proposed method ensures that for each layer suitable additional basic auxiliary information is available to decode the components included in any layer up to the corresponding layer, without requiring correct reception or decoding (or generally knowledge) of any higher layers. In the case of a compressed HOA representation, the proposed method ensures that, in vector encoding mode, a suitable Vvector is available for all components belonging to layers up to the highest applicable layer. In particular, the proposed method eliminates the case in which the elements of the V-vector corresponding to components at higher levels are not explicitly reported. Accordingly, information included in the layers up to the highest applicable layer is sufficient to decode (eg, reconstruct) any components belonging to the layers up to the highest applicable layer. This ensures suitable recovery of the corresponding reconstructed NAA representations for lower levels, even if higher levels could not be correctly received by the decoder. On the other hand, the proposed method makes it possible to take full advantage of the reduction in required bandwidth that can be achieved by using multi-level coding.

Embodiments of this aspect may relate to embodiments of the previous aspect.

In accordance with another aspect, a method for decoding a compressed representation of an audio or sound field is described. The compressed representation of sound can be encoded at multiple hierarchical levels. The plurality of hierarchical levels may include a base level and one or more hierarchical incremental levels. The plurality of layers may have components of a basic compressed audio representation or sound field assigned to them. In other words, the plurality of layers may include components of the underlying compressed auxiliary information. Components can be assigned to the corresponding levels in the corresponding component groups. Multiple components may be complementary components. The base layer may include basic auxiliary information for decoding the underlying compressed audio representation. Each layer may include a portion of enhancement auxiliary information including parameters for enhancing the underlying rendered audio representation available from data included in the corresponding layer and any layers below the corresponding layer. The method may include receiving data payloads respectively associated with a plurality of hierarchical levels. The method may further include determining a first layer index indicating the highest usable layer among the plurality of layers to be used for decoding the base compressed audio representation into the base reconstructed audio or sound field representation. The method may further include obtaining a base reconstructed audio representation from components assigned to the highest applicable layer and any layers below the highest applicable layer using the base auxiliary information. The method may further include determining a second level index that indicates how much of the enhancement auxiliary information should be used to improve (eg, enhance) the underlying reconstructed audio representation. The method may further include obtaining a reconstructed audio or sound field representation from the base reconstructed audio representation with reference to a second level index.

Configured in this manner, the proposed method ensures that the reconstructed audio representation is of optimal quality using available (eg, correctly received) information to the best extent possible.

In embodiments, components of the underlying compressed audio representation may correspond to monaural signals (eg, monaural transport signals). Monaural cues can represent either predominant audio cues or sequences of NOA presentation coefficients. Monaural signals can be quantized.

In embodiments, the basic support information may include information that specifies decoding (eg, decoding) of one or more of the plurality of components individually, independently of the other components. For example, the basic auxiliary information may represent auxiliary information related to individual monaural signals, independent of other monaural signals. Thus, the basic auxiliary information may be referred to as independent basic auxiliary information.

In embodiments, the enhancement auxiliary information may represent enhancement auxiliary information. The enhancing auxiliary information may include prediction parameters for the base compressed audio representation to improve (eg, enhance) the base reconstructed audio representation that are available from the base compressed audio representation and the base auxiliary information.

- 4 043574

In embodiments, the method may further include determining for each layer whether the corresponding layer was received correctly. The method may further include defining the first level index as the index of that level that is immediately below the lowest level that was not received correctly.

In embodiments, determining the second level index may include either determining the second level index to be equal to the first level index, or determining an index value to be the second level index that indicates that no enhancement auxiliary information should be used in obtaining the reconstructed audio representation. . In the latter case, the reconstructed sound representation may be equivalent to the base reconstructed sound representation.

In embodiments, data payloads may be received and processed for successive time slots, such as time slots of equal size. Consecutive time slots may be frames. Thus, the method can operate on a frame-by-frame basis. The method may further include determining the second level index to be equal to the first level index if the compressed audio representations for successive time intervals can be decoded independently of each other.

In embodiments, data payloads may be received and processed for successive time slots, eg, time slots of equal size. Consecutive time slots may be frames. Thus, the method can operate on a frame-by-frame basis. The method may further include, for a given time slot among successive time slots, determining for each layer whether the corresponding layer was received correctly if the compressed audio representations for successive time slots cannot be decoded independently of each other. The method may further include determining the first level index for a given time interval to be the lesser index of the first level index of a time interval preceding the given time interval and the level index immediately below the lowest level that was not received correctly.

In embodiments, the method may further include, for a given time slot, determining whether the first level index for a given time slot is equal to the first level index for a previous time slot if the compressed audio representations for successive time slots cannot be decoded independently of each other. The method may further include determining that the second level index for a given time interval is equal to the first level index for a given time interval if the first level index for a given time interval is equal to the first level index for a previous time interval. The method may further include defining an index value as a second level index that indicates that no enhancement auxiliary information should be used in obtaining a reconstructed audio representation if the first level index for a given time interval is not equal to the first level index for a previous time interval. interval.

In embodiments, a base layer may include at least one piece of additional base auxiliary information associated with a corresponding layer and including information that specifies the decoding of one or more components among components assigned to the corresponding layer depending on other components assigned the appropriate level and any levels below the appropriate level. The method may further include, for each piece of additional basic auxiliary information, decoding the portion of the additional basic auxiliary information by reference to components assigned to its corresponding layer and any layers below the corresponding layer. The method may further include updating a portion of the additional basic auxiliary information by reference to components assigned to the highest applicable level and any levels between the highest applicable level and the corresponding level. The basic reconstructed audio representation may be obtained from the components assigned to the highest applicable level and any levels below the highest applicable level, using the basic auxiliary information and an adjusted portion of the additional basic auxiliary information obtained from the portions of the additional basic auxiliary information corresponding to the levels up to the most high applicable level.

In embodiments, the additional basic auxiliary information may include information that specifies decoding (eg, decoding) of one or more of the plurality of components depending on the other components. For example, additional basic auxiliary information may represent auxiliary information related to individual monaural signals depending on other monaural signals. Thus, the additional basic auxiliary information may be referred to as dependent basic auxiliary information.

In accordance with another aspect, a method for decoding a compressed representation of audio or

- 5 043574 sound field. The compressed representation of sound can be encoded at multiple hierarchical levels. The plurality of hierarchical levels may include a base level and one or more hierarchical incremental levels. The plurality of layers may have components of a basic compressed audio representation or sound field assigned to them. In other words, the plurality of layers may include components of the underlying compressed auxiliary information. Components can be assigned to the corresponding levels in the corresponding component groups. Multiple components may be complementary components. The base layer may include basic auxiliary information for decoding the underlying compressed audio representation. The base layer may further include at least one piece of additional base auxiliary information associated with the corresponding layer and including information that specifies the decoding of one or more components among the components assigned to the corresponding layer depending on the other components assigned to the corresponding layer and any levels below the appropriate level. The method may include receiving data payloads respectively associated with a plurality of hierarchical levels. The method may further include determining a first layer index indicating the highest usable layer among the plurality of layers to be used for decoding the base compressed audio representation into the base reconstructed audio or sound field representation. The method may further include, for each piece of additional basic auxiliary information, decoding the portion of the additional basic auxiliary information by reference to components assigned to its corresponding layer and any layers below the corresponding layer. The method may further include, for each piece of additional basic auxiliary information, updating the portion of the additional basic auxiliary information by reference to components assigned to the highest applicable level and any levels between the highest applicable level and the corresponding level. The basic reconstructed audio representation may be obtained from the components assigned to the highest applicable level and any levels below the highest applicable level, using the basic auxiliary information and an adjusted portion of the additional basic auxiliary information obtained from the portions of the additional basic auxiliary information corresponding to the levels up to the most high applicable level. The method may further comprise defining a second layer index that is either equal to the first layer index or indicates omission of enhancement auxiliary information during decoding.

Configured in this way, the proposed method ensures that the additional core auxiliary information that is ultimately used to decode the base compressed audio representation does not include redundant elements, thereby realizing more efficient actual decoding of the base compressed audio representation.

Embodiments of this aspect may relate to embodiments of the previous aspect.

In accordance with another aspect, an encoder is described for layered encoding of a compressed representation of an audio or sound field. The compressed audio representation may include a basic compressed audio representation that includes a plurality of components. Multiple components may be complementary components. The compressed audio representation may further include basic auxiliary information for decoding the basic compressed audio representation into a basic reconstructed audio or sound field representation. The compressed audio representation may further include enhancement auxiliary information including parameters for enhancing (eg, expanding) the underlying reconstructed audio representation. The encoder may include a processor configured to perform some or all of the steps of the methods in accordance with the first aspect mentioned above and the second aspect mentioned above.

In accordance with another aspect, a decoder for decoding a compressed representation of an audio or sound field is described. The compressed representation of sound can be encoded at multiple hierarchical levels. The plurality of hierarchical levels may include a base level and one or more hierarchical incremental levels. The plurality of layers may have components of a basic compressed audio representation or sound field assigned to them. In other words, the plurality of layers may include components of the underlying compressed auxiliary information. Components can be assigned to the corresponding levels in the corresponding component groups. Multiple components may be complementary components. The base layer may include basic auxiliary information for decoding the underlying compressed audio representation. Each layer may include a portion of enhancement auxiliary information including parameters for enhancing (eg, enhancing) the underlying rendered audio representation available from the data included in the corresponding layer and any layers below the corresponding layer. The decoder may include a processor configured to perform some or all of the steps of the methods in accordance with the third aspect mentioned above and the fourth aspect mentioned

- 6 043574 higher aspect.

In accordance with other aspects, methods, devices and systems are directed to decoding a compressed Higher Order Ambisonics (HOA) representation of a sound or sound field (spatial sound or higher order sound field). The device may have a receiver configured to receive, or a method may receive, a bit stream containing a compressed representation of the HOA corresponding to a plurality of hierarchical levels that include a base level and one or more hierarchical enhancement levels. The plurality of layers have components of a basic compressed sound representation or sound field assigned to them, the components being assigned to corresponding layers in respective component groups. The apparatus may have a decoder configured to decode, or a method may decode, a compressed representation of the NLA based on basic auxiliary information that is associated with the base layer and based on enhancement auxiliary information that is associated with one or more hierarchical enhancement layers. The basic auxiliary information may include basic independent auxiliary information related to the first individual monaural signals that will be decoded independently of other monaural signals. Each of the one or more hierarchical enhancement layers may include a portion of enhancement auxiliary information including parameters for enhancing the underlying rendered audio representation available from the data included in the respective layers and any layers below the corresponding layer.

Basic independent supporting information may indicate that the first individual monaural signals represent a directional signal with a falling direction. The basic auxiliary information may further include basic dependent auxiliary information related to the second individual monaural signals that will be decoded dependently on the other monaural signals. The basic dependent auxiliary information may include vector-based signals that are distributed along directions in the sound field, the direction distribution being defined by a vector. The vector components are set to zero and are not part of the compressed vector representation.

Components of the underlying compressed sound representation may correspond to monaural cues that represent either predominant audio cues or sequences of NOA representation coefficients. The bit stream includes data payloads correspondingly associated with multiple hierarchical levels. The enhancing auxiliary information may include parameters related to at least one of spatial prediction, directional subband signal synthesis, and parametric surround duplication. The enhancement auxiliary information may include information that makes it possible to predict missing parts of the sound or sound field based on the directional signals. It may be further determined for each level whether the corresponding level was received correctly, and the index of the level that is immediately below the lowest level that was not received correctly.

In accordance with another aspect, a program is described. The program may be adapted to be executed on a processor and to perform some or all of the steps of the method set forth herein when executed on a computing device.

In accordance with yet another aspect, a storage medium is described. The storage medium may comprise a program adapted to be executed on a processor and to perform some or all of the steps of the method set forth herein when executed on a computing device.

Statements made with respect to any of the above-mentioned aspects or embodiments thereof also apply to corresponding other aspects or embodiments thereof, as one skilled in the art will appreciate. Repetition of these statements for each aspect or embodiment has been omitted for brevity.

The methods and devices, including the preferred embodiments set forth herein, may be used alone or in combination with other methods and systems disclosed herein. In addition, all aspects of the methods and devices set forth herein may be combined in any manner. In particular, the features of the claims can be combined with each other in any way.

Method steps and device features may be interchanged in various ways. In particular, the details of the disclosed method may be implemented as a device configured to perform some or all of the steps of the method, and vice versa, as one skilled in the art will understand.

Brief description of drawings

The invention is explained below in an illustrative manner with reference to the accompanying figures.

Fig. 1 is a flowchart illustrating an example of a layered encoding method in accordance with embodiments of the disclosure;

- 7 043574 fig. 2 is a block diagram schematically illustrating an example of an encoder stage in accordance with embodiments of the disclosure;

fig. 3 is a flowchart illustrating an example of a method for decoding a compressed representation of a sound or sound field that has been encoded in a plurality of hierarchical levels, in accordance with embodiments of the disclosure;

fig. 4A and 4B are block diagrams schematically illustrating examples of a decoder stage, in accordance with embodiments of the disclosure;

fig. 5 is a block diagram schematically illustrating an example of a hardware implementation of an encoder in accordance with embodiments of the disclosure; and fig. 6 is a block diagram schematically illustrating an example of a hardware implementation of a decoder in accordance with embodiments of the disclosure.

Carrying out the invention

First, a compressed audio (or sound field) representation (hereinafter referred to as a compressed audio representation for brevity) to which the methods and encoders/decoders of the present disclosure are applicable will be described. In general, a complete compressed sound (or sound field) representation (hereinafter referred to as a full compressed sound representation for brevity) may contain (e.g., consist of) the following three components: a base compressed sound (or sound field) representation (hereinafter referred to as a base compressed representation for brevity). sound representation), basic auxiliary information and enhancement auxiliary information.

The basic compressed audio representation itself contains (eg, consists of) multiple components (eg, complementary components). The basic compressed audio representation can take into account by far the largest percentage of the overall compressed audio representation. The basic compressed sound representation may consist of monaural transport signals representing either the predominant sound signals or a sequence of coefficients of the original PLA representation.

The basic auxiliary information is needed to decode the basic compressed audio representation and is expected to be much smaller in size compared to the basic compressed audio representation. This can be done down to its largest portion of disjointed parts, each of which determines the recovery of only one particular component of the underlying compressed audio representation. The basic auxiliary information may comprise a first part, which may be known as independent basic auxiliary information, and a second part, which may be known as additional basic auxiliary information.

Both the first and second parts, independent basic auxiliary information and additional basic auxiliary information, may determine the recovery of specific components of the underlying compressed audio representation. The second part is optional and may be omitted. In this case, the compressed audio representation can be said to contain the first part (eg, basic supporting information).

The first part (eg, basic auxiliary information) may contain auxiliary information describing individual (complementary) components of the basic compressed audio representation, independent of other (complementary) components. In particular, the first part (eg, basic auxiliary information) may determine the decoding of one or more of the plurality of components individually, independently of the other components. Thus, the first part may be referred to as independent basic supporting information.

The second (optional) part may contain auxiliary information, also known as additional basic auxiliary information, which may describe individual (complementary) components of the basic compressed audio representation depending on other (complementary) components. This second part may also be referred to as dependent basic auxiliary information. In particular, a dependency may have the following properties:

The dependent underlying auxiliary information for each individual (complementary) component of the underlying compressed audio representation may reach its greatest extent when other specific (complementary) components are not contained in the underlying compressed audio representation.

In the event that additional specific (complementary) components are added to the underlying compressed audio representation, the dependent basic auxiliary information for the individual (complementary) component in question may become a subset of the original dependent basic auxiliary information, thereby reducing its size.

Improving supporting information is also optional. It can be used to enhance or enhance (eg, parametrically enhance or expand) the underlying compressed audio representation. Its size can also be expected to be much smaller than that of the basic compressed audio representation.

Thus, in embodiments, the compressed audio representation may comprise a base compressed audio representation comprising a plurality of components, basic auxiliary information for decoding (e.g., decompressing) the base compressed audio representation to a base reconstructed audio representation or sound field, and enhancement auxiliary information , including parameters for enhancing or enhancing (e.g., parametric enhancement or expansion) the underlying reconstructed audio representation. The compressed audio representation may also include additional basic auxiliary information for decoding (e.g., reconstructing) the base compressed audio representation to the base reconstructed audio representation, which may include information that specifies the decoding of one or more of the plurality of components depending on the corresponding other components.

One example of this type of full compressed audio representation is defined by a Higher Order Ambisonics (HOA) compressed representation of a sound field, as defined by the MPEG-H 3D Audio Standard Preview (Ref. 1), Chapter 12 and Appendix C. 5. Thus, a compressed representation of sound may correspond to a compressed PLA representation of sound (or sound field).

For this example, the basic compressed audio field representation (the basic compressed audio representation) may contain multiple components (eg, may be identified by them). The components may represent (e.g., match) monaural signals. The monaural signals may be quantized monaural signals. Monaural signals can represent either predominant sound signals or sequences of coefficients of the ambient NOA component of the sound field.

Basic supporting information may describe, among other things, for each of these monaural signals how it contributes spatially to the sound field. For example, the underlying auxiliary information may define the predominant audio signal as a purely directional signal, meaning a general plane wave with some direction of incidence. Alternatively, the underlying auxiliary information may define the monaural signal as a sequence of coefficients of the original PLA representation having some index. The basic supporting information can also be divided into a first part and a second part as stated above. The first part is auxiliary information (eg, independent basic auxiliary information) related to specific individual monaural signals. This independent basic supporting information is independent of the existence of other monaural signals. Such auxiliary information may, for example, define a monaural signal to represent a directional signal (eg, indicating a general plane wave) with some direction of incidence. Alternatively, the monaural signal can be defined as a sequence of coefficients of the original representation of the NOA, having some index. The first part may be referred to as independent basic supporting information. In general, the first part (eg, basic auxiliary information) may determine the decoding of one or more of the plurality of monaural signals individually, independently of other monaural signals.

The second part is auxiliary information (eg, additional basic auxiliary information) related to specific individual monaural signals. This supporting information is dependent on the existence of other monaural signals. Such auxiliary information can be used, for example, if monaural signals are defined as vector-based signals (see, for example, reference 1, section 12.4.2.4.4). These signals are distributed along directions in the sound field, and the directional distribution can be defined by a vector. In some mode (see, for example, CodedWecLength=1), the individual components of this vector are implicitly set to zero and are not part of the compressed vector representation. These components are those with indices equal to the indices of the coefficient sequences of the original PLA representation and part of the underlying compressed audio representation. This means that if individual vector components are encoded, their total number may depend on the underlying compressed audio representation. In particular, the total number may depend on what sequences of coefficients the original PLA representation contains.

If the coefficient sequences of the original HOA representation are not contained in the underlying compressed audio representation, the dependent underlying auxiliary information for each vector-based signal consists of all vector components and has its largest size. In the event that sequences of original HOA representation coefficients with certain indices are added to the base compressed audio representation, the vector components with these indices are removed from the auxiliary information for each vector-based signal, thereby reducing the size of the dependent base auxiliary information for the vector-based signals.

The enhancement auxiliary information (e.g., enhancement auxiliary information) may comprise parameters related to (broadband) spatial prediction (see reference 1, section 12.4.2.4.3) and/or parameters related to directional subband signal synthesis and parametric audio duplication environment.

Parameters related to (broadband) spatial prediction can be used to (linearly) predict missing parts of the sound field from directional signals.

Directional subband synthesis and parametric surround duplication are compression tools that were recently introduced into the MPEG-H 3D audio standard via an amendment [see link 2, section 1]. These two tools allow frequency-dependent parametric prediction of additional monaural signals to be spatially distributed to complement the spatially incomplete or imperfectly compressed representation of the NOA. The prediction may be based on sequences of coefficients of the underlying compressed audio representation.

It is important to note that the above-mentioned complementary contributions to the sound field are represented in the compressed PLA representation not through additional quantized signals, but through additional auxiliary information of a comparatively much smaller size. Therefore, the two mentioned encoding tools are particularly suitable for compressing PLA representations at low data rates.

A second example of a compressed representation of one or more monaural signals with the structure mentioned above may contain encoded spectral information for disjoint frequency bands up to some upper frequency, which can be considered a basic compressed representation; basic auxiliary information defining the encoded spectral information (eg, by the number and width of encoded frequency bands); and enhancing auxiliary information comprising (eg, consisting of) spectral band copy (SBR) parameters that describe how to parametrically recreate, from the base compressed representation, spectral information for higher frequency bands that are not considered in the base compressed representation.

The present disclosure provides a method for layered encoding of a complete compressed representation of a sound (or sound field) having the structure mentioned above.

Compression may be frame-based in the sense that it provides compressed representations (in the form of data packets, or equivalent to frame payloads) for successive time slots. Time intervals can have equal or different sizes. These data packets may be expected to contain a validity flag, a value indicating their size, and the actual compressed data. In the following, without intentional limitation, it will be assumed that the compression is frame-based. Additionally, unless otherwise specified, and without intentional restriction, the focus will be on processing a single frame, and therefore the frame index will be omitted. Each frame payload of a given full compressed audio representation (or sound field) is assumed to contain J data packets (or frame payloads), each for one component of the underlying compressed audio representation, which are denoted BSRCj, j=1,... ,J. In addition, the packet is assumed to contain independent basic auxiliary information (basic auxiliary information), denoted BSI _I , defining individual components BSRCj of the basic compressed audio representation, independent of other components. Optionally, the packet may be further contemplated to contain dependent basic auxiliary information (additional basic auxiliary information), designated as a BSID, identifying individual components BSRCj of the base compressed audio representation depending on the other components.

The information contained in the two data packets BSII and BSID may optionally be grouped into a single Basic Ancillary Information data packet BSI. A single BSI data packet can be said to contain, among other things, J parts, each of which defines one distinct component BSRCj of the underlying compressed audio representation. Each of these parts can be said to in turn contain a piece of independent auxiliary information and optionally a piece of dependent auxiliary information.

Ultimately, it may include an enhancement auxiliary information (enhancement auxiliary information) payload, denoted ESI, describing how to improve or expand the reconstructed sound (or sound field) based on the complete underlying compressed audio representation.

The proposed multi-level coding solution addresses the steps required to enable both the compression part, which includes packaging data packets for transmission, and the reception and recovery part. Each part will be described in detail below.

First, compression and packaging (for example, for transmission) will be described. In particular, the components and elements of a complete compressed audio representation (or sound field) in the case of multi-level coding will be described.

Fig. 1 schematically illustrates a flowchart of an example compression and packaging method (eg, an encoding method or a layered encoding method for a compressed audio or sound field representation).

The assignment (eg, distribution) of individual payloads to the base layer and (M-1) enhancement layers can be achieved through a transport layer packer. Fig. 2 schematically illustrates a block diagram of an example of assignment/distribution of individual payloads.

As stated above, the overall compressed audio representation 2100 may refer, for example, to a compressed HOA representation containing a base compressed audio representation. The overall compressed audio representation 2100 may comprise a plurality of components (eg, monaural signals) 2110-1, ... 2110-J, independent basic auxiliary information (base auxiliary information) 2120, optional enhancement auxiliary information (enhancing auxiliary information) 2140, and optional dependent basic auxiliary information (additional basic auxiliary information) 2130. Basic auxiliary information 2120 may be information for decoding a basic compressed audio representation into a basic reconstructed audio or sound field representation. Basic auxiliary information 2120 may include information that specifies decoding of one or more components (eg, monaural signals) individually, independent of other components. Enhancement auxiliary information 2140 may include parameters for enhancing (eg, expanding) the underlying rendered audio representation. Additional basic auxiliary information 2130 may be (additional) information for decoding a basic compressed audio representation into a basic reconstructed audio representation and may include information that specifies decoding of one or more of a plurality of components depending on the corresponding other components.

Fig. 2 illustrates the underlying assumption that there are multiple hierarchical levels, including one base level (core level) and one or more (hierarchical) enhancement levels. For example, there may be only M levels, i.e. one base level and M-1 upgrade levels. A set of hierarchical levels has a successively increasing level index. The lowest level index value (for example, level 1 index) corresponds to the base level. It is further understood that the levels are ordered from the base level, through the enhancement levels, up to the overall highest enhancement level (ie, the overall highest level).

The proposed method can be implemented on a frame-by-frame basis (i.e., frame by frame). In particular, the compressed audio representation 2100 may be compressed across successive time slots, eg, time slots of equal size. Each time slot can correspond to a frame. The steps described below may be performed for each successive time slot (eg, frame).

At step S1010 in FIG. 1, a plurality of components 2110 is divided into a plurality of component groups. Each of the plurality of groups is then assigned (eg, added or distributed) to a corresponding one of the plurality of hierarchical levels. In this case, the number of groups corresponds to the number of levels. For example, the number of groups can be equal to the number of levels, so that there is one group of components for each level. As stated above, the plurality of layers may include a base layer and one or more (eg, M-1) hierarchical enhancement layers.

In other words, the basic compressed audio representation is divided into parts that will be assigned to individual layers. Without loss of generality, the grouping can be described by M+1 numbers Jm, m=0,...,M, where J ₀ =1 and J _M =J+1, as a result of which the components RSRCj are assigned to the m-th level for Jm- 1<j<Jm.

In step S1020, the component groups are assigned to their respective levels. At step S1030, the basic auxiliary information 2120 is added (eg, distributed) to the base level (ie, the lowest of the plurality of hierarchical levels).

Thus, due to its small size, it is proposed to include the complete basic auxiliary information (basic auxiliary information and optional additional basic auxiliary information) in the base layer to avoid unnecessary fragmentation thereof.

If the compressed audio representation in question contains dependent core auxiliary information (additional core auxiliary information), the method may further comprise (not shown in FIG. 1) decomposing the additional core auxiliary information into a plurality of additional core auxiliary information parts 2130-1,..., 2130-M. supporting information. Portions of additional basic auxiliary information can then be added (eg, distributed) to the base layer. In other words, pieces of additional basic auxiliary information may be included in the base layer. Each piece of additional basic auxiliary information may be associated with a corresponding layer and may include information that specifies the decoding of one or more components assigned to the corresponding layer depending on other components assigned to the corresponding layer and any layers below the corresponding layer.

Thus, while the independent basic auxiliary information BSI[ (basic auxiliary information) 2120 is left unchanged for assignment, the dependent basic auxiliary information must be processed specifically for layered encoding so that

- 11 043574 allow correct decoding on the receiver side on the one hand, and reduce the size of the dependent basic auxiliary information for transmission on the other hand. It is proposed to decompose the dependent basic auxiliary information into parts designated as BSID, _m , w=1,...,M, where the m-th part contains the dependent basic auxiliary information for each of the components BSRCj, J _m-1 <j<J _m of the base compressed audio representation assigned to the th-th layer, under the assumption that optional dependent base auxiliary information exists for the compressed audio representation in question. In case corresponding dependent auxiliary information does not exist, for the compressed audio representation of the BSID parts, _m may be assumed to be empty.

Each piece of dependent basic auxiliary information BSID, _m may depend on all components BSRCj, 1<j<J _m contained in all layers up to the mth (i.e., contained in all layers j=1,..., m).

If the BSII packet of independent basic auxiliary information is negligibly small in size, it is reasonable to hold it as a whole and append it to the base layer. Optionally, a similar decomposition as for dependent basic auxiliary information can also be performed for independent basic auxiliary information, providing BSII packets, _m , m=1,...,M. This is useful for reducing the size of the base layer by adding (assigning) pieces of independent base auxiliary information to layers with corresponding components of the base compressed audio representation.

At step S1040, a plurality of enhancement auxiliary information pieces 2140-1,...,2140-M may be determined. Each piece of enhancement auxiliary information may include parameters for enhancing (eg, enhancing) the reconstructed audio representation available from data included in the corresponding layer and any layers below the corresponding layer.

The reason for performing this step is that in the case of multi-level encoding, it is important to realize that the enhancement auxiliary information must be calculated for each additional level, since it is intended to improve the preliminary reconstructed sound (or sound field), which, however, depends on the available levels for reconstruction . In particular, the preliminary reconstructed audio (or sound field) for a given highest decodable level (highest applicable level) depends on the components included in the highest decodable level and any levels below the highest decodable level.

Therefore, the compression must provide M individual enhancement auxiliary information data packets (enhancement auxiliary information pieces), denoted by ESI _m , m=1,...,M, where the enhancement auxiliary information in the mth data packet ESI _m is calculated to improve a representation of sound (or sound field) derived from all data contained in the base layer of enhancement layers with indices below m (eg, all data contained in the mth layer and any levels below the mth layer).

In step S1050, a plurality of enhancement auxiliary information pieces 2140-1,...,2140-M are assigned (eg, added or allocated) to a plurality of layers. Each of the plurality of pieces of enhancement auxiliary information is assigned to a corresponding one of the plurality of levels. For example, each of the plurality of layers includes a corresponding piece of enhancing auxiliary information.

The assignment of the base and/or enhancement auxiliary information to the respective layers may be indicated in configuration information that is generated by the encoding method. In other words, the correspondence between the basic and/or enhancement auxiliary information and the corresponding layers may be indicated in the configuration information. In addition, the configuration information may indicate, for each layer, components of the underlying compressed audio representation that are assigned (eg, enabled) to that layer. Portions of additional basic auxiliary information included in the base layer may still correspond to layers other than the base layer.

To summarize, the compression stage provides a frame data packet, designated FRAME, which has the following composition:

FRAME = [BSRCi ... BSRC _; BSIj BSI _Dil ... BSI _DM ES^ ... ESI _M ] ₍₁₎

In addition, the BSII and BSI packets _D , _m for m=1,...,M could be combined into a single BSI packet, in which case the frame data packet designated FRAME would have the following composition:

FRAME = [BSRCi BSRC ₂ ... BSRC _; BSI E5C ESI ₂ ... ESI _M ] ₍₂₎

The order of individual payloads with a frame data packet can generally be arbitrary.

Individual data packets can then be grouped into payloads, which are defined as special data packets that contain a validity flag, a value indicating their size, and the actual compressed presentation data. The use of payloads allows simple demultiplexing at the receiver end, offering the advantage of being able to discard irrelevant payloads without having to analyze them. One possible grouping is given as an assignment (eg allocation) of each BSRCj packet j=1,...,J to an individual payload denoted as ^{in ρ} ί, assignment (eg allocation) of the mth data packet of enhancement auxiliary information ESIm and m th data packet BSIDm dependent auxiliary information to one enhancement payload, denoted as ^EP t m=1,...,M;

assigning the BSII Independent Basic Ancillary Information Packet to a separate Ancillary Information Payload designated BSIP.

Optionally, if the size of the independent basic auxiliary information is large, each m-th of its components, BSI _I>m , m=1,...,M, may be assigned (e.g., allocated) to an enhancement payload, i.e.

In this case, the BSIP Ancillary Information payload is empty and may be ignored.

Another option is to assign all dependent BSID data packets, _m basic auxiliary information to the BSIP auxiliary information payload, which is reasonable if the size of the dependent basic auxiliary information is small.

Ultimately, a frame data packet, designated FRAME, may be provided having the following composition:

FRAME = [BPj ...^P _] BSlPEP ₁ -EP _m ] _m

The order of individual payloads with a frame data packet can generally be arbitrary.

The method may further comprise (not shown in FIG. 1) generating, for each of the multiple layers of a transport layer packet (e.g., base layer packet 2200 and M-1 packets 2300-1, .., 2300-(M-1)) an enhancement layer including data of the corresponding layer (eg, components, base auxiliary information and enhancement auxiliary information for the base layer, or components and enhancement auxiliary information for one or more enhancement layers).

Transport layer packets for different layers may have different transmission priorities. Thus, the method may further comprise (not shown in FIG. 1) generating a transport stream for transmitting data of multiple layers, with the base layer having the highest transmission priority and the hierarchical enhancement layers having decreasing transmission priorities. In this case, a higher transmission priority may correspond to a greater degree of error protection, and vice versa.

Unless the steps require certain other steps as prerequisites, the above steps may be performed in any order, and it is intended that the exemplary order shown in FIG. 1 is not restrictive.

Fig. 3 illustrates a method for decoding a compressed audio or sound field representation for decoding or reconstruction. Examples of the corresponding reception and recovery stage are schematically illustrated in the block diagrams of FIGS. 4 A and 4 B.

As follows from the previous description, a compressed audio representation can be encoded at multiple hierarchical levels. A plurality of layers may have components assigned to them (eg, may include) components of a basic compressed audio representation, the components being assigned to corresponding layers in respective component groups. The base layer may include basic auxiliary information for decoding the underlying compressed audio representation. Each layer may include one of the above-mentioned pieces of enhancement auxiliary information including parameters for enhancing the underlying rendered audio representation available from the data included in the corresponding layer and any layers below the corresponding layer.

The proposed method can be implemented on a frame-by-frame basis (i.e., frame by frame). In particular, the reconstructed representation of the sound or sound field can be generated for successive time intervals, for example, time intervals of equal size. The time slots can be, for example, frames. The steps described below can be performed for each successive time intervals (eg, frames).

In step S3010, data payloads (eg, transport layer packets) corresponding to the plurality of layers are received. The data payloads may be received as part of a bitstream that contains a compressed PLA representation of a sound or sound field, a representation corresponding to multiple hierarchical levels. Hierarchical levels include a base level and one or more hierarchical enhancement levels. The plurality of layers have assigned to them components of the underlying compressed representation of the sound or sound field. The components are assigned to the corresponding levels in the corresponding component groups.

The individual layer packets may be multiplexed to provide the received frame packet with a complete compressed audio representation. The received frame packet may be designated as

[BSIj BSI _D ί BSI _{D m} ESI^ BSRC^ ... BSRC^ jj ... ESI^

Alternatively, the BSII and BSI packets _D , _m for m=1, ..., M are combined into a single BSI packet, the individual layer packets may be multiplexed to provide the received frame packet of a full compressed audio representation, denoted by

I[BSI ESL· BSRCi ... BSRCA _Ί τ ... ESI _M BSRC. ...BSRC,1

[ ^{1 1 7} 1 (5)

In terms of payloads, the received frame packet can be given as

FRAME = [VR _g ...VR _] V5GR~ЁР ₁ - ₍₆₎

The received frame packet may then be transmitted to decompressor or decoder 4100. If the individual layer transmission was error-free, the validity flag of at least a portion of the contained enhancement ancillary information payload (eg, a corresponding portion of the enhancement ancillary information) is set to true. In the event of an error due to an individual layer transmission, the correctness flag in at least the enhancement ancillary information payload at that layer is set to false. Therefore, the correctness of a layer packet can be determined based on the correctness of the enhancement auxiliary information payload contained (eg, based on its correctness flag).

At decompressor 4100, the received frame packet may be demultiplexed. For this purpose, the size information of each payload can be used to avoid unnecessary data analysis of individual payloads.

In step S3020, a first layer index indicating the highest layer (eg, the highest usable layer or the highest decodable layer) is determined from the plurality of layers to be used for decoding the base compressed audio representation into the base reconstructed audio or sound field representation.

In addition, in step S3020, a value (eg, level index) in the highest level (highest applicable level) that will be used to restore the basic audio representation may be selected. The highest enhancement layer that will actually be used to restore the base audio representation is specified as NB-1. Since each layer contains exactly one enhancement auxiliary information payload (a portion of the enhancement auxiliary information), it can be determined based on the enhancement auxiliary information payload whether the containing level is correct (for example, it was accepted correctly). Therefore, selection can be achieved using all the enhancement auxiliary information payloads ESI _m , m=1,...,M (or, respectively, ^m, m=1,...,M).

In step S3030, a basic reconstructed audio representation is obtained. The basic reconstructed audio representation can be obtained from the components assigned to the highest applicable level indicated by the first level index, and any levels below that highest applicable level using the basic auxiliary information (or in general using the basic auxiliary information).

The base compressed audio representation component payloads BSRC ₁ ,...,BSRCJ may be provided along with (all) the base auxiliary information payloads (e.g. BSI or BSII and BSI _D , _m , m=1, ..., M) and the value NB to the base representation recovery processor 4200. The base representation reconstruction processor 4200 (illustrated in FIGS. 4A and 4B) reconstructs the base audio representation (or sound field) using only those components of the compressed audio base representation contained in the lowest NB layers, which are the base layer and NB-1. improving levels (i.e., levels up to the level indicated by the first level index). As an alternative, the base representation recovery processor 4200 may be provided with only the base audio compressed representation component payloads contained in the lowest NB layers along with the corresponding base ancillary information payloads.

The required information about which components of the underlying compressed audio representation (or sound field) are contained at the individual layers is assumed to be known to the decompressor 4100 from the configuration information data packet that is assumed to be sent and received before the frame data packets.

To provide BSI data packets _D , _m , m=1, ..., N _B of dependent auxiliary information and ESINE data packet of enhancement auxiliary information, all enhancement payloads can be input to the partial analyzer 4400 (see FIG. 4B) of the decompressor 4100 together with NE value and N _B value.

- 14 043574

The analyzer can discard all payloads and data packets that will not be used for the actual recovery. If the NE value is zero, then all enhancement ancillary information data packets may be assumed to be empty.

If the base layer includes at least one dependent base auxiliary information payload (part of the additional base auxiliary information) corresponding to the corresponding layer, decoding each individual dependent base auxiliary information payload (for example, BSI _D ,m, m=1, . .., N _B (additional basic auxiliary information part)) may include (i) decoding the additional basic auxiliary information portion by reference to components assigned to its corresponding layer and any layers below the corresponding layer (pre-decoding), and (ii) correcting a portion of the additional basic auxiliary information by reference to the components assigned to the highest applicable level and any levels between the highest applicable level and the corresponding level (correction). Here, the additional basic auxiliary information corresponding to the corresponding layer includes information that determines the decoding of one or more components among the components assigned to the corresponding layer depending on the other components assigned to the corresponding layer and any layers below the corresponding layer.

The basic reconstructed audio representation may then be derived (eg, generated) from the components assigned to the highest applicable level and any levels below the highest applicable level, using the basic auxiliary information and the adjusted additional basic auxiliary information portions derived from the additional basic auxiliary information portions. , corresponding to levels up to the highest applicable level.

In particular, pre-decoding each payload BSI _D , _m , m=1, ..., N _B , may involve exploiting its dependence on the first J _m -1 components BSRC1, ..., BSRC( _Jm )_1 base th compressed representation of the sound contained in the first m levels, which was assumed at the encoding stage.

Sequential correction of each BSI payload _D , _m , m=1, ..., N _B , may involve taking into account that the underlying audio component is finally recreated from the first JNB-1 components

BSRC1, ..., BSRC(j _n b)_ the underlying compressed representation of the audio contained in the first N _B >m layers, which is more components than expected for pre-decoding. Therefore, correction can be achieved by discarding inadequate information, which is possible due to the initially assumed property of the dependent base auxiliary information that if some complementary components are added to the underlying compressed audio representation, the dependent base auxiliary information for each individual (complementary) component becomes a subset of the original one.

In step S3040, the second level index may be determined. The second level index may indicate the portion(s) of enhancement auxiliary information that should be used to enhance (eg, enhance) the underlying reconstructed audio representation.

In addition to the first layer index, an NE index (second layer index) of the enhancement auxiliary information payload (part of the second enhancement information) to be used for reconstruction may be determined. The second index N _E of the level can always either be equal to the first index N _B of the level or be equal to zero. Improvement can be achieved either always according to the base sound representation derived from the highest applicable level, or never.

At step S3050, the reconstructed audio or sound field representation is obtained (eg, generated) from the base reconstructed audio representation with reference to the second level index.

Thus, the reconstructed audio representation is obtained by (parametrically) enhancing or extending the basic reconstructed audio representation, for example, by using the enhancement auxiliary information (a portion of the enhancement auxiliary information) indicated by the second level index. As discussed below, the second level index may indicate not to use any enhancement auxiliary information at all at this stage. The recreated sound representation will then match the base recreated sound representation.

For this purpose, the reconstructed audio base representation together with all enhancement auxiliary information payloads ESI1,...,ESI _M , base auxiliary information payloads (e.g. BSI or BSI _b and BSI _D , _m , m=1, ..., M), and the NE value are provided to the enhanced representation recovery processor 4300 (illustrated in FIGS. 4A and 4B), which calculates the final enhanced audio (or sound field) representation 2100' using only the enhancement auxiliary information payload ESINE, and discarding all

- 15 043574 other payloads of improving auxiliary information. As an alternative, only the enhancement auxiliary information payload ESINE may be provided to the enhancement view recovery processor 4300 instead of all the enhancement auxiliary information payloads. If the NE value is zero, all enhancement auxiliary information payloads are discarded (or alternatively, no enhancement auxiliary information payload is provided) and the recreated final enhanced audio representation 2100' is equal to the recreated main audio representation. The ESINE enhancement auxiliary information payload may be obtained by the partial analyzer 4400.

Fig. 3 also generally illustrates the decoding of a compressed HLA representation based on basic auxiliary information that is associated with a base layer and based on enhancement auxiliary information that is associated with one or more hierarchical enhancement layers.

Unless the steps require certain other steps as prerequisites, the above steps may be performed in any order, and it is intended that the exemplary order shown in FIG. 3 is not restrictive.

Next, details of selecting layers to restore (selecting first and second layer indices) in steps S3020 and S3040 will be described.

Determining the first level index may include determining for each level whether the corresponding level was received correctly. Defining the first level index may further include defining the first level index as the index of that level that is immediately below the lowest level that was not correctly received. Whether a layer has been received correctly can be determined by judging whether the enhancement auxiliary information payload of that layer has been correctly received. This, in turn, can be done by evaluating the correctness flags in the enhancement ancillary information payloads.

Determining the second level index may generally involve either defining the second level index to be equal to the first level index, or defining an index value as the second level index (for example, index value 0) that indicates that no enhancement should be used. supporting information when obtaining a reconstructed representation of the sound.

In the case that all data packets of a frame can be recovered independently of each other, both the NB number of the highest layer (highest applicable layer) to actually use for recovering the basic audio representation, and the enhancement auxiliary information payload index NE to use for recovery can be set equal to the highest number of valid enhancement ancillary information payloads, which itself can be determined by evaluating the correctness flags in the enhancement ancillary information payloads. By using knowledge of the size of each enhancement auxiliary information payload, complex analysis of the actual payload data to determine its correctness can be avoided.

Thus, the second level index can be determined to be equal to the first level index if the compressed audio representations for successive time intervals can be decoded independently. In this case, the reconstructed base audio representation may be enhanced based on the highest applicable level enhancement auxiliary information payload.

In case this differential reconstruction with inter-frame dependencies is used, the solution from the previous frame should be considered in addition. It should be noted that with differential reconstruction, typically independent frame data packets are transmitted at regular time intervals to allow reconstruction to begin at times when the determination of the NB and NE values become frame independent, and is performed as described above.

For a detailed explanation of the proposed frame-dependent solution, the highest number (eg, level index) of the correct enhancement auxiliary information payload for the kth frame is denoted as L(k), the highest level number (eg, level index) to select and use for restoring the basic audio representation is designated as NB(k), and the number (eg, level index) of the enhancement auxiliary information payload to use for restoration is designated as NE(k).

Using these notations, the highest level number to use to reconstruct the underlying sound representation NB(k) can be calculated according to

W _B (k) = min(/V _B (k - l),L(k)). ₍₇₎

By choosing N _B (k) no greater than N _B (k-1) and L (k), it is ensured that all information required for differential reconstruction of the underlying audio representation is available.

Thus, if compressed audio representations for successive time intervals

- 16 043574 (for example, frames) cannot be decoded independently of each other, the determination of the first level index may include determining for each level whether the corresponding level was received correctly, and determining the first level index for a given time interval as the lesser index of the first index the level of the time interval preceding the given time interval, and the level index that is immediately below the lowest level that was not correctly accepted.

The enhancement auxiliary information payload number NE(k) to be used for recovery can be determined according to „ No.(0 ifN _B W = N _B No.-l)

Oh else

L and eise ₍ θ _}

However, selecting 0 for N _E (k) indicates that the reconstructed base representation of the sound should not be improved or expanded using enhancing auxiliary information.

This means in particular that, provided that the highest layer number N _B (k) to be used for reconstructing the basic audio representation does not change, the same corresponding enhancement layer number is selected. However, in case of changing N _B (k), the improvement is prohibited by setting N _E (k) equal to zero. Due to the assumed differential recovery of the enhancement auxiliary information, changing it according to N _B (k) is not possible since this would require restoring the corresponding level of enhancement auxiliary information in the previous frame, which is assumed not to have been performed.

Thus, if compressed audio representations for successive time intervals (eg, frames) cannot be decoded independently of each other, determining the second level index may include determining whether the first level index for a given time interval is equal to the first level index for a previous time interval. If the first level index for a given time interval is equal to the first level index for a previous time interval, the second level index for a given time interval may be determined (eg, selected) to be equal to the first level index for a given time interval. On the other hand, if the first level index for a given time interval is not equal to the first level index for a previous time interval, the index value may be determined (eg, selected) as the second level index, which indicates that no enhancement auxiliary information should be used when receiving a reconstructed representation of the sound.

Alternatively, if during recovery all enhancement auxiliary information payloads with numbers up to N _E (k) are recovered in parallel, the selection rule in equation (4) can be replaced:

= . _Q.

E D (U)

Finally, it should be noted that for differential reconstruction, the number of the highest usable layer N _B can only increase in independent frame data packets, while decrement is possible in each frame.

It is understood that the proposed layered compressed audio representation method can be implemented by an encoder for layered compressed audio representation encoding. Such an encoder may include corresponding blocks configured to perform the corresponding steps described above. An example of such an encoder 5000 is illustrated schematically in FIG. 5. For example, such encoder 5000 may include a component division unit 5010 configured to perform the above step S1010, a component assignment unit 5020 configured to perform the above step S1020, a basic auxiliary information assignment unit 5030 configured to perform the above step S1030, an enhancement auxiliary information splitting unit 5040 configured to execute the above-mentioned step S1040, and an enhancement auxiliary information assigning unit 5050 configured to execute the above-mentioned step S1050. It is further understood that the corresponding blocks of such an encoder may be implemented by a processor 5100 of a computing device, which is configured to perform the processing performed by each of the respective blocks, i.e., it is configured to perform some or all of the steps mentioned above, as well as any additional steps of the proposed encoding method. The encoder or computing device may further include a memory 5200 that can be accessed by the processor 5100.

It is further contemplated that the proposed method for decoding a compressed audio representation that is encoded in a plurality of hierarchical levels can be implemented by a decoder for decoding a compressed audio representation that is encoded in a plurality of hierarchical levels. Such a decoder may include corresponding blocks configured to perform the corresponding steps described above. An example of such a decoder 6000 is illustrated schematically in FIG. 6. For example, such a decoder 6000 may include a receiving unit 6010 configured to

-

Claims (8)

complete the above-mentioned step S3010, the first level index determining unit 6020, configured to perform the above-mentioned step S3020, the base recreation unit 6030, configured to perform the above-mentioned step S3030, the second level index determining unit 6040, configured to perform the above-mentioned step S3040, and a block 6050 configured to perform the above-mentioned step S3050. It is further understood that the corresponding blocks of such an encoder may be implemented by a processor 6100 of a computing device, which is configured to perform the processing performed by each of the respective blocks, i.e., it is configured to perform some or all of the steps mentioned above, as well as any additional steps of the proposed encoding method. The encoder or computing device may further include a memory 6200 that can be accessed by the processor 6100. It should be noted that the description and drawings only illustrate the principles of the proposed methods and devices. Thus, it will be apparent that those skilled in the art will be able to create various structures that, although not expressly described or shown herein, embody the principles of the invention and are included within the spirit and scope thereof. In addition, all examples given herein are expressly intended for educational purposes only to assist the reader in understanding the principles of the proposed methods and devices, and the concepts brought by the inventors to the development of the field of technology, and should be construed as not limiting such special in the manner of the given examples and conditions. Moreover, all statements herein setting forth the principles, aspects and embodiments of the invention, as well as specific examples thereof, are intended to cover their equivalents. The methods and apparatus described herein may be implemented as software, firmware, and/or hardware. Some components, for example, may be implemented as software running on a digital signal processor or microprocessor. Other components, for example, may be implemented as hardware and/or as application-specific integrated circuits. Signals encountered in the described methods and apparatus may be stored on media such as random access memory or optical storage media. They can be carried over networks such as radio networks, satellite networks, wireless networks or wired networks such as the Internet. Literature Cited 1: ISO/IEC JTC1/SC29/WG11 23008- 3:2015(E). Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio, February 2015. 2: ISO/IEC JTC1/SC29/WG11 23008- 3:2015/PDAM3. Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio, AMENDMENT 3: MPEG-H 3D Audio Phase 2, July 2015. CLAIM 1. A method for decoding a compressed representation of an audio or sound field of a Higher Order Ambisonics (HOA) system, the method comprising the steps of receiving a bit stream containing a compressed representation of the HOA, wherein the bit stream contains a plurality of hierarchical levels that include a base level and two or more hierarchical enhancement layers, wherein the bitstream contains at least a data payload corresponding to the plurality of hierarchical layers, and wherein the bitstream also contains basic side information that is associated with the base layer, and enhancement side information that is associated with two or more hierarchical enhancement layers, wherein the plurality of hierarchical layers have compressed HLA representation components of the sound or sound field assigned to them, wherein the two or more hierarchical enhancement layers comprise the highest applicable hierarchical enhancement layer, wherein each of the two or more hierarchical enhancement layers includes a portion of the enhancement auxiliary information including parameters for improving the underlying rendered audio representation available from the data included in the corresponding level and any levels below the corresponding level; and decoding the compressed SLAM representation based on the basic auxiliary information that is associated with the base layer and based on the enhancement auxiliary information portion that - 18 043574 is associated with the highest applicable hierarchical improvement level, and based on the second part of the improvement auxiliary information, which is associated with any other level of two or more hierarchical improvement levels. 2. The method according to claim 1, in which the parameters contain at least one of the following: spatial prediction, synthesis of directional subband signals and parametric duplication of the sound environment. 3. The method of claim 1, wherein the enhancing auxiliary information includes information that enables missing parts of the sound or sound field to be predicted from the directional signals. 4. The method according to claim 1, further comprising the steps of determining for each level whether the corresponding level was accepted correctly; and determining the index of the level immediately below the lowest level that was not accepted correctly. 5. The method of claim 4, further comprising determining an additional level index that is either equal to the level index or indicates exclusion of enhancement auxiliary information during decoding. 6. The method of claim 1, wherein the base layer includes at least one piece of additional base auxiliary information associated with the corresponding layer, and includes information that determines the decoding of one or more components among the components assigned to the corresponding layer, depending on other components assigned to the corresponding level and any levels below the corresponding level, the method for each piece of additional basic auxiliary information comprising decoding the piece of additional basic auxiliary information by reference to components assigned to its corresponding level and any levels below the corresponding one level; and adjusting a portion of the additional basic auxiliary information by reference to components assigned to the highest applicable hierarchical enhancement level and any levels between the highest applicable hierarchical enhancement level and the corresponding layer, wherein the basic reconstructed audio representation is obtained from the components assigned to the highest applicable hierarchical enhancement level and any levels below the highest applicable hierarchical enhancement level, using the basic auxiliary information and adjusted portions of the additional basic auxiliary information obtained from the portions of the additional basic auxiliary information corresponding to the levels up to the highest applicable hierarchical enhancement level. 7. A permanent storage medium carrying computer-executable code, which, when executed by a processor, causes the processor to perform the method of claim 1. 8. An apparatus for decoding a compressed representation of an audio or audio field of a Higher Order Ambisonics (HOA) system, the apparatus comprising a receiver for receiving a bitstream containing a compressed representation of the HOA, wherein the bitstream comprises a plurality of hierarchical levels that include a base layer and two or more hierarchical enhancement layers, wherein the bitstream contains at least a data payload corresponding to the plurality of hierarchical layers, and wherein the bitstream also contains basic auxiliary information that is associated with the base layer, and enhancement auxiliary information that is associated with two or more hierarchical enhancement layers, wherein the plurality of hierarchical layers have compressed audio or sound field HLA representation components assigned to them, wherein the two or more hierarchical enhancement layers comprise the highest applicable hierarchical enhancement layer, and wherein each of the two or more hierarchical enhancement layers includes a portion of the enhancement supporting information including parameters for improving the underlying rendered audio representation available from the data included in the corresponding levels and any levels below the corresponding level; and a decoder for decoding the compressed NOA representation based on the base support information that is associated with the base layer, and based on the portion of the enhancement support information that is associated with the base layer, and based on the portion of the enhancement support information that is associated with the highest applicable hierarchical enhancement layer. , and not based on a second piece of improving auxiliary information that is associated with any other level of two or more hierarchical improving levels. -