RU2797457C1 - Determining the coding and decoding of the spatial audio parameters - Google Patents

Abstract

FIELD: audio coding.

SUBSTANCE: present invention relates to an apparatus and methods for coding parameters associated with a sound field. The apparatus comprises means configured to generate spatial audio direction metadata parameters for a time-frequency block; generating coded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; outputting or storing coded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for coded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for coded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for coded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution are greater than a predetermined threshold.

EFFECT: possibility of obtaining the most efficient quantization resolution for encoding spatial audio parameters.

17 cl, 11 dwg

Description

FIELD OF TECHNOLOGY

The present invention relates to, but is not limited to, apparatus and methods for encoding direction-related parameters of a sound field for encoding direction-related time-frequency domain parameters for an audio encoder and decoder.

BACKGROUND OF THE INVENTION

Parametric spatial audio processing is audio signal processing in which the spatial aspect of a sound is described using a set of parameters. For example, in parametric spatial audio capture from microphone arrays, a typical and effective choice is to estimate, based on microphone array signals, a set of parameters such as sound directions in frequency bands and the ratio between the directional and non-directional parts of the captured sound in frequency bands. These parameters are known to describe well the perceived spatial properties of the captured sound at the location of the microphone array. These parameters can be appropriately used for surround sound synthesis, for stereo headphones, for loudspeakers, or for other formats such as ambisonics.

Directions and ratios of directed to total energy in frequency bands thus represent a parameterization that is particularly effective for capturing spatial audio.

A set of parameters consisting of a direction parameter in frequency bands and an energy ratio parameter in frequency bands (indicating the directionality of sound) can also be used as spatial metadata (which can also include other parameters such as coherence, propagation coherence, number of directions, distance, and etc.) for the audio codec. For example, these parameters can be estimated from the audio signals captured by the microphone array, and, for example, a stereo signal can be generated from the microphone array signals for transmission with spatial metadata. The stereo signal may be encoded with, for example, an AAC encoder. The decoder may decode the audio signals into PCM signals and process the audio in frequency bands (using spatial metadata) to obtain a spatial output such as binaural output.

The solution described above is particularly suitable for encoding captured spatial audio from microphone arrays (eg, in mobile phones, virtual reality cameras, standalone microphone arrays). However, it may be desirable for such an encoder to also have other types of input signals than those captured by the microphone array, such as speaker signals, audio object signals, or ambisonic signals.

First-order Ambisonics (FOA) input analysis to extract spatial metadata has been described in detail in scientific literature related to Directional Audio Coding (DirAC) and Harpex (Harmonic planewave expansion) . This is due to the fact that there are microphone arrays that directly transmit the FOA signal (more precisely, its variant, the format B signal), so the analysis of such an input has been the subject of research in this area.

An additional input for the encoder is also a multi-channel speaker input, such as 5.1 or 7.1 surround inputs.

However, with respect to the directional components of the metadata, which may include the angular height, azimuth (and energy ratio, which is 1-diffusion) of the resulting direction for each time/frequency subband considered, the quantization of these directional components is a hot topic of research.

SUMMARY OF THE INVENTION

According to a first aspect, an apparatus is provided, comprising means configured to: generate spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; outputting or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined threshold, while the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for a time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits.

Means configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution may be configured to: determine a first quantization resolution to map between the spatial audio direction metadata parameter values and the index value; generating indices associated with spatial audio direction metadata parameters based on the mapping using the first quantization resolution; encoding, optionally, indexes using fixed rate coding or entropy coding based on whether fixed rate coding or entropy coding uses fewer bits.

Means configured to determine a first quantization resolution for mapping between spatial audio direction metadata parameter values and an index value may be configured to determine a first quantization resolution for mapping between spatial audio direction metadata parameter values and an index value based on an energy ratio value associated with the spatial audio direction metadata parameter.

Means configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution when the difference between the specified number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first resolution quantization is within a predetermined threshold, may be configured to: determine a second quantization resolution to map between spatial audio direction metadata parameter values and an index value; generating indices associated with the spatial audio direction metadata parameters based on the mapping using the second quantization resolution for the spatial audio direction metadata parameters that have been encoded at a fixed rate using the first quantization resolution.

The means may also be configured to output or store: entropy-coded indices associated with the spatial audio direction metadata parameters based on a mapping using a first quantization resolution for the spatial audio direction metadata parameters; and fixed rate encoded indices associated with the spatial audio direction metadata parameters based on a mapping using a second quantization resolution for the spatial audio direction metadata parameters.

The means may also be configured to order the encoded indices such that the entropy encoded indices precede the fixed rate encoded indices.

The means may also be configured to generate an indicator when the first or second quantization resolution is used.

Means configured to generate encoded spatial audio direction metadata parameters for a time-frequency block based on the third quantization resolution may be configured to: determine a third quantization resolution to map between the spatial audio direction metadata parameter values and the index value based on that the number of bits used for fixed rate coding using the third quantization resolution is always equal to or less than the specified number of bits; generating indices associated with spatial audio direction metadata parameters based on the mapping using the third quantization resolution; and encoding, optionally, the indices using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits.

The means may also be configured to output selectively encoded indices using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits.

The means may also be configured to generate an indicator when a third quantization resolution is determined.

According to a second aspect, an apparatus is provided, comprising means configured to: receive encoded spatial audio direction metadata parameters for a time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded spatial audio direction metadata parameters for a time-frequency block based on a quantization resolution that is always equal to or less than a specified number of bits, when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, the means is configured to: decode a first part of the encoded spatial audio direction metadata parameters for the time-frequency block based on another quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decoding, when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded parameters of the spatial audio direction metadata for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

The means may also be configured to determine said different quantization resolution to map between the spatial audio direction metadata parameter values and the index value.

Means configured to determine said different quantization resolution to map between the spatial audio direction metadata parameter values and the index value may be configured to determine said different quantization resolution based on the energy ratio value associated with the spatial audio direction metadata parameter.

The means may also be configured to determine the reduced bit quantization resolution to map between the spatial audio direction metadata parameter values and the index value.

The means may also be configured to generate a mapping from the indices associated with the spatial audio direction metadata parameters to an angular height value and/or an azimuth value based on the quantization resolution.

According to a third aspect, a method is provided, including: generating spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; deriving or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined threshold, wherein the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for a time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits.

Generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution may include: determining a first quantization resolution to map between spatial audio direction metadata parameter values and an index value; generating indices associated with spatial audio direction metadata parameters based on the mapping using the first quantization resolution; encoding, optionally, the indexes using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits.

Determining a first quantization resolution for mapping between spatial audio direction metadata parameter values and an index value may include determining a first quantization resolution for mapping between spatial audio direction metadata parameter values and an index value based on an energy ratio value associated with the spatial audio direction metadata parameter .

Generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the difference between the specified number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is in within a given threshold, may include: determining a second quantization resolution to map between spatial audio direction metadata parameter values and an index value; generating indices associated with the spatial audio direction metadata parameters based on the mapping using the second quantization resolution for the spatial audio direction metadata parameters that have been encoded at a fixed rate using the first quantization resolution.

The method may also include deriving or storing entropy-coded indices associated with the spatial audio direction metadata parameters based on a mapping using a first quantization resolution for the spatial audio direction metadata parameters; and fixed rate encoded indices associated with the spatial audio direction metadata parameters based on a mapping using a second quantization resolution for the spatial audio direction metadata parameters.

The method may also include ordering the encoded indices such that the entropy-encoded indices precede the fixed-rate-encoded indices.

The method may also include generating an indicator when the first or second quantization resolution is used.

Generating encoded spatial audio direction metadata parameters for a time-frequency block based on a third quantization resolution may include: determining a third quantization resolution to map between the spatial audio direction metadata parameter values and an index value based on that the number of bits used to encode with fixed rate using the third quantization resolution, always equal to or less than the specified number of bits; generating indices associated with spatial audio direction metadata parameters based on the mapping using the third quantization resolution; and encoding, optionally, the indexes using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits.

Further, the method may include deriving selectively encoded indices using fixed rate coding or entropy coding based on whether the fixed coding or entropy coding uses fewer bits.

The method may also include generating an indicator when the third quantization resolution is determined.

According to a fourth aspect, a method is provided, including: receiving encoded spatial audio direction metadata parameters for a time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded parameters of the spatial audio direction metadata for the time-frequency block based on the quantization resolution, which is always equal to or less than the specified number of bits, when the indicator identifies that the encoded parameters of the spatial audio direction metadata were encoded based on the quantization resolution, which is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, the method includes: decoding a first part of the encoded spatial audio direction metadata parameters for the time-frequency block on based on another quantization resolution, the first part containing entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decoding when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

The method may also include determining said different quantization resolution to map between the spatial audio direction metadata parameter values and the index value.

Determining said different quantization resolution to map between the spatial audio direction metadata parameter values and the index value may include determining said different quantization resolution based on the energy ratio value associated with the spatial audio direction metadata parameter.

The method may include determining a reduced bit quantization resolution to map between spatial audio direction metadata parameter values and an index value.

The method may include generating a mapping from indices associated with spatial audio direction metadata parameters to an angular height value and/or an azimuth value based on a quantization resolution.

According to a fifth aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, by means of the at least one processor, cause the device to perform at least: generating metadata parameters of the direction of the spatial audio signal for the time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; deriving or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined threshold, while the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for a time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits.

An apparatus that generates encoded spatial audio direction metadata parameters for a time-frequency block based on the first quantization resolution may: determine a first quantization resolution to map between the spatial audio direction metadata parameter values and the index value; generate indexes associated with spatial audio direction metadata parameters based on the mapping using the first quantization resolution; selectively encode the indexes using fixed rate coding or entropy coding depending on whether the fixed coding or entropy coding uses fewer bits.

The device that determines the first quantization resolution for mapping between the spatial audio direction metadata parameter values and the index value can: determine the first quantization resolution for mapping between the spatial audio direction metadata parameter values and the index value based on the energy ratio value associated with the metadata parameter direction of the spatial audio signal.

An apparatus that generates encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution when the difference between the specified number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution , is within a predetermined threshold, may: determine a second quantization resolution for mapping between spatial audio direction metadata parameter values and an index value; generate indexes associated with the spatial audio direction metadata parameters based on the mapping using the second quantization resolution for the spatial audio direction metadata parameters that have been encoded at a fixed rate using the first quantization resolution.

The device may output or store: entropy-coded indices associated with the spatial audio direction metadata parameters based on a mapping using a first quantization resolution for the spatial audio direction metadata parameters; and fixed rate encoded indices associated with the spatial audio direction metadata parameters based on a mapping using a second quantization resolution for the spatial audio direction metadata parameters.

The device may order the encoded indices such that the entropy-encoded indices precede the fixed-rate-encoded indices.

The device may generate an indicator when the first or second quantization resolution is used.

A device that generates encoded spatial audio direction metadata parameters for a time-frequency block based on the third quantization resolution may: determine a third quantization resolution to map between the spatial audio direction metadata parameter values and the index value based on that the number of bits used for fixed rate coding using the third quantization resolution, always equal to or less than the specified number of bits; generate indexes associated with the spatial audio direction metadata parameters based on the mapping using the third quantization resolution; and selectively encode the indexes using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits.

The apparatus can output selectively encoded indices using fixed rate coding or entropy coding based on whether the fixed coding or entropy coding uses fewer bits.

The device may generate an indicator when the third quantization resolution is determined.

According to a sixth aspect, there is provided an apparatus comprising at least one processor and at least one storage device including computer program code, the at least one storage device and the computer program code being configured to, by means of at least one processor, cause the device to perform at least: receiving the encoded parameters of the spatial audio direction metadata for the time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decode the encoded spatial audio direction metadata parameters for the time-frequency block based on a quantization resolution that is always equal to or less than a specified number of bits, when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, the device decodes the first part of the encoded spatial audio direction metadata parameters for the time-frequency block based on the other quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decodes, when the difference between the given number of bits and the number of bits used to encode the first part, is less than the number of bits needed to encode the second part of the encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution, the second part comprising fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decodes the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

The device may also determine said other quantization resolution to map between the spatial audio direction metadata parameter values and the index value.

A device that determines said different quantization resolution for mapping between the spatial audio direction metadata parameter values and the index value can determine said different quantization resolution based on the energy ratio value associated with the spatial audio direction metadata parameter.

The device may also determine the reduced bit quantization resolution to map between the spatial audio direction metadata parameter values and the index value.

In addition, the device may generate a mapping from indices associated with the spatial audio direction metadata parameters to an angular height value and/or an azimuth value based on the quantization resolution.

According to a seventh aspect, an apparatus is provided, comprising: a generating circuit configured to generate spatial audio direction metadata parameters for a time-frequency block; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; a comparison circuit configured to compare the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a predetermined number of bits; an output or storage circuit configured to output or store the encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution , less than the specified number of bits; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number bits, and the difference between the predetermined number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than the predetermined number of bits and is within the predetermined threshold; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number bits and the difference between the predetermined number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the predetermined threshold, the third quantization resolution being determined such that the number of bits used for the encoded the spatial audio direction parameters for the time-frequency block based on the third quantization resolution is always equal to or less than the specified number of bits.

According to an eighth aspect, an apparatus is provided, comprising: a receiving circuit configured to receive encoded spatial audio direction metadata parameters for a time-frequency block; a receiving circuit configured to receive an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; a decoding circuit configured to decode encoded spatial audio direction metadata parameters for a time-frequency block based on a quantization resolution that is always equal to or less than a predetermined number of bits when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on the quantization resolution , which is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, the device comprises: a decoding circuit configured to decode a first part of the encoded spatial audio direction metadata parameters for a time-frequency block based on another quantization resolution, the first part containing entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; a decoding circuit configured to decode when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution , the second part containing the fixed rate encoded spatial audio direction metadata parameters for the time-frequency block based on the reduced bit quantization resolution, otherwise to decode the second part containing the fixed rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution.

According to a ninth aspect, a computer program is provided, containing instructions [or a computer-readable medium containing program instructions] for causing a device to perform at least the following: generating spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; outputting or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number bits and the difference between the predetermined number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the predetermined threshold, the third quantization resolution being determined such that the number of bits used for the encoded the spatial audio direction parameters for the time-frequency block based on the third quantization resolution is always equal to or less than the specified number of bits.

According to a tenth aspect, a computer program is provided, comprising instructions [or a computer-readable medium containing program instructions] for causing a device to at least: receive encoded spatial audio direction metadata parameters for a time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded spatial audio direction metadata parameters for a time-frequency block based on a quantization resolution that is always equal to or less than a specified number of bits, when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than the specified number of bits, performing: decoding the first part of the encoded spatial audio direction metadata parameters for the time-frequency block based on another quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decoding, when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded parameters of the spatial audio direction metadata for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

According to an eleventh aspect, a computer-readable storage medium is provided, comprising program instructions causing a device to perform at least: generating spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; deriving or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number bits and the difference between the predetermined number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the predetermined threshold, the third quantization resolution being determined such that the number of bits used for the encoded the spatial audio direction parameters for the time-frequency block based on the third quantization resolution is always equal to or less than the specified number of bits.

According to a twelfth aspect, a computer-readable storage medium is provided, comprising program instructions causing a device to perform at least: receiving encoded spatial audio direction metadata parameters for a time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded parameters of the spatial audio direction metadata for the time-frequency block based on the quantization resolution, which is always equal to or less than the specified number of bits, when the indicator identifies that the encoded parameters of the spatial audio direction metadata were encoded based on the quantization resolution, which is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, performing: decoding the first part of the encoded spatial audio direction metadata parameters for the time-frequency block based on another quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decoding, when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded parameters of the spatial audio direction metadata for the time-frequency block based on the other quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

According to a thirteenth aspect, an apparatus is provided, comprising: means for generating spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; means for comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a predetermined number of bits; means for outputting or storing the encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits ; means for generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits, and the difference between the predetermined number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than the predetermined number of bits and is within the predetermined threshold; generating circuit means configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than of the specified number of bits and the difference between the specified number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the predetermined threshold, while the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits.

According to a fourteenth aspect, an apparatus is provided, comprising: means for receiving encoded spatial audio direction metadata parameters for a time-frequency block; means for receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; means for decoding the encoded spatial audio direction metadata parameters for the time-frequency block based on a quantization resolution that is always equal to or less than a predetermined number of bits, when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, means for: decoding a first portion of the encoded spatial audio direction metadata parameters for the time-frequency block at based on another quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; means for decoding, when the difference between the predetermined number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution, the second part , containing the fixed rate encoded parameters of the spatial audio direction metadata for the time-frequency block based on the reduced bit quantization resolution, otherwise means for decoding the second part containing the fixed-rate encoded parameters of the spatial audio direction metadata for the time-frequency block on based on the other quantization resolution mentioned.

According to a fifteenth aspect, a computer-readable medium is provided, comprising program instructions causing a device to perform at least: generating spatial audio direction metadata parameters for a time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; deriving or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; a generating circuit configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number bits and the difference between the predetermined number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the predetermined threshold, the third quantization resolution being determined such that the number of bits used for the encoded the spatial audio direction parameters for the time-frequency block based on the third quantization resolution is always equal to or less than the specified number of bits.

According to a sixteenth aspect, a computer-readable medium is provided, comprising program instructions causing a device to perform at least: receiving encoded spatial audio direction metadata parameters for a time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded parameters of the spatial audio direction metadata for the time-frequency block based on the quantization resolution, which is always equal to or less than the specified number of bits, when the indicator identifies that the encoded parameters of the spatial audio direction metadata were encoded based on the quantization resolution, which is always equal to or less than the specified number of bits; and, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than the specified number of bits, performing: decoding the first part of the encoded spatial audio direction metadata parameters for the time-frequency block based on another quantization resolution, wherein the first part contains entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said other quantization resolution; decoding, when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded parameters of the spatial audio direction metadata for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

A device containing means for performing the operations of the method described above.

A device configured to perform the operations of the method as described above.

A computer program containing program instructions that cause the computer to carry out the method described above.

The computer program product stored on the medium can cause the device to perform the method described here.

The electronic device may include the device described here.

The chipset may contain the device described here.

Embodiments of the present invention are directed to solving the problems of the prior art.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention, reference will now be made to the accompanying drawings by way of example.

In FIG. 1 schematically shows a device system suitable for implementing some embodiments of the invention.

In FIG. 2 schematically shows a metadata encoder in accordance with some embodiments of the invention.

In FIG. 3 shows a flowchart of energy ratio encoding and quantization resolution determination as shown in FIG. 2, in accordance with some embodiments of the invention.

In FIG. 4a-4c show flowcharts for generating a direction index and encoding a direction index as shown in FIG. 2, in accordance with some embodiments of the invention.

In FIG. 5 shows a block diagram of the entropy encoding of direction indices as shown in FIG. 4a-4c, in accordance with some embodiments of the invention.

In FIG. 6 shows another block diagram of the direction index entropy coding as shown in FIG. 4a-4c, in accordance with some embodiments of the invention.

In FIG. 7 schematically shows a metadata decoder according to some embodiments of the invention.

In FIG. 8 is a block diagram of the operations of the metadata decoder shown in FIG. 7, in accordance with some embodiments of the invention.

In FIG. 9 schematically shows an example of a device suitable for implementing the device shown.

EMBODIMENTS FOR CARRYING OUT THE INVENTION

The following describes in more detail a suitable apparatus and possible mechanisms for providing efficient metadata parameters derived from spatial analysis. The following discussions describe the multi-channel system with respect to the implementation of a multi-channel microphone. However, as discussed above, the input format can be any suitable input format such as multi-channel loudspeaker, ambisonic (FOA/HOA), etc. It is understood that in some embodiments of the invention, the location of the channel is based on the location of the microphone or is a virtual location or direction. In addition, in the example, the output of the system is a multi-channel loudspeaker device. However, it is understood that the output may be reproduced to the user by means other than speakers. Furthermore, the multi-channel speaker signals may be summarized as two or more playback audio signals.

The metadata includes at least the angular height, azimuth, and energy ratio of the resulting direction for each considered time/frequency subband. The direction parameter components, azimuth and angular height are extracted from the audio data and then quantized to the specified quantization resolution. The resulting indexes must also be compressed for efficient transmission. High bit rates require high quality, lossless metadata coding.

The concept discussed below is to improve the quality of the encoded and quantized metadata representation in situations where, after initial quantization and encoding, the resulting bit rate exceeds the bit rate allowed by the codec. In such embodiments, a method is provided for obtaining an intermediate quantization resolution without any re-evaluation of the entropy coding bits or any additional change signaling. Therefore, reduction is performed only for those subbands that use fixed rate coding, and implicit signaling is implemented by reordering the subbands when recording the bitstream to be output.

In some embodiments of the invention, this can also be implemented using methods that reduce the values of the variables to be encoded. The reduction may be implemented in some embodiments of the invention for the case where there are more symbols. The change can be made by subtracting the index to be encoded from the number of available symbols and encoding the resulting difference. In some embodiments of the invention, to represent the azimuth, this corresponds to the presence of sound sources located with a shift back. In addition, the change can also be implemented in some embodiments of the invention by checking if all indices are even or odd, and encoding the values divided by two. To represent angular height, in some embodiments of the invention, this corresponds to the location of sound sources mainly at the top or bottom of the sound stage.

In some embodiments of the invention, the MASA metadata encoding, for example in the IVAS codec, is configured to first estimate the number of bits for the direction data based on the values of the quantized energy ratios for each time-frequency tile (tile). In addition, the entropy encoding of the original quantization resolution is checked. If the resulting sum is greater than the number of available bits, the number of bits may be proportionally reduced for each time-frequency tile to match the number of available bits, however, the quantization resolution is not unnecessarily adjusted when the bit rate allows (for example, for higher bit rates).

In FIG. 1 shows an example apparatus and system for implementing embodiments of the invention. The system 100 is shown with an "analysis" part 121 and a "synthesis" part 131. The “analysis” part 121 is the part from receiving the multi-channel speaker signals to encoding the metadata and the downmix signal, and the “synthesis” part 131 is the part from decoding the encoded metadata and the downmix signal to presenting the re-generated signal (for example, in the form of a multi-channel signal). loudspeaker).

The inputs to system 100 and "analysis" portion 121 are multi-channel signals 102. The following examples describe microphone channel input, however any suitable input (or synthetic multi-channel) format may be implemented in other embodiments of the invention. For example, in some embodiments of the invention, the spatial analyzer and spatial analysis may be implemented outside of the encoder. For example, in some embodiments of the invention, spatial metadata associated with audio signals may be provided to the encoder as a separate bitstream. In some embodiments of the invention, spatial metadata may be provided as a set of spatial index (direction) values.

The multi-channel signals are transmitted to the downmixer 103 and to the analysis processor 105 .

In some embodiments of the invention, the downmix module 103 is configured to receive multi-channel signals and downmix the signals to a predetermined number of channels and output the downmix signals 104 . For example, the downmix module 103 may be configured to generate a downmix of two audio channels of multi-channel signals. The given number of channels may be any suitable number of channels. In some embodiments of the invention, the downmix module 103 is optional and the multi-channel signals are passed unprocessed to the encoder 107 in the same manner as the downmix signal in this example.

In some embodiments, the analysis processor 105 is also configured to receive multi-channel signals and analyze the signals to create metadata 106 associated with the multi-channel signals and thus associated with the downmix signals 104. The analysis processor 105 may be configured to generate metadata that may comprise, for each time-frequency analysis interval, a direction parameter 108 and an energy ratio parameter 110 (and in some embodiments, a coherence parameter and a diffuseness parameter).

The direction and ratio of the energy may, in some embodiments of the invention, be considered as spatial sound parameters. In other words, the surround sound parameters contain parameters intended to describe the sound field created by multi-channel signals (or two or more reproduced audio signals in general).

In some embodiments of the invention, the generated parameters may differ from one frequency range to another. So, for example, in range X, all parameters are generated and passed, while in range Y, only one of the parameters is generated and passed, and in range Z, no parameters are generated or passed. A practical example of this would be that for some frequency bands, such as the highest band, some parameters are not required for perceptual reasons. Downmix signals 104 and metadata 106 may be passed to an encoder 107.

Encoder 107 may include an audio encoder core 109 that is configured to receive downmix signals 104 (or other signals) and generate a suitable encoding of those audio signals. Encoder 107 in some embodiments of the invention may be a computer (running suitable software stored in memory and at least one processor) or, alternatively, a specific device using, for example, an FPGA or ASIC. Encoding may be implemented using any suitable scheme. In addition, encoder 107 may include a metadata encoder/quantizer 111 that is configured to receive metadata and output information in an encoded or compressed form. In some embodiments, the encoder 107 may also interleave, multiplex into a single data stream, or embed metadata into encoded downmix signals prior to transmission or storage, as shown in FIG. 1 dotted line. The multiplexing may be implemented using any suitable scheme.

On the decoder side, the received or extracted data (stream) may be obtained by the decoder/demultiplexer 133. The decoder/demultiplexer 133 may demultiplex the encoded streams and pass the encoded audio stream to the downmix extractor 135, which is configured to decode audio signals to obtain downmix signals. Similarly, decoder/demultiplexer 133 may include a metadata extractor 137 that is configured to receive encoded metadata and generate metadata. Decoder/demultiplexer 133 in some embodiments of the invention may be a computer (with suitable software stored in memory and at least one processor) or, alternatively, a specific device using, for example, an FPGA or ASIC.

The decoded metadata and downmix audio signals may be passed to the synthesis processor 139 .

The "synthesis" portion 131 of system 100 also shows a synthesis processor 139 configured to receive the downmix and metadata and recreate in any suitable format the synthesized spatial audio in the form of multi-channel signals 110 (this may be a multi-channel loudspeaker format, or in some embodiments, any suitable output format such as stereo signals or ambisonic signals, depending on the use case) based on downmix signals and metadata.

Thus, first the system (part of the analysis) is configured to receive multi-channel audio signals. The system (part of the analysis) is then configured to generate a downmix or otherwise to generate a suitable transport audio signal (eg by selecting some audio channels). The system is then configured to encode for storing/transmitting the downmix signal (or, more generally, the transport signal). Thereafter, the system can store/transmit the encoded mixing signal and metadata. The system can extract/receive the encoded downmix signal and metadata. The system is then configured to extract the downmix and metadata from the encoded downmix parameters and metadata, such as demultiplexing and decoding the encoded downmix parameters and metadata.

The system (synthesis part) is configured to synthesize an output multi-channel audio signal based on the extracted multi-channel audio downmix and metadata.

With regard to FIG. 2, an example analysis processor 105 and metadata encoder/quantizer 111 (as shown in FIG. 1) are described in more detail, in accordance with some embodiments of the invention.

The analysis processor 105 in some embodiments of the invention comprises a time-to-frequency domain converter 201.

In some embodiments, the time-to-frequency domain converter 201 is configured to receive multi-channel signals 102 and apply a suitable time-to-frequency domain transform, such as a Short Time Fourier Transform (STFT), to convert the time domain input signals into suitable time-frequency signals. These time-frequency signals may be transmitted to a spatial analyzer 203 and to a signal analyzer 205 .

Thus, for example, time-frequency signals 202 can be represented in a time-frequency domain representation using

s _i (b,n),

where b is the frequency bin index, n is the frequency-time block (frame) index, ai is the channel index. In another expression, n can be thought of as a time index with a lower sampling rate than the original signals in the time domain. These frequency bins may be grouped into subbands, which group one or more bins into a subband with band index k=0,...,K-1. Each subrange k has the lowest bin b _k,low and the highest bin b _k,high , and the subrange contains all bins b _k,low to b _k,high . The width of the subbands may be approximated by any suitable distribution. For example, the Equivalent Rectangular Bandwidth (ERB) scale or the Bark scale is used.

In some embodiments, analysis processor 105 includes a spatial analyzer 203. Spatial analyzer 203 may be configured to receive time-frequency signals 202 and estimate direction parameters 108 based on those signals. Direction parameters can be determined based on any sound based on the definition of "direction".

For example, in some embodiments of the invention, the spatial analyzer 203 is configured to estimate the direction with two or more input signals. This is the simplest configuration for "direction" estimation, more complex processing can be done with even more signals.

Thus, the spatial analyzer 203 can be configured to provide the azimuth and/or angular height for each frequency band and temporal time-frequency block in the audio frame, denoted as azimuth ϕ(k,n) and angular height θ(k,n) . The direction parameters 108 may also be passed to the direction parser/index generator 215.

The spatial analyzer 203 can also be configured to determine the energy ratio parameter 110. The energy ratio may represent the energy of the audio signal, which is considered to be coming from a given direction. The ratio of directional energy to total energy r(k,n) can be estimated, for example, using a measure of direction estimation stability, or using any measure of correlation, or any other suitable method to obtain a ratio parameter. The energy ratio can be passed to the average energy ratio generator/quantization resolution determination module 211.

Thus, the analysis processor is configured to receive a multi-channel time domain signal or other format such as microphone or ambisonic audio signals.

Thereafter, the analysis processor may apply a time-to-frequency domain transformation (eg, STFT) to generate suitable time-frequency domain signals for analysis, and then apply direction analysis to determine direction and energy ratio parameters.

The analysis processor can then be configured to output the specified parameters.

Although directions and relationships are expressed here for each time index n, in some embodiments of the invention, the parameters can be combined over several time indexes. The same applies to the frequency axis, as it has been shown that the direction of several frequency bins b can be expressed by a single direction parameter in the range k, which includes several frequency bins b. The same applies to all the spatial parameters discussed here.

As also shown in FIG. 2, in accordance with some embodiments of the invention, an example of a metadata encoder/quantizer 111 is shown.

As discussed above, the audio spatial metadata includes azimuth, angular height, and energy ratio data for each subband. In the MASA format, direction data is represented by 16 bits, so azimuth is approximately represented by 9 bits and angular height by 7 bits. The energy ratio is represented by 8 bits. For each frame, there are N=5 subbands and M=4 time blocks, making it necessary to store (16+8)xMxN bits to store uncompressed metadata for each frame. The version with higher frequency resolution may have 20 or 24 frequency subbands. Although the following examples use MASA bit allocations, it is understood that other embodiments of the invention may be implemented with different bit allocation or subband or time block selections, and these are only typical examples.

The metadata encoder/quantizer 111 may comprise an average energy ratio generator/quantization resolution determination module 211. The average energy ratio generator/quantization resolution determination unit 211 may be configured to receive energy ratios and from analysis to generate a suitable ratio encoding based on them. For example, to obtain given energy ratios (eg, directed energy to total energy ratios, as well as diffuse energy to total energy ratios and residual energy to total energy ratios) and encode/quantize them. These encoded forms may be passed to encoder 217.

In some embodiments, the energy ratio average generator/quantization resolution determination module 211 is configured to encode each energy ratio value using a predetermined number of bits. For example, in the above case where there are N=5 subbands, 3 bits are used to encode each energy ratio value. Thus, the average energy ratio generator/quantization resolution determination module 211 can be configured to apply scalar non-uniform quantization using 3 bits for each subband.

In addition, the energy ratio average generator/quantization resolution determining unit 211 is configured to generate only one weighted average value for each subband, which is transmitted to the encoder for transmission/storage, instead of controlling the transmission/storage of all energy ratio values for all TF blocks.

In some embodiments of the invention, this average value is calculated taking into account the total energy of each time-frequency block and the weighting applied on the basis of subbands having more energy.

In addition, the average energy ratio generator/quantization resolution determining unit 211 is configured to determine the quantization resolution for the direction parameters (in other words, the quantization resolution for the angular height and azimuth values) for all time-frequency blocks in the frame. This bit allocation, for example, can be defined as bits_dir0[0:N-1][0:M-1] and can be passed to the direction parser/index generator 215.

As shown in FIG. 3, the operations of the average energy ratio generator/quantization resolution determination unit 211 can be summarized. The first step is to obtain the ratio values as shown in FIG. 3 at step 301. The subband loop is then started in FIG. 3 at step 303. The subband loop comprises a first act of using a given number of bits (eg, 3) to represent an energy ratio value based on a weighted average of the energy ratio for all values in the time block (where the weighting is determined by the audio energy value), as shown in FIG. . 3 at step 305. Then, the second step is to determine the quantization resolution for the azimuth and angular height for the entire time block of the current subband based on the energy ratio value, as shown in FIG. 3 at step 307.

Alternatively, this can be represented in pseudocode as follows:

1. For each subrange i=1:N

a. Use 3 bits to encode the corresponding energy ratio value

b. Set the quantization resolution for azimuth and angular height for the entire time block of the current subband. The quantization resolution is set by providing a predetermined number of bits given by the energy ratio value, bits_dir0[0:N-1][0:M-1]

2. End of cycle.

The metadata encoder/quantizer 111 may include a direction analyzer/index generator 215. The direction index generator 215 is configured to receive direction parameters 108 (such as azimuth ϕ(k,n) and angular height θ(k,n)) and extract quantization bits and on its basis generating quantized output data. In some embodiments of the invention, the quantization is based on the arrangement of spheres forming a spherical grid arranged in rings on the "surface" sphere, which are determined by a look-up table determined by a given quantization resolution. In other words, the spherical grid uses the idea of covering the sphere with smaller spheres and treating the centers of the smaller spheres as points defining a grid of nearly equidistant directions. Thus, the smaller spheres define cones or solid angles around the central point, which can be indexed according to any suitable indexing algorithm. Although spherical quantization is described here, any suitable quantization, linear or non-linear, may be used.

For example, in some embodiments of the invention, bits for the direction parameters (azimuth and angular height) are allocated in accordance with the bits_direction[] table; if the energy ratio has index i, the number of bits for the direction is bits_direction[i].

The structure of the direction quantizers for different bit resolutions is given by the following variables:

"no_theta" corresponds to the number of angular height values in the "Northern Hemisphere" of the sphere of directions, including the equator. "no_phi" corresponds to the number of azimuth values at each angular height for each quantizer.

For example, for 5 bits, there are 4 angular heights corresponding to [0, 30, 60, 90], and 4-1=3 negative angular heights [-30, -60, -90]. For the first angular height value, 0, there are 12 equidistant azimuths, for angular heights of 30 and -30 there are 7 equidistant azimuths, and so on.

All quantization structures, except for the structure corresponding to 4 bits, have the difference between successive angular height values specified by 90 degrees divided by the number of "no_theta" angular height values. This is an example, and any other suitable allocation may be implemented. For example, in some embodiments of the invention, a 4-bit sphere grid may be implemented, which may not have points below the equator. Similarly, a 3-bit distribution can be distributed over a sphere, or limited to just the equator. In this way, the indices can be thought of as encoding the direction parameters at a fixed rate.

Having determined the direction indices, the direction parser/index generator 215 can then be configured to entropy encode the azimuth and angular height indices. Entropy coding is implemented for one frequency subband at a time, by encoding all temporal subframes for that subband. This means that, for example, the best GR order is determined for 4 values corresponding to the temporal subframes of the current subband. In addition, as discussed herein, when there are multiple ways to encode values for the same subrange, one of the ways is selected as described below. Entropy encoding of the azimuth and angular height indices in some embodiments of the invention can be implemented using the Golomb-Rice encoding method with two possible values of the Golomb-Rice parameter. In some embodiments of the invention, entropy coding may also be implemented using any suitable entropy coding technique (eg, Huffman, arithmetic coding...).

Having fixed-rate and entropy-coded direction indices (the angular height and azimuth indices in this example), the direction analyzer/index generator 215 can be configured to compare, for each of the subbands, the number of bits used in the entropy coding (EC) method. ), with the number of bits used in the fixed rate coding method, and for each coding method selection sub-band that uses fewer bits. Thus, bits_EC is the sum of the bits used in each subband, regardless of whether fixed or variable rate coding is used. For subbands where fixed rate coding is used, the number of bits used for each direction is defined as bits_dir0[i][j], where "i" is the subband index and "j" is the temporal subframe index.

Assume that the bits for each subband after entropy encoding are as follows:

The number of bits used to encode the time block or frame is then compared to the number of available bits. For example, in some embodiments of the invention, a Delta value can be calculated, which is the difference between the number of bits used to encode a temporary block or frame (bits_EC) and the number of available bits.

In some embodiments of the invention, the direction analyzer/index generator 215 is configured to determine whether the delta value (Delta) is negative. In other words, whether the number of bits for encoded direction indices (using fixed-rate and entropy-coded subbands) exceeds the number of available bits.

If the number of bits used is not greater than the number of available bits (or the Delta value is positive or non-negative), then encoder 217 is configured to use coded direction indices (bits_EC) and signals which subframes are entropy coded and which are fixed rate coded. For example, in some embodiments of the invention, the encoder is configured to transmit 1 bit to indicate that an entropy coding method and fixed rate coding are being used, as well as 1 bit per subband, which is then used to indicate whether the subband is fixed rate or entropy encoded. -coded. The coded subbands are then grouped. For example, entropy-coded subbands are grouped, followed by fixed-rate-coded subbands.

This is shown, for example, in FIG. 4a, where the initial operation following step 309 is an operation of determining direction indices (azimuth and angular height) based on the quantization resolution set by bits_dir0[0:N-1][0:M-1], in other words, performing encoding at a fixed speed, as shown in Fig. 4a at step 400.

After generating the indices, the next operation is to entropy-code the direction indices, as shown in FIG. 4a at step 401.

Having generated an entropy-coded and fixed-rate-coded form for all subbands, then for each subband, the option that uses fewer bits is selected, and the bits used for the time block or frame (as bits_EC) are determined, as shown in FIG. 4a at step 403.

Then, the difference between the used bits and the available bits (Delta=bits_EC-bits_available) is determined, as shown in FIG. 4a at step 405.

The next operation may be to determine if the number of bits for the encoded direction indices exceeds the number of available bits (in other words, is the Delta value negative?), as shown in FIG. 4a at step 407.

If the determination results in the answer that the number of bits for the coded direction indices does not exceed the number of available bits (in other words, the Delta value is not negative or positive), then the coded direction indices are used and, in addition, the selections are signaled (in other words, indicators are generated for signaling which subframes are entropy coded and which are fixed rate coded), as shown in FIG. 4a at step 408. In some embodiments, 1 bit per subband is used to indicate that an entropy coding selection method is used, 1 bit is used per subband to indicate which of them are fixed rate coded or entropy coded. , and then grouping the encoded metadata such that first all the entropy-coded subbands are packed into the bitstream, and then the fixed-rate-encoded subbands are packed.

In some embodiments of the invention, where the number of bits for encoded direction indices is greater than the number of available bits (or Delta is negative), then the direction parser/index generator 215 is configured to determine if the number of bits used for encoded direction indices, the number of bits available, by the quantization resolution reduction threshold. The quantization resolution reduction threshold may, in some embodiments, be computed based on the number of fixed rate coded subbands, the number of bits that can be reduced on a per time-frequency tile (or time-frequency block) basis, before the quantization quality is significantly degrade, and the number of subframes in the block. For example, in some embodiments of the invention, the minimum number of bits that can be used is 3 (although any other suitable minimum number of bits can be used). This can be represented as Delta>=FRB*BM*M where FRB=number of fixed rate subbands per subframe, BM=maximum number of bits that can be reduced from each TF tile, and M=number of time blocks or time subframes.

When the determination results in a response that the difference is less than the quantization resolution reduction threshold, then the direction analyzer/index generator 215 is configured to recalculate the number of bits used for fixed rate coding by changing the quantization resolution. In some embodiments of the invention, the quantization resolution is reduced for each TF tile of subbands encoded at a fixed rate, up to the maximum reduction of BM bits (in other words, until the minimum number of bits used is reached) and until the number of bits for a frame is not reduced to the number of available bits. In some embodiments of the invention, the reduction is performed 1 bit per TF at a time so that the quantization resolution in the TF changes uniformly. In addition, in some embodiments, the reduction is applied from lower subbands to higher subbands. The reduction is such that at the end of the quantization resolution reduction, the number of bits used for the time block is bits_EC1, not bits_EC. In other words, the decrease is such that "bits_EC1" must match "bits_available".

After applying the quantization grant for fixed rate subframes, encoder 217 is configured to use encoded direction indices (bits_EC1) and signals which subframes are entropy encoded and which are fixed rate encoded. For example, in some embodiments of the invention, the encoder is configured to transmit 1 bit to indicate that the entropy coding method and fixed rate coding are being used, as well as 1 bit for each subband, which is then used to indicate whether the subband is fixed rate encoded or entropy coded. The coded subbands are then grouped. For example, entropy-coded subbands are grouped, followed by fixed-rate-coded subbands.

When the determination results in a response that the difference is greater than or equal to the quantization resolution reduction threshold, then the direction analyzer/index generator 215 is configured to reduce the number of allocated bits for quantization bits_dir1[0:N-1][0:M-1] so that the sum of the allocated bits was equal to the number of available bits remaining after encoding the energy ratios.

In addition, direction analyzer/index generator 215 can be configured to start subband encoding using the reduced number of available bits after energy ratio encoding. This differs from the quantization resolution reduction described above in that both fixed-rate and variable-rate encoded forms (entropy-coded forms) are encoded again.

The reduced rate coded direction indices and the signaled usage of the fixed rate coded subbands can then be encoded at encoder 217. In other words, a bit can be used to signal whether the subband has been coded using an entropy coding method or a fixed rate coding method, and then the bits for the coded subbands are transmitted.

This is shown, for example, in Fig. 4b, where step 407 is followed by an operation of determining whether the difference exceeds the number of available bits by a quantization resolution reduction threshold, as shown in FIG. 4b at step 409.

If the difference is less than the quantization resolution reduction threshold, then the method is configured to recalculate the number of bits for fixed-rate subband coding by changing the quantization resolution for the fixed-rate-coded subbands (in other words, without changing the entropy-coded subbands), as shown in fig. 4b at 410.

After recalculating the number of bits for fixed rate subband coding, bits using encoded direction indices (with fixed rate subframes with changed quantization resolution) are output and, in addition, selections are signaled (in other words, generated indicators to signal which subframes are entropy coded and which are fixed rate coded), as shown in FIG. 4b at step 412. In some embodiments, use 1 bit to signal that an entropy coding selection method is used, use 1 bit for each subband to indicate which are fixed rate or entropy coded, and then group encoded metadata such that first all entropy-coded subbands are packed into the bitstream, and then the resolution-changed, fixed-rate encoded subbands are packed.

In some embodiments of the invention, reduced bit rate coding may be implemented by running a loop for each subband up to the penultimate subband N-1. This loop determines the allowed number of bits for the current subband bits_allowed=sum(bits_dir1[i][0:M-1]). Then, having determined the allowed number of bits for the current subband, the direction parser/index generator 215 can be configured to encode the indices using fixed rate coding with a reduced bit allocation bits_fixed=bits_allowed.

The direction analyzer/index generator 215 can then be configured to either select fixed rate encoding or use entropy encoding based on a method that uses fewer bits, i.e. chooses the smallest of bits_fixed or bits_ec. In addition, the direction parser/index generator 215 can be configured to use one bit to indicate which of the two encoding methods has been selected. Therefore, the number of bits used to encode a subband is nb=min(bits_fixed, bits_ec)+1.

The direction analyzer/index generator 215 can then be configured to determine if there are available bits in relation to the allowed bits, in other words, diff=allow_bits-nb>0. Where there is a difference between the number of bits available and the number of bits used in a subband, the difference can be distributed to later subbands, e.g. by updating bits_dir1[i+1:N-1][0_M-1], otherwise the parser direction/index generator 215 may be configured to subtract a bit from the next subband allocation bits_dir1[i+1][0].

For the final subband N, the direction parser/index generator 215 can be configured to encode direction indices using a fixed rate encoding method and using bits_dir1[N-1][0:M-1].

As shown in FIG. 4c, these reduced bit rate operations (in other words, step 413 in FIG. 4b) can be shown as an example of a flowchart. The first step is to run a loop for subbands 1 to the penultimate (N-1) subband as shown in FIG. 4c at step 421.

In the loop for the current subband, the number of allowed bits for encoding is determined as shown in FIG. 4c at step 423.

Then, a fixed rate encoding method is used to encode the indices using the reduced number of bits, as shown in FIG. 4c at step 425.

Then, either fixed rate coding or entropy coding is selected based on which method uses fewer bits, and the selection can furthermore be indicated by one bit, as shown in FIG. 4c at step 427.

Determining whether there are any remaining bits available based on the difference between the number of allowed bits and the number of bits used by the selected coding, and reallocating the remaining bits to later subband assignments is shown in FIG. 4c with step 429.

The cycle is then completed and it may repeat for the next subband, as shown in FIG. 4c at step 431.

Finally, the last subband is encoded using a fixed rate method using the remaining bit allocation as shown in FIG. 4c at step 433.

Thus, the method can be summarized as follows:

1. For each subrange i=1:N

a. encode the value of the energy ratio

b. determine the direction indices based on the quantization resolution (for the entire time block of the current subband) based on the encoded value of the energy ratio

3. End of cycle

4. Entropy coding of direction indices

5. Selecting for each subband whether fixed rate (index) coding or entropy coding uses fewer bits, determining block bits used

6. If there are more block bits used than available bits

A. If the difference between the used block bits and the available bits is less than the quantization resolution change threshold

i. Recalculation of bits used by changing the quantization resolution of fixed rate coded subbands.

ii. Generating output based on the signaled method, the signaled choices, and then the grouped subbands depending on whether they were encoded using a fixed rate (changed quantization resolution) or an entropy method.

b. Otherwise

i. Decreasing the allocated number of bits, bits_dir1[0:N-1][0:M-1], so that the sum of the allocated bits is equal to the number of available bits left after encoding the energy ratios.

ii. Recoding for each subband i=1:N-1

1. Calculate the allowed bits for the current subrange: bits_allowed=sum(bits_dir1[i][0:M-1])

2. Encoding direction parameter indices by using fixed rate coding with reduced allocated bits, bits_fixed=bits_allowed, or using entropy coding, bits_ec; choosing an encoding that uses fewer bits and using one bit to indicate the method: nb=min(bits_fixed, bits_ec)+1;

3. If there are available bits relative to allowed bits: (if diff=allowed_bits-nb>0)

A. Redistributing the difference, diff, to the next subranges by updating bits_dir1[i+1:N-1][0 M-1]

4. Otherwise

A. Subtract one bit from bits_dir1[i+1][0]

5. End of the conditional statement

iii. End of cycle

iv. Coding of direction parameter indexes for the last subband with fixed rate approach using bits_dir1[N-1][0:M-1].

c. End of conditional statement

7. Otherwise

8. Generating output based on the signaled method, the signaled choices, and then the grouped subbands depending on whether they were encoded using a fixed rate coding method or an entropy coding method.

9. End

In some implementations, the optimization of the entropy encoding of the angular height and azimuth values may be performed separately and is described in more detail below with reference to FIG. 5 and 6.

For example, in FIG. 5 shows an example in which some embodiments of the invention apply a series of index checks and optimizations to attempt to reduce the number of bits needed to entropy encode direction indexes.

In some embodiments of the invention, the determination of direction indices begins as shown in FIG. 5 from step 501. In this example, the bits needed to entropy encode the index definition shown is the angular height index definition. However, as described below, a similar approach can be applied to the determination of the azimuth index.

In some embodiments, the display is generated such that an elevation (or azimuth) value of 0 has an index of 0, and incremental index values are assigned to incremental positive and negative elevation (azimuth) values, as shown in FIG. 5 at step 503.

After mapping is generated, the mapping is applied to audio sources (eg, in the form of codeword output generation based on a look-up table) as shown in FIG. 5 at step 505.

After the indices are generated, in some embodiments of the invention, a check is made to determine if all the indices are located within the same hemisphere, as shown in FIG. 5 at step 507.

If all indices are located within the same hemisphere, then the index values can be divided by two (rounded up) and an indicator can be generated indicating which hemisphere all indices are located within, and then these values can be entropy encoded, as shown in fig. 5 at step 509.

If all indices are not located within the same hemisphere, then de-mean entropy coding can be applied to the indices. De-averaged entropy coding may be configured to first remove the average index value for the subframes to be coded, then convert the indices to positive ones, and then encode them with a suitable entropy coding, such as Golomb-Rice coding, as shown in FIG. 5 at step 510.

After applying entropy coding, in some embodiments of the invention, a check can be applied to determine if all time subframes have the same value or angular height (azimuth) index, as shown in FIG. 5 at step 511.

If all temporal subframes have the same azimuth (azimuth) value or index, then an indicator is generated indicating a plurality of azimuth (azimuth) values or indices, as shown in FIG. 5 at step 513, otherwise the method proceeds directly to step 517.

The next operation is to provide the number of bits needed for the entropy-coded indices and any indicator bits, as shown in FIG. 5 at step 517.

For example, with respect to angular height values, the angular height index may be determined from the codebook in the region [-90; 90], which is formed such that an angular height value of 0 returns a zero-index codeword and alternatively assigns increasing indices to positive and negative codewords spaced from the zero angular height value.

For example, in some embodiments of the invention, a codebook with codewords {-90, -60, -30, 0, 30, 60, 90} is implemented, which creates indexes {6, 4, 2, 0, 1, 3, 5}. This indexing gives indexes with a smaller value for directions that are more likely in a general sense (in practical examples, directions are near the equator). Another observation is that if the sound sources are farther from the equator, which corresponds to higher index values, they tend to be either above or below the equator. In some embodiments of the invention, the encoder may be configured to check if all sound sources are above (or all sound sources below) the equator, and if so for all temporal subframes for the subband, then the indices are divided by 2 to create indices with a lower value , which can be encoded more efficiently.

In some embodiments of the invention, the estimation of the number of bits for the angular height indices can be implemented in the C language as follows:

The special case of the same angular height values for all time subframes is also checked and signaled.

The mean_removed_GR() function in the above example is configured to first remove the average index value for the encoded subframes, then convert the indices to positive ones, and then encode them using Golomb-Rice coding.

This can be implemented, for example, in C language as follows:

The odd_even_mean_removed_GR() function is configured to first check if all indices are odd or all even, signal their occurrence and indicate the type (odd or even), and then encode the indices divided by 2.

In some embodiments of the invention, a number of entropy encoding optimizations are performed and then the smallest value is selected. This, for example, can be illustrated in relation to the coding of azimuth values, as shown in FIG. 6. In some embodiments of the invention, the determination of direction indices begins as shown in FIG. 6, at step 601.

In some embodiments, the display is generated such that an azimuth value of 0 has an index of 0, and increasing index values are assigned to increasing positive and negative azimuth values, as shown in FIG. 6 at step 503.

After mapping is generated, the mapping is applied to audio sources (eg, in the form of codeword output generation based on a look-up table) as shown in FIG. 6 at step 605.

In this example, the azimuth index can be determined from an additional codebook. In this example, a zero azimuth value corresponds to a reference direction, which can be forward, positive values to the left, and negative values to the right. In this example, the azimuth value index is assigned such that the values (-150, -120, -90, -60, -30, 0, 30, 60, 90, 120, 150, 180) are assigned the following indices (10, 8, 6, 4, 2, 0, 1, 3, 5, 7, 9, 11). In some embodiments, the odd/even approach may be checked for azimuth (corresponding to left/right positioning).

In this example, higher index values are assigned to values from the back side or back of the "capturing environment".

The encoding of subframe azimuth indices in some embodiments of the invention may be performed based on the following:

1. Determining the number of azimuth indices to be encoded for the current subband (as shown in FIG. 6 at step 607).

2. Finding the maximum number of symbols for the tiles of the current subband (as shown in FIG. 6 at step 609).

3. If there are more symbols than the threshold (as shown in FIG. 6, step 611)

A. Encoding (as shown in FIG. 6 at step 613) the azimuth values by checking the encoding of the values given by the complementary values: no_symb-index_azimuth.

i. Estimating the number of bits when encoding indices as if they were ahead. Use of Golomb-Rice coding, order selective, with removed mean. The GR order can be 2 or 3. The GR order can also be set to different values, depending on the default range for the number of characters.

ii. Estimation of the number of bits when encoding complementary indices using GR encoding, order-selective, de-averaged.

iii. Using an encoding method that uses fewer bits, and using a bit to indicate which method is used.

4. Otherwise

A. Azimuth index coding by using GR coding with order 1 or 2 removed mean (as shown in FIG. 6 at step 615).

5. End

6. Checking if GR encoding with the removed minimum gives the best result, and whether it is better to use it (as shown in Fig. 6 at step 617). In C language, the code looks like this:

In FIG. 7 shows an example of a metadata extractor 137 suitable for decoding coded metadata encoded by an encoder as shown in FIG. 2.

The metadata extraction module 137 in some embodiments of the invention includes a demultiplexer 701 configured to receive encoded signals and output encoded energy ratio values to energy ratio decoder 703, as well as output signaling bits to entropy encoding mode detector 705 and subband detector 707, and encoded indices to the decoder 709 indexes.

The metadata extractor 137 may further comprise an energy ratio decoder 703 configured to receive and decode encoded energy ratios to generate decoded energy ratios. The decoded energy relationships 704 can be output. In addition, the energy ratio decoder 703 can generate an energy ratio quantization resolution value 708 based on the encoded energy ratio value and transmit it to the index decoder and the direction index value to direction value (AZ/EL) converter 711 .

The metadata extractor 137 may further comprise an Entropy Coding (EC) mode detector 705 . The EC mode detector may read the first bit in the block, which indicates whether the entire block has been encoded in fixed rate mode (in other words, whether the block contains encoded index values and thus no need for entropy decoding) or whether the block has fixed-rate hybrid entropy coding is implemented.

Thus, the entropy encoding mode detector 705 may be configured to control the index decoder 709 based on the first bit (mode indicator).

The metadata extractor 137 may further comprise a subband detector 707 . Subband detector 707 may read the following bits (eg, if there are 5 subbands, there are 5 bits) in a block that indicates to the block which subbands were encoded according to the fixed rate coding method and which subbands were encoded according to the entropy coding method.

Thus, the subband detector 707 may be configured to control the index decoder 709 based on the read bits (subband indicators).

Module 137 extraction of metadata, in addition, may contain the decoder 709 index. An index decoder 709 having received encoded metadata values for the subbands may be controlled by a subband detector 707 and an entropy mode detector 705.

Thus, for example, index decoder 709 may be configured to decode fixed rate encoded metadata values when the mode indicator indicates that hybrid mode is disabled.

In addition, index decoder 709 may be configured to decode the entropy-coded subbands based on the subband indicators. After reading and decoding the entropy values, the difference between the available bits and the read bits (indicator bits and entropy-coded direction index bits) is determined. The index decoder 709 is also configured to determine whether the difference is less than the number of bits required for fixed rate encoding of the remaining encoded subbands based on the energy ratio quantization resolution value 708 (bits_available-bits_read)<sum(bits_dir0[i][ j]), where i=fixed rate coded subband index and j=0:M-1.

If the difference is less than the number of bits assigned based on the energy ratio quantization resolution value 708, then the index decoder is configured to determine whether coding has been implemented using a fixed rate subband quantization resolution change, and decoding is performed on the fixed rate subbands by based on reduced quantization resolutions defined in the same way as implemented in the encoder. If the difference is correct, then the original resolution is used to decode the subbands at a fixed rate.

The decoded direction parameters 712 may then be output. Thus, in some embodiments of the invention, there may be two levels of reduction.

A more precise reduction level (when the difference is small enough), which is signaled as follows:

The initial number of bits for each time-frequency block is determined by the energy quantization factor. First, it is signaled whether the EU subband or fixed rate coding is used. The subbands that are EC encoded were written first, so when they are read, it is known how many bits they used. Also known is the number of available bits and a predetermined number of bits for subbands encoded at a fixed rate. If the predetermined number of bits + entropy-coded subband bits fit within the available bits, all is well, there is no reduction; otherwise there is a slight decrease.

A coarser or "harder" reduction in which one bit at the beginning is transmitted to indicate to the decoder whether the bit allocation is reduced to the number of bits available or not (corresponding to block 411).

In FIG. 8, for example, shows the operation of the metadata extractor as shown in FIG. 7 in block diagram form.

Thus, the method includes receiving encoded data as shown in FIG. 8 at step 801.

The encoded data is demultiplexed as shown in FIG. 8, at step 803.

Then, the EC mode signaling bit is read to determine whether a hybrid entropy coding method has been applied, and to determine whether EC fine mode coding (or EC coarse mode) has been applied, as shown in FIG. 8, at step 805.

Where the EC mode signaling bit indicates that a coarse rate reduction has been applied, decoding is performed based on rate reduction based decoding only (coarse rate reduction energy ratio quantization enabled in some embodiments), as shown in FIG. 8 at step 806.

If the EC mode signaling bit indicates that hybrid entropy coding and fixed rate coding were used and that exact rate reduction was required (changing only the quantization resolution) or no rate reduction was required, then the next operation is to read the subband signaling bits to determine whether which subbands were entropy encoded and which subbands were fixed rate encoded as shown in FIG. 8 at step 807.

The bits of the grouped entropy-coded subband are read and decoded to generate direction indices that can be converted to directions based on the original energy ratio quantization resolution, as shown in FIG. 8 at step 809.

The next operation is to determine whether the difference between the bits available for the block and the bits read (signaling bits and entropy-coded bits) is less than the number of bits needed to encode the remaining bits at a fixed rate according to the original energy ratio quantization resolution. , as shown in FIG. 8 at step 811.

If the difference is less than the required number of bits, then decoding can be performed in "fine" rate reduction coding based on the modified quantization resolution method as shown in FIG. 8 at step 813.

If the difference is not less than (or equal to) the required number of bits, then decoding can be performed based on coding based on the original quantization resolution method, as shown in FIG. 8 at step 812.

In FIG. 9 shows an example of an electronic device that can be used as an analysis or synthesis device. The device may be any suitable electronic device or equipment. For example, in some embodiments, device 1400 is a mobile device, user equipment, tablet computer, computer, audio device, and so on.

In some embodiments, the device 1400 includes at least one processor or central processing unit 1407. The processor 1407 may be configured to execute various program codes, such as the methods described herein.

In some embodiments, device 1400 includes memory 1411. In some embodiments, at least one processor 1407 is coupled to memory 1411. Memory 1411 may be any suitable storage medium. In some embodiments, memory 1411 includes a program code section for storing program codes implemented by processor 1407. In addition, in some embodiments, memory 1411 may also include a stored data section for storing data, such as data that has been processed or is about to be processed. in accordance with embodiments of the invention as described here. The implemented program code stored in the program code section and the data stored in the stored data section can be retrieved by the processor 1407 whenever necessary via a memory-to-processor connection.

In some embodiments, the device 1400 includes a user interface 1405. In some embodiments, the user interface 1405 may be coupled to a processor 1407. In some embodiments, the processor 1407 may control the operation of the user interface 1405 and receive input from the user interface 1405. In In some embodiments, the user interface 1405 may allow a user to enter commands into the device 1400, such as using a keyboard. In some embodiments, the user interface 1405 may allow a user to receive information from the device 1400. For example, the user interface 1405 may include a display configured to display information from the device 1400 to the user. User interface 1405 may, in some embodiments, comprise a touch screen or touch interface capable of both allowing information to be entered into device 1400 and displaying information to a user of device 1400. In some embodiments, user interface 1405 may be a user interface for communicating with positioning module as described here.

In some embodiments of the invention, the device 1400 contains an input/output port 1409. Port 1409 input/output in some embodiments of the invention contains a transceiver. The transceiver in such embodiments may be coupled to the processor 1407 and configured to communicate with another device or electronic devices, such as through a wireless communication network. The transceiver or any suitable transceiver or transmitter and/or receiving means, in some embodiments of the invention, may be configured to communicate with other electronic devices or equipment via a wired connection or wired connection.

The transceiver may communicate with other devices over any suitable known communication protocol. For example, in some embodiments, the transceiver may use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as, for example, IEEE 802.X, a suitable short-range radio frequency communication, such as Bluetooth, or infrared data communication channel (IRDA, infrared data communication pathway).

Port 1409 input/output of the transceiver can be configured to receive signals and, in some embodiments of the invention, to determine the parameters, as described here, using the processor 1407, executing the appropriate code.

In general, various embodiments of the invention may be implemented in special purpose hardware or circuits, software, logic circuits, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, although the invention is not limited to this. While various aspects of the invention may be illustrated and described in the form of block diagrams, block diagrams, or using some other graphical representation, it is understood that the blocks, devices, systems, techniques, or methods described herein may be implemented, as non-limiting examples, in hardware, software, firmware, special purpose circuits or logic circuits, general purpose hardware or controller or other computing devices, or combinations thereof.

Embodiments of the present invention may be implemented by computer software executable by a mobile device data processor, such as a processor, or by hardware, or a combination of software and hardware. Further in this regard, it should be noted that any logic flow blocks, as shown in the drawings, may represent program steps, or interrelated logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks built into a processor, on magnetic media such as a hard disk drive or floppy disks, and on optical media such as, for example, DVDs and CDs.

The memory may be of any type suitable for the local technical environment and may be implemented using any suitable storage technology, for example, through semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. . Data processors may be of any type suitable for the local technical environment and may include one or more general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs, digital signal processors), application specific integrated circuits (ASICs). ), gate-level circuits, and processors based on multi-core processor architectures, as non-limiting examples.

Embodiments of the invention may be implemented in various components such as integrated circuit modules. The design of integrated circuits in general is a highly automated process. Sophisticated and powerful software tools are available to convert a logic level circuit into a semiconductor circuit layout ready to be etched and formed on a semiconductor substrate.

Programs such as those provided by Synopsys, Inc. of Mountain View, Calif., and Cadence Design of San Jose, Calif., automatically route conductors and place components on a semiconductor chip using commonly accepted design rules as well as libraries of pre-stored design modules. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (eg Opus, GDSII, etc.) can be submitted to a semiconductor manufacturing facility or "factory" for fabrication.

Above, by means of illustrative and non-limiting examples, a complete description of an illustrative embodiment of the present invention has been presented. However, various modifications and adaptations may be apparent to those skilled in the relevant arts from the foregoing description when read in conjunction with the accompanying drawings and claims. However, all such modifications of the principles of the present invention are within the scope of the invention as defined by the appended claims.

Claims (51)

1. A device for encoding parameters of spatial sound, containing means configured for: generating spatial audio direction metadata parameters for the time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; outputting or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined number of bits and the difference between the predetermined the number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined threshold, while the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for a time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits. 2. The apparatus of claim 1, wherein the means configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution is configured to: determining a first quantization resolution to map between spatial audio direction metadata parameter values and an index value; generating indices associated with spatial audio direction metadata parameters based on said mapping using a first quantization resolution; encoding, optionally, the indexes using fixed rate coding or entropy coding based on whether fixed rate coding or entropy coding uses fewer bits. 3. The apparatus of claim 2, wherein means configured to determine a first quantization resolution for mapping between spatial audio direction metadata parameter values and an index value is configured to determine a first quantization resolution for mapping between spatial audio direction metadata parameter values and a value an index based on the energy ratio value associated with the spatial audio direction metadata parameter. 4. The device according to any one of paragraphs. 2, 3, wherein means configured to generate encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the difference between the specified number of bits and the number of bits used for the encoded spatial audio direction parameters for the frequency-time block time block based on the first quantization resolution, is within the specified threshold, configured to: determining a second quantization resolution to map between spatial audio direction metadata parameter values and an index value; generating indices associated with the spatial audio direction metadata parameters based on a mapping using the second quantization resolution for the spatial audio direction metadata parameters that have been encoded at a fixed rate using the first quantization resolution. 5. The apparatus of claim 4, wherein the means is also configured to output or store: entropy-coded indices associated with the spatial audio direction metadata parameters based on a mapping using a first quantization resolution for the spatial audio direction metadata parameters; And fixed rate encoded indices associated with the spatial audio direction metadata parameters based on a mapping using a second quantization resolution for the spatial audio direction metadata parameters. 6. The apparatus of claim 5, wherein the means is also configured to order the encoded indices such that the entropy encoded indices precede the fixed rate encoded indices. 7. The device according to any one of paragraphs. 1-6, wherein the means is also configured to generate an indicator when the first or second quantization resolution is used. 8. The device according to any one of paragraphs. 1-7, wherein means configured to generate encoded spatial audio direction metadata parameters for a time-frequency block based on a third quantization resolution is configured to: determining a third quantization resolution to map between the spatial audio direction metadata parameter values and the index value based on that the number of bits used for fixed rate coding using the third quantization resolution is always equal to or less than the specified number of bits; generating indices associated with spatial audio direction metadata parameters based on the mapping using the third quantization resolution; And encoding, optionally, the indexes using fixed rate coding or entropy coding based on whether fixed rate coding or entropy coding uses fewer bits. 9. The apparatus of claim 8, wherein the means is also configured to output selectively encoded indices using fixed rate coding or entropy coding based on whether the fixed rate coding or entropy coding uses fewer bits. 10. The device according to any one of paragraphs. 1-9, wherein the means is also configured to generate an indicator when the third quantization resolution is determined. 11. An apparatus for decoding spatial audio parameters, comprising means configured to: receiving encoded spatial audio direction metadata parameters for the time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded spatial audio direction metadata parameters for a time-frequency block based on a quantization resolution that is always equal to or less than a specified number of bits, when the indicator identifies that the encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than the specified number of bits; And, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a specified number of bits, the means is configured to: decoding a first part of encoded spatial audio direction metadata parameters for the time-frequency block based on a different quantization resolution, the first part containing entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution; decoding, when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded parameters of the spatial audio direction metadata for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization. 12. The apparatus of claim 11, wherein the means is also configured to determine said different quantization resolution to map between the spatial audio direction metadata parameter values and the index value. 13. The apparatus of claim 12, wherein the means configured to determine said different quantization resolution for mapping between spatial audio direction metadata parameter values and the index value is configured to determine said different quantization resolution based on an energy ratio value associated with the metadata parameter direction of the spatial audio signal. 14. The device according to any one of paragraphs. 11-13, wherein the means is also configured to determine a reduced bit quantization resolution to map between spatial audio direction metadata parameter values and an index value. 15. The device according to any one of paragraphs. 11-14, wherein the means is configured to generate a mapping from indices associated with spatial audio direction metadata parameters to an angular height value and/or an azimuth value based on the quantization resolution. 16. A method for encoding parameters of spatial sound, including: generating spatial audio direction metadata parameters for the time-frequency block; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution; comparing the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution with a given number of bits; deriving or storing encoded spatial audio direction metadata parameters for the time-frequency block based on the first quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the second quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is less than a predetermined number of bits and is within a predetermined threshold; generating encoded spatial audio direction metadata parameters for the time-frequency block based on the third quantization resolution, when the number of bits used for encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than the specified number of bits and the difference between the specified the number of bits and the number of bits used for the encoded spatial audio direction parameters for the time-frequency block based on the first quantization resolution is greater than a predetermined threshold, wherein the third quantization resolution is determined such that the number of bits used for the encoded spatial audio direction parameters for a time-frequency block based on the third quantization resolution, always equal to or less than the specified number of bits. 17. A method for decoding spatial audio parameters, including: receiving encoded spatial audio direction metadata parameters for the time-frequency block; receiving an indicator configured to identify whether encoded spatial audio direction metadata parameters have been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits; decoding the encoded parameters of the spatial audio direction metadata for the time-frequency block based on the quantization resolution, which is always equal to or less than the specified number of bits, when the indicator identifies that the encoded parameters of the spatial audio direction metadata were encoded based on the quantization resolution, which is always equal to or less than the specified number of bits; And, when the indicator identifies that the encoded spatial audio direction metadata parameters have not been encoded based on a quantization resolution that is always equal to or less than a predetermined number of bits, the method includes: decoding a first part of encoded spatial audio direction metadata parameters for the time-frequency block based on a different quantization resolution, the first part containing entropy-encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution; decoding when the difference between the given number of bits and the number of bits used to encode the first part is less than the number of bits needed to encode the second part of the encoded spatial audio direction metadata parameters for the time-frequency block based on said different quantization resolution, the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on reduced bit quantization resolution, otherwise decoding the second part containing fixed-rate encoded spatial audio direction metadata parameters for the time-frequency block based on said other resolution quantization.

Images