RU2810027C2 - Audio encoder and audio decoder - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: processing audio data. The technical result is achieved by representing each parameter as a vector through an index value that can take N values; calculating one or more symbols, the calculation including: calculating the difference between the index value of the second element and the index value of its previous element in the vector: and applying a modulo N operation to the difference; associating at least each of one second element with a corresponding symbol out of one or more symbols, and encoding at least each of one second element by entropy encoding the symbol associated with at least one second element based on the probability table , containing the probabilities of the symbols.

EFFECT: increase of efficiency and quality of the encoded audio signal.

7 cl, 15 dwg

Description

Cross reference to related applications

This application claims priority to the filing date of U.S. Provisional Patent Application No. 61/827264, filed May 24, 2013, incorporated herein by reference.

Technical field

The disclosure herein generally relates to audio encoding. In particular, it relates to the encoding and decoding of a parameter vector in an audio encoding system. The disclosure further relates to a method and apparatus for recovering an audio object in an audio decoding system.

Prior Art

Traditional audio systems use a channel approach. Each channel, for example, can represent the content of one speaker or one array of speakers. Possible encoding schemes for such systems include discrete multi-channel encoding or parametric encoding, such as the MPEG surround sound standard.

Subsequently, a new approach was developed. This approach is object-oriented. In a system using an object-oriented approach, a three-dimensional audio scene is represented by audio objects with their associated positional metadata. These audio objects move in the 3D audio scene as the audio signal plays. The system may further include so-called underlay channels, which can be described as stationary audio objects that are directly converted to speaker positions, for example, a traditional audio system, as described above.

A problem that may arise in an object-oriented audio system is how to efficiently encode and decode an audio signal and maintain the quality of the encoded signal. A possible encoding scheme includes, on the encoder side, creating a downmix signal containing a number of channels from audio objects and underlay channels and auxiliary information that enables reconstruction of the audio objects and underlay channels at the decoder side.

MPEG Spatial Audio Object Coding (MPEG SAOC) describes a system for parametric coding of audio objects. The system sends auxiliary information, such as an upmix matrix, that describes object properties through parameters such as level difference and object cross-correlation. These parameters are then used to control the reconstruction of audio objects on the decoder side. This process can be mathematically complex and often must be based on assumptions about properties of audio objects that are not explicitly described by parameters. The method presented in MPEG SAOC may reduce the required bit rate for an object-oriented audio system, but further improvement may be necessary to further improve efficiency and quality as described above.

Brief description of drawings

Exemplary embodiments are described below with reference to the accompanying drawings, in which:

Fig. 1 is a general block diagram of an audio encoding system in accordance with an exemplary embodiment;

Fig. 2 is a generalized block diagram of the exemplary upmix matrix encoder shown in FIG. 1;

Fig. 3 shows an approximate probability distribution for the first element in a vector of parameters corresponding to an element in the upmix matrix determined by the audio coding system of FIG. 1;

Fig. 4 shows an exemplary probability distribution for at least one differentially modulo encoded second element in a vector of parameters corresponding to an element in the upmix matrix determined by the audio coding system of FIG. 1;

Fig. 5 is a general block diagram of an audio decoding system in accordance with an exemplary embodiment;

Fig. 6 is a generalized block diagram of the upmix matrix decoder shown in FIG. 5;

Fig. 7 describes an encoding method for second elements in a parameter vector corresponding to an element in the upmix matrix determined by the audio encoding system of FIG. 1;

Fig. 8 describes an encoding method for a first element in a vector of parameters corresponding to an element in the upmix matrix determined by the audio encoding system of FIG. 1;

Fig. 9 describes parts of the encoding method of FIG. 7 for the second elements in the example parameter vector;

Fig. 10 describes parts of the encoding method of FIG. 8 for the first element in the example parameter vector;

Fig. 11 is a generalized block diagram of the second exemplary upmix matrix encoder shown in FIG. 1;

Fig. 12 is a general block diagram of an audio decoding system according to an exemplary embodiment;

Fig. 13 describes a coding method for sparse row coding of an upmix matrix;

Fig. 14 describes parts of the encoding method of FIG. 10 for an example upmix matrix row;

Fig. 15 describes parts of the encoding method of FIG. 10 for an example upmix matrix row;

All data is schematic and generally shows only those parts that are required to clarify the disclosure, while other parts may be omitted or simply assumed. Unless otherwise indicated, like reference numbers indicate like parts in the different drawings.

Detailed Description of the Invention

In view of the foregoing, the goal is to provide encoders and decoders and associated methods that provide increased efficiency and quality of the encoded audio signal.

I. Review: Encoder

According to a first aspect, exemplary embodiments provide encoding methods, encoders, and computer program products for encoding. The proposed methods, encoders and computer program products may generally have identical features and advantages.

According to exemplary embodiments, a method is provided for encoding a vector of parameters in an audio coding system, wherein each parameter corresponds to an aperiodic quantity, the vector has a first element and at least one second element, the method comprising: representing each parameter in the vector by an index value , which can take N values; associating each of the at least one second element with a symbol, the symbol being calculated by: calculating a difference between the index value of the second element and the index value of its previous element in the vector; applying the operation modulo N to the difference. The method further comprises the step of encoding each of the at least one second element by entropy encoding a symbol associated with the at least one second element based on a probability table containing probabilities of the symbols.

The advantage of this method is that the number of possible symbols is reduced by approximately a factor of two compared to traditional delta coding strategies in which modulo N is not applied to the delta. Consequently, the size of the probability table is reduced by approximately a factor of two. As a result, less storage is required to store the probability table and, since the probability table is often stored in expensive storage in the encoder, the encoder can thereby become less expensive. In addition, the speed of searching for a symbol in the probability table can be increased. An additional advantage is that coding efficiency can be increased since all symbols in the probability table represent possibilities that must be associated with a particular second element. It can be compared to traditional difference coding strategies, in which only approximately half of the symbols in the probability table represent possibilities for association with a particular second element.

According to embodiments, the method further comprises associating a first element in the vector with a symbol, wherein the symbol is calculated by: shifting an index value representing the first element in the vector by an offset value; applying a modulo N operation to the index value being shifted. The method further comprises the step of encoding the first element by entropy encoding a symbol associated with the first element using an identical probability table that is used to encode the at least one second element.

This embodiment takes advantage of the fact that the probability distribution of the index value of the first element and the probability distribution of the symbols of at least one second element are similar, although shifted relative to each other by an offset value. As a consequence, an identical probability table can be used for the first element in the vector instead of a dedicated probability table. This may result in lower storage requirements and a less expensive encoder as discussed above.

In an embodiment, the offset value is equal to the difference between the most likely index value for the first element and the most likely symbol for at least one second element in the probability table. This means that the peaks of the probability distributions overlap. Consequently, virtually identical encoding efficiency is maintained for the first element compared to if a dedicated probability table for the first element was used.

In embodiments, the first element and at least one second element of the parameter vector correspond to different frequency bands used in the audio coding system in a particular time frame. This means that data corresponding to multiple frequency bands can be encoded in an identical operation. For example, the parameter vector may correspond to an upmix or restore coefficient that varies across multiple frequency bands.

According to an embodiment, the first element and at least one second element of the parameter vector correspond to different time frames used in the audio coding system in a particular frequency band. This means that data corresponding to multiple time frames can be encoded in an identical operation. For example, the parameter vector may correspond to an upmix or restore factor that varies across multiple time frames.

In embodiments, the probability table is converted to a Huffman encoding table, wherein a symbol associated with an element in the vector is used as an index of the encoding table, and wherein the encoding step comprises encoding each of the at least one second element by representing the second element using a codeword in a codebook, which is indexed by the codebook index associated with the second element. By using a symbol as a codebook index, the search speed of a codeword to represent an element can be increased.

In embodiments, the encoding step comprises encoding a first element in the vector using an identical Huffman codebook that is used to encode at least one second element by representing the first element with a codeword in a Huffman codebook that is indexed by the codebook index associated with the first element. Therefore, only one Huffman encoding table needs to be stored in the encoder memory, which can result in a less expensive encoder as above.

According to a further embodiment, the parameter vector corresponds to an element in the upmix matrix determined by the audio coding system. This may reduce the required bit rate in an audio encoding/decoding system since the upmix matrix can be efficiently encoded.

According to exemplary embodiments, a computer-readable medium is provided comprising machine code instructions adapted to perform any method of the first aspect when executed on a device having processing capabilities.

According to exemplary embodiments, an encoder is provided for encoding a vector of parameters in an audio encoding system, each parameter corresponding to an aperiodic quantity, the vector having a first element and at least one second element, the encoder comprising: a receiving component adapted to receive the vector; an indexing component adapted to represent each parameter in the vector by an index value that can take N values; an association component adapted to associate each at least one second element with a symbol, the symbol being calculated by: calculating the difference between the index value of the second element and the index value of its previous element in the vector; applying the operation modulo N to the difference. The encoder further includes an encoding component for encoding each of the at least one second element by entropy encoding a symbol associated with the at least one second element based on a probability table containing probabilities of the symbols.

II. Review: Decoder

According to a second aspect, exemplary embodiments provide decoding methods, decoders, and computer program products for decoding. The proposed methods, decoders, and computer program products may generally have identical features and advantages.

The advantages regarding features and arrangements as presented in the encoder overview above may generally be applicable to corresponding features and arrangements for a decoder.

According to exemplary embodiments, a method is provided for decoding a vector of entropy coded symbols in an audio decoding system into a vector of parameters associated with an aperiodic quantity, the vector of entropy coded symbols comprises a first entropy coded symbol and at least one second entropy coded symbol, and the parameter vector comprises a first element and at least one second element, the method comprising: representing each entropy encoded symbol in the vector of entropy encoded symbols by a symbol that can take N integer values through the use of a probability table; associating the first entropy encoded symbol with an index value; associating each of the at least one second entropy coded symbol with an index value, wherein the index value of the at least one second entropy coded symbol is calculated by: calculating the sum of the index value associated with the entropy coded symbol preceding the second entropy coded symbol in a vector of entropy-coded symbols, and a symbol representing a second entropy-coded symbol; applying an operation modulo N to a sum. The method further comprises the step of representing at least one second element of the parameter vector by a parameter value corresponding to an index value associated with the at least one second entropy-coded symbol.

According to exemplary embodiments, the step of representing each entropy coded symbol in the entropy coded symbol vector by a symbol is performed using an identical probability table for all entropy coded symbols in the entropy coded symbol vector, wherein the index value associated with the first entropy coded symbol is calculated by the following : offset the symbol representing the first entropy-coded symbol in the vector of entropy-coded symbols by an offset value; applying a modulo N operation to a shifted symbol. The method further comprises the steps of: representing a first element of the parameter vector by a parameter value corresponding to an index value associated with the first entropy encoded symbol.

According to an embodiment, the probability table is converted into a Huffman encoding table, and each entropy encoded symbol corresponds to a codeword in the Huffman encoding table.

According to further embodiments, each codeword in the Huffman codebook is associated with a codebook index, and the step of representing each entropy-coded symbol in the vector of entropy-coded symbols by the symbol comprises representing the entropy-coded symbol by a codebook index associated with the codeword corresponding to the entropy-coded symbol. symbol.

In embodiments, each entropy coded symbol in the vector of entropy coded symbols corresponds to different frequency bands used in the audio decoding system in a particular time frame.

According to an embodiment, each entropy coded symbol in the vector of entropy coded symbols corresponds to different time frames used in the audio decoding system in a particular frequency band.

According to embodiments, the parameter vector corresponds to an element in the upmix matrix used by the audio decoding system.

According to exemplary embodiments, a computer-readable medium is provided comprising machine code instructions adapted to perform any method of the second aspect when executed on a device having processing capabilities.

According to exemplary embodiments, a decoder is provided for decoding a vector of entropy-coded symbols in an audio decoding system into a vector of parameters associated with an aperiodic quantity, the vector of entropy-coded symbols comprises a first entropy-coded symbol and at least one second entropy-coded symbol, and the parameter vector comprises the first element and at least the second element, the decoder comprises: a receiving component configured to receive a vector of entropy encoded symbols; an indexing component configured to represent each entropy-coded symbol in the vector of entropy-coded symbols by a symbol that can take N integer values by using a probability table; an association component configured to associate the first entropy encoded symbol with an index value; an association component further configured to associate each at least one second entropy encoded symbol with an index value, wherein the index value of the at least one second entropy encoded symbol is calculated by: calculating the sum of the index value associated with the entropy encoded symbol preceding the second entropy encoded symbol in the vector of entropy encoded symbols, and a symbol representing the second entropy encoded symbol; applying an operation modulo N to a sum. The decoder further includes a decoding component configured to represent the at least one second element of the parameter vector by a parameter value corresponding to an index value associated with the at least one second entropy-coded symbol.

III. Review: Sparse Matrix Encoder

According to a third aspect, exemplary embodiments provide encoding methods, encoders, and computer program products for encoding. The proposed methods, encoders and computer program products may generally have identical features and advantages.

According to exemplary embodiments, a method is provided for encoding an upmix matrix in an audio encoding system, wherein each row of the upmix matrix contains M elements capable of recovering a time-frequency tile of an audio object from a downmix signal containing M channels, the method comprising: for each row in the upmixing matrix: selecting a subset of elements from the M row elements in the upmixing matrix; representing each element in the selected subset of elements by a value and position in the upmixing matrix; encoding the value and position in the upmix matrix of each element in a selected subset of elements.

As used herein, the term “downmix signal containing M channels” means a signal that contains M signals or channels, each of the channels being a combination of a plurality of audio objects including audio objects to be reconstructed. The number of channels is typically greater than one, and in many cases the number of channels is five or more.

As used herein, the term "upmix matrix" means a matrix having N rows and M columns that allows N audio objects to be reconstructed from a downmix signal containing M channels. The elements on each row of the upmix matrix correspond to one audio object and provide coefficients that must be multiplied with the M downmix channels to reconstruct the audio object.

As used herein, by position in the upmix matrix, generally means a row and column index that indicates the row and column of the matrix element. The term "position" can also mean the index of a column in a given row of the upmix matrix.

In some cases, sending all the elements of the upmix matrix per time-frequency tile requires an undesirably high bit rate in the audio encoding/decoding system. The advantage of the method is that only a subset of the upmix matrix elements need to be encoded and transmitted to the decoder. This can reduce the required bit rate of an audio encoding/decoding system because less data is transmitted and the data can be encoded more efficiently.

Audio encoding/decoding systems typically partition time-frequency space into time-frequency tiles, for example, by applying suitable filter banks to the input audio signals. By a time-frequency tile, a portion of the time-frequency space corresponding to a time domain and frequency subband is generally designated. The time interval may typically correspond to the duration of a time frame used in an audio encoding/decoding system. A subband typically may correspond to one or more adjacent frequency subbands defined by a filter bank used in the encoding/decoding system. In the case where a sub-band corresponds to several adjacent frequency sub-bands defined by a filter bank, this allows for the possibility of having uneven sub-bands in the audio decoding process, for example, wider sub-bands for the higher frequencies of the audio signal. In the wideband case, in which the audio encoding/decoding system controls the entire frequency range, a subband of the time-frequency tile may correspond to the entire frequency range. The above-described method discloses encoding steps for encoding an upmix matrix in an audio encoding system to enable reconstruction of an audio object during one such time-frequency tile. However, it should be understood that the method may be repeated for each time-frequency tile of the audio encoding/decoding system. It should also be understood that multiple time-frequency tiles may be encoded simultaneously. Typically, adjacent time-frequency tiles may overlap a bit in time and/or frequency. For example, overlap in time may be equivalent to linear interpolation of the elements of the reconstruction matrix in time, i.e. from one time interval to the next. However, this disclosure is directed to other parts of the encoding/decoding system, and the implementation of overlap in time and/or frequency between adjacent time-frequency tiles is left to those skilled in the art.

In embodiments, for each row in the upmix matrix, the positions in the upmix matrix of a selected subset of elements vary across multiple frequency bands and/or across multiple time frames. Accordingly, the selection of elements may depend on a particular time-frequency tile, such that different elements may be selected for different time-frequency tiles. This provides a more flexible encoding method that improves the quality of the encoded signal.

In embodiments, the selected subset of elements contains an identical number of elements for each row of the upmix matrix. In further embodiments, the number of elements selected may be exactly one. This reduces encoder complexity as the algorithm only needs to select an identical number of element(s) for each line, i.e. the element(s) that are most important when performing decoder-side upmixing.

In embodiments, for each row in the upmix matrix and for a plurality of frequency bands or a plurality of time frames, the element values of the selected subsets of elements form one or more parameter vectors, wherein each parameter in the parameter vector corresponds to one of the plurality of frequency bands or multiple time frames, and wherein one or more parameter vectors are encoded using the method according to the first aspect. In other words, the meanings of the selected elements can be effectively encoded. Advantages regarding features and arrangements as presented in the review of the first aspect above may generally be acceptable for this embodiment.

In embodiments, for each row in the upmix matrix and for a plurality of frequency bands or a plurality of time frames, the element positions of the selected subsets of elements form one or more parameter vectors, wherein each parameter in the parameter vector corresponds to one of the plurality of frequency bands or multiple time frames, and wherein one or more parameter vectors are encoded using the method according to the first aspect. In other words, the positions of selected elements can be effectively encoded. Advantages regarding features and arrangements as presented in the review of the first aspect above may generally be acceptable for this embodiment.

According to exemplary embodiments, a computer-readable medium is provided comprising machine code instructions adapted to perform any method of the third aspect when executed on a device having processing capabilities.

According to exemplary embodiments, an encoder is provided for encoding an upmix matrix in an audio encoding system, wherein each row of the upmix matrix contains M elements capable of reconstructing a time-frequency tile of an audio object from a downmix signal containing M channels, wherein the encoder comprises: a receive a component adapted to receive each row in the upmix matrix; a selection component adapted to select a subset of elements from the M row elements in the upmixing matrix; an encoding component adapted to represent each element in the selected subset of elements by a value and position in the upmix matrix, wherein the encoding component is further adapted to encode the value and position in the upmix matrix of each element in the selected subset of elements.

IV. Review: Sparse Matrix Decoder

According to a fourth aspect, exemplary embodiments provide decoding methods, decoders, and computer program products for decoding. The proposed methods, decoders, and computer program products may generally have identical features and advantages.

The advantages regarding features and arrangements, as presented in the overview of the sparse matrix encoder above, may generally be valid for the corresponding features and arrangements for the decoder.

According to exemplary embodiments, there is provided a method for reconstructing a time-frequency tile of an audio object in an audio decoding system, comprising: receiving a downmix signal containing M channels; receiving at least one coded element representing a subset of M row elements in the upmix matrix, each coded element comprising a value and a row position in the upmix matrix, the position indicating one of the M downmix signal channels to which the coded element corresponds; and recovering a time-frequency tile of the audio object from the downmix signal by forming a linear combination of downmix channels that correspond to at least one encoded element, wherein in said linear combination each downmix channel is multiplied by the value of its corresponding encoded element.

Thus, according to this method, a time-frequency tile of an audio object is reconstructed by forming a linear combination of a subset of downmix channels. A subset of downmix channels corresponds to those channels for which encoded upmix coefficients are received. Thus, the method provides the ability to reconstruct an audio object even though only a subset, eg, a sparse subset, of the upmix matrix is received. By forming a linear combination of only the downmix channels that correspond to at least one encoded element, the complexity of the decoding process can be reduced. An alternative is to form a linear combination of all the downmix signals and then multiply some of them (that do not correspond to at least one coded element) with the value zero.

According to embodiments, the positions of at least one coded element vary across multiple frequency bands and/or across multiple time frames. In other words, different elements of the upmix matrix may be encoded for different time-frequency tiles.

According to embodiments, the number of elements of at least one coded element is one. This means that the audio object is reconstructed from one downmix channel in each time-frequency tile. However, the single downmix channel used to reconstruct the audio object may vary between different time-frequency tiles.

In embodiments, for a plurality of frequency bands or a plurality of time frames, the values of at least one encoded element form one or more vectors, wherein each value is represented by an entropy coded symbol, wherein each symbol in each entropy coded symbol vector corresponds to one from a plurality of frequency bands or one of a plurality of time frames, and wherein one or more vectors of entropy encoded symbols are decoded using the method according to the second aspect. In this way, the values of the upmix matrix elements can be efficiently encoded.

In embodiments, for a plurality of frequency bands or a plurality of time frames, the positions of at least one encoded element form one or more vectors, each position being represented by an entropy coded symbol, wherein each symbol in each entropy coded symbol vector corresponds to one from a plurality of frequency bands or a plurality of time frames, and wherein one or more vectors of entropy encoded symbols are decoded using the method according to the second aspect. In this way, the element positions of the upmix matrix can be efficiently encoded.

According to exemplary embodiments, a computer-readable medium is provided comprising machine code instructions adapted to perform any method of the third aspect when executed on a device having processing capabilities.

According to exemplary embodiments, a decoder is provided for reconstructing a time-frequency tile of an audio object, comprising: a receiving component configured to receive a downmix signal containing M channels and at least one encoded element representing a subset of M row elements in the upmix matrix mixing, each coded element contains a value and a row position in the upmix matrix, the position indicating one of the M downmix signal channels to which the coded element corresponds; and a recovery component configured to recover a time-frequency tile of an audio object from the downmix signal by forming a linear combination of downmix channels that correspond to at least one encoded element, wherein in said linear combination each downmix channel is multiplied by a value its corresponding coded element.

V. Exemplary Embodiments

Fig. 1 shows a general block diagram of an audio encoding system 100 for encoding audio objects 104. The audio encoding system includes a downmix component 106 that creates a downmix signal 110 from audio objects 104. The downmix signal 110, for example, may be 5.1 or 7.1 surround signals. , which are backward compatible with established audio decoding systems such as Dolby Digital Plus, or MPEG standards such as AAC, USAC or MP3. In further embodiments, the downmix signal is not backwards compatible.

To be able to recover audio objects 104 from the downmix signal 110, upmix parameters are determined in the upmix parameter analysis component 112 from the downmix signal 110 and audio objects 104. For example, the upmix parameters may correspond to elements of an upmix matrix that enables the reconstruction of audio objects 104 from the downmix signal 110. The upmix parameter analysis component 112 processes the downmix signal 110 and audio objects 104 relative to individual time-frequency tiles. Thus, upmix parameters are determined for each time-frequency tile. For example, an upmix matrix may be determined for each time-frequency tile. For example, the upmix parameter analysis component 112 may operate in a frequency domain, such as a quadrature mirror filter (QMF) domain, which enables frequency-selective processing. For this reason, the downmix signal 110 and audio objects 104 can be converted into the frequency domain by processing the downmix signal 110 and audio objects 104 in the filter bank 108. This may, for example, be accomplished by applying a QMF transform or any other suitable transform.

The upmix parameters 114 may be organized in a vector format. The vector may represent an upmix parameter for reconstructing a particular audio object from audio objects 104 in different frequency bands in a particular time frame. For example, a vector may correspond to a specific matrix element in the upmix matrix, where the vector contains the values of the specific matrix element for subsequent frequency bands. In additional embodiments, a vector may represent upmix parameters for reconstructing a particular audio object from audio objects 104 at different time frames in a particular frequency band. For example, a vector may correspond to a specific matrix element in the upmix matrix, wherein the vector contains the values of the specific matrix element for subsequent time frames, but at the same frequency band.

Each parameter in the vector corresponds to an aperiodic quantity, for example, a quantity that takes a value between -9.6 and 9.4. By aperiodic quantity, in general, we mean a quantity for which there is no periodicity in the values that the quantity can take. This is in contrast to a periodic quantity such as an angle, for which there is a clear periodic correspondence between the values that the quantity can take. For example, for an angle, a periodicity of 2π is provided, so that, for example, a zero angle corresponds to an angle of 2π.

The upmix parameters 114 are then received by the upmix matrix encoder 102 in vector format. Next, the upmix matrix encoder will be explained in detail in connection with FIG. 2. The vector is received by receiving component 202 and has a first element and at least one second element. The number of elements depends on, for example, the number of frequency bands in the audio signal. The number of elements may also depend on the number of time frames of the audio signal being encoded in one encoding operation.

The vector is then indexed by indexing component 204. The indexing component is configured to represent each parameter in the vector by an index value that can take on a predetermined number of values. This representation can be performed in two steps. First the parameter is quantized, and then the quantized value is indexed by the index value. As an example, in case each parameter in the vector can take a value between -9.6 and 9.4, this can be accomplished by using quantization steps of 0.2. The quantized values can then be indexed using indices 0-95, i.e. 96 different values. In the following examples, the index value is in the range 0-95, but this is of course only an example, other index value ranges are equally possible, for example 0-191 or 0-63. Smaller quantization steps may result in a less distorted decoded audio signal on the decoder side, but may also result in a higher required bit rate for transmitting data between the audio encoding system 100 and the decoder.

The indexed values are then sent to an association component 206, which associates every at least one second element with a symbol using a modulo differential encoding strategy. The association component 206 is configured to calculate the difference between the index value of the second element and the index value of the previous element in the vector. By using only the traditional differential encoding strategy, the difference can be any value in the range from -95 to 95, i.e. it has 191 possible meanings. This means that when the difference is encoded using entropy coding, a probability table containing 191 probabilities is required, i.e. one probability for each of the 191 possible difference values. In addition, coding efficiency is reduced because for each difference, approximately half of the 191 probabilities are impossible. For example, if the second element to be differentially encoded has an index value of 90, possible differences range from -5 to +90. Typically, having an entropy encoding strategy in which some probabilities are impossible for each value to be encoded reduces encoding efficiency. The differential coding strategy in this disclosure can overcome this problem and simultaneously reduce the number of required codes to 96 by applying a modulo 96 operation to the difference. The association algorithm can therefore be expressed as:

where b is an element in the differentially encoded vector, N _Q is the number of possible index values, and is the symbol associated with element b.

In some embodiments, the probability table is converted to a Huffman encoding table. In this case, the character associated with the element in the vector is used as the codebook index. Encoding component 208 may then encode each of the at least one second element by representing the second element with a codeword in a Huffman codebook that is indexed by the codebook index associated with the second element.

Any other suitable entropy encoding strategy may be implemented in encoding component 208. As an example, such an encoding strategy may be a range encoding strategy or an arithmetic encoding strategy.

It is further shown that the entropy of the modulo operation approach is always lower than or equal to the entropy of the traditional differential approach. Entropy, , the traditional differential approach is as follows:

Where is the probability of a simple differential value of the n index.

Entropy, , the approach to performing the operation modulo is as follows:

where q(n) is the probability of the differential index value n modulo as specified by the following:

Because of this we have the following:

Substitution in the outputs of the last summation

Additionally,

When comparing sums term by term, since

and similarly:

we have that

As shown above, the entropy for the modulo operation approach is always lower than or equal to the entropy of the traditional differential approach. The case in which the entropy is equal is the rare case where the data to be encoded is pathological data, i.e. data with a deviating pattern of change that in most cases does not apply, for example, to an upmix matrix.

Since the entropy of the modulo approach is always lower than or equal to the entropy of the traditional differential approach, entropy encoding of symbols computed by the modulo approach should result in a lower or at least identical bit rate compared to entropy encoding symbols computed using the traditional differential approach. In other words, entropy encoding of symbols computed using the modulo approach is, in most cases, more efficient than entropy encoding of symbols computed using the traditional differential approach.

An additional advantage, as mentioned above, is that the number of required probabilities in the probability table in the modulo approach is approximately half of the number of required probabilities in the traditional non-modulo approach.

The above describes an approach to perform a modulo operation to encode at least one second element in a parameter vector. The first element may be encoded by using the indexed value by which the first element is represented. Since the probability distribution of the index value of the first element and the differential value modulo of at least one second element may be significantly different (see Fig. 3 for the probability distribution of the indexed first element and Fig. 4 for the probability distribution of the differential value modulo, t .e. symbol for at least one second element), a dedicated probability table for the first element may be necessary. This requires that both the audio encoding system 100 and the corresponding decoder have such a dedicated probability table in their storage.

However, the inventors observe that the shape of the probability distributions in some cases can be quite similar, although shifted relative to each other. This observation may be used to approximate the probability distribution of the indexed first element by a shifted version of the symbol probability distribution of the at least one second element. Such a shift may be implemented by adapting the association component 206 to associate the first element in the vector with a symbol by shifting the index value representing the first element in the vector by an offset value and then applying modulo 96 (or the corresponding value) to the shifted index value.

The computation of the symbol associated with the first element can therefore be expressed as:

Such an achieved symbol is used by encoding component 208, which encodes the first element by entropy encoding the symbol associated with the first element using an identical probability table that is used to encode at least one second element. The offset value may be equal to, or at least close to, the difference between the most likely index value for the first element and the most likely symbol for at least one second element in the probability table. In fig. 3, the most probable index value for the first element is indicated by arrow 302. Provided that the most probable symbol for at least one second element is zero, the value indicated by arrow 302 should be the offset value used. By using the offset approach, the peaks of the distributions in Figs. 3 and 4 are combined. This approach eliminates the need for a dedicated probability table for the first element, and therefore saves storage space in the audio encoding system 100 and associated decoder while maintaining substantially identical encoding efficiency that a dedicated probability table provides.

In the event that entropy encoding of at least one second element is performed using a Huffman codebook, encoding component 208 may encode the first element in the vector using an identical Huffman codebook that is used to encode at least one second element. element by representing the first element with a codeword in a Huffman codebook, which is indexed by the codebook index associated with the first element.

Since search speed may be important when encoding a parameter in an audio decoding system, the storage device on which the coding table is stored is advantageously a high-speed storage device and is therefore expensive. By using only one probability table, the encoder can therefore be less expensive than if two probability tables are used.

It may be noted that the probability distributions shown in FIG. 3 and fig. 4 are often calculated in advance for the training data set and are therefore not calculated when encoding the vector, but of course it is possible to calculate distributions on the fly during encoding.

It may also be noted that the above description of the audio encoding system 100 using a vector from an upmix matrix as a parameter vector to be encoded is merely an exemplary application. The method for encoding a parameter vector according to this disclosure may be used in other applications in an audio encoding system, for example, when encoding other internal parameters in a downmix encoding system, for example, parameters used in a parametric bandwidth extension system, to for example, spectrum band replication (SBR).

Fig. 5 is a general block diagram of an audio decoding system 500 for re-creating coded audio objects from a coded downmix signal 510 and a coded upmix matrix 512. The encoded downmix signal 510 is received by the receive downmix component 506, in which the signal has already been decoded, and, if not in a suitable frequency domain, is converted to a suitable frequency domain. The decoded downmix signal 516 is then sent to the upmix component 508. In the upmix component 508, encoded audio objects are reconstructed using the decoded downmix signal 516 and the decoded upmix matrix 504. More specifically, the upmix component 508 may perform a matrix operation in which the decoded upmix matrix 504 is multiplied by a vector containing the decoded downmix signals 516. The following describes the decoding process of the upmix matrix. The audio decoding system 500 further includes a rendering component 514 that outputs an audio signal based on the reconstructed audio objects 518 depending on which playback module is connected to the audio decoding system 500.

The encoded upmix matrix 512 is received by the upmix matrix decoder 502, which is explained in detail below in connection with FIG. 6. The upmix matrix decoder 502 is configured to decode a vector of entropy encoded symbols in an audio decoding system into a vector of parameters associated with an aperiodic quantity. The entropy-coded symbol vector contains a first entropy-coded symbol and at least one second entropy-coded symbol, and the parameter vector contains a first element and at least a second element. The encoded upmix matrix 512 is therefore received by the receiving component 602 in vector format. Decoder 502 further includes an indexing component 604 configured to represent each entropy encoded symbol in the vector by a symbol that can take on N values through the use of a probability table. N, for example, may be 96. The association component 606 is configured to associate the first entropy encoded symbol with an index value by any suitable means, depending on the encoding method used to encode the first element in the parameter vector. The symbol for each of the second codes and the index value for the first code are then used by an association component 606 that associates each of the at least one second entropy encoded symbol with an index value. The index value of at least one second entropy encoded symbol is calculated by first calculating the sum of the index value associated with the entropy encoded symbol preceding the second entropy encoded symbol in the vector of entropy encoded symbols and the symbol representing the second entropy encoded symbol. Then, the modulo N operation is applied to the sum. Assume, without loss of generality, that the minimum index value is 0 and the maximum index value is N-1, for example 95. The association algorithm can therefore be expressed as:

where b is an element in the vector to be decoded, and is the number of possible index values.

The upmix matrix decoder 502 further includes a decoding component 608, which is configured to represent at least one second element of the parameter vector by a parameter value corresponding to an index value associated with the at least one second entropy-coded symbol. Thus, this representation is a decoded version of a parameter encoded, for example, by the audio encoding system 100 shown in FIG. 1. In other words, this representation is equal to the quantized parameter encoded by the audio encoding system 100 shown in FIG. 1.

According to one embodiment of the present invention, each entropy coded symbol in the entropy coded symbol vector is represented by a symbol using an identical probability table for all entropy coded symbols in the entropy coded symbol vector. The advantage of this is that only one probability table needs to be stored in the decoder memory. Since lookup speed may be important when decoding an entropy-coded symbol in an audio decoding system, the storage device on which the probability table is stored is advantageously a high-speed storage device and is therefore expensive. By using only one probability table, the decoder can therefore be less expensive than if two probability tables are used. According to this embodiment, associate component 606 may be configured to associate a first entropy-coded symbol with an index value by a first offset of the symbol representing the first entropy-coded symbol in the vector of entropy-coded symbols by an offset value. The modulo N operation is then applied to the shifted symbol. The association algorithm can therefore be expressed as:

The decoding component 608 is configured to represent the first element of the parameter vector by a parameter value corresponding to an index value associated with the first entropy encoded symbol. Thus, this representation is a decoded version of a parameter encoded by, for example, the audio encoding system 100 shown in FIG. 1.

Next, the differential encoding method for an aperiodic quantity will be explained in detail in connection with FIG. 7-10.

Fig. 7 and 9 describe the encoding method for the second four (4) elements in the parameter vector. The input vector 902 therefore contains five parameters. Parameters can take any value between the minimum value and maximum value. In this example, the minimum value is 9.6 and the maximum value is 9.4. The first step S702 in the encoding method is to represent each parameter in the vector 902 by an index value that may have N values. In this case, N is chosen to be 96, which means that the quantization step size is 0.2. This produces a vector 904. The next step S704 is to calculate the difference between each of the second elements, i.e. the top four parameters in vector 904, and its previous element. The resulting vector 906 therefore contains four differential values: these are the top four values in vector 906. As can be seen in FIG. 9, differential values can be negative, zero and positive. As explained above, it is preferable to have differential values that can only take N values, in this case 96 values. To achieve this, in the next step S706 of this method, a modulo 96 operation is applied to the second elements in the vector 906. The resulting vector 908 does not contain negative values. Such reached symbol shown in vector 908 is then used to encode second elements of the vector in final step S708 of the method shown in FIG. 7 by entropy encoding a symbol associated with at least one second element based on a probability table containing probabilities of the symbols shown in vector 908.

As can be seen in FIG. 9, the first element is not processed after the indexing step S702. In fig. 8 and 10, a method for encoding the first element in an input vector is described. An assumption identical to the assumption in the above description of FIG. 7 and 9 regarding the minimum and maximum values of the parameters and the number of possible index values, is valid when describing FIG. 8 and 10. The first element 1002 is received by the encoder. In the first encoding method step S802, a first element parameter is represented by an index value 1004. In the next step S804, the indexed value 1004 is shifted by the offset value. In this example, the offset value is 49. This value is calculated as described above. In the next step S806, a modulo 96 operation is applied to the index value 1006 to be shifted. The resulting value 1008 can then be used in encoding step S802 to encode the first element by entropy encoding the symbol 1008 using an identical probability table that is used to encode the at least one second element in FIG. 7.

Fig. 11 shows an embodiment 102' of the upmix matrix encoding component 102 of FIG. 1. The upmix matrix encoder 102' may be used to encode the upmix matrix in an audio coding system, for example, the audio coding system 100 shown in FIG. 1. As described above, each row of the upmix matrix contains M elements allowing the audio object to be reconstructed from a downmix signal containing M channels.

At low overall target bit rates, encoding and sending all M upmix matrix elements per object and T/F tile, one for each downmix channel, may require an undesirably high bit rate. This can be reduced by "thinning" the upmix matrix, i.e. trying to reduce the number of non-zero elements. In some cases, four of the five elements are zeros, and only one downmix channel is used as the basis for reconstructing the audio object. Sparse matrices have different probability distributions of encoded indices (absolute or differential) compared to non-sparse matrices. In cases where the upmix matrix contains a significant portion of zeros such that the value zero becomes more likely than 0.5, and Huffman encoding is used, the encoding efficiency is reduced because the Huffman encoding algorithm is inefficient when a particular value, such as zero, has a probability greater than 0.5. In addition, since many elements in the upmix matrix have a value of zero, they do not contain information. A strategy could therefore be to select a subset of the upmix matrix elements and only encode that and pass it to the decoder. This may reduce the required bit rate of the audio encoding/decoding system since less data is transmitted.

To increase the coding efficiency of the upmix matrix, a dedicated sparse matrix coding mode can be used, which is explained in more detail below.

Encoder 102' includes a receive component 1102 adapted to receive each row in the upmix matrix. Encoder 102' further includes a selection component 1104 adapted to select a subset of elements from the M row elements in the upmix matrix. In most cases, the subset contains all elements that do not have a null value. But according to some embodiment, the selection component may choose not to select an element that has a non-zero value, for example, an element that has a value close to zero. In embodiments, the selected subset of elements may contain an identical number of elements for each row of the upmix matrix. To further reduce the required bit rate, the number of elements selected may be one (1).

Encoder 102' further includes an encoding component 1106, which is configured to represent each element in a selected subset of elements by a value and position in the upmix matrix. The encoding component 1106 is further configured to encode the value and position in the upmix matrix of each element in the selected subset of elements. This may, for example, be configured to encode the value using differential modulo encoding as described above. In this case, for each row in the upmix matrix and for a plurality of frequency bands or multiple time frames, the element values of the selected subsets of elements form one or more parameter vectors. Each parameter in the parameter vector corresponds to one of a plurality of frequency bands or a plurality of time frames. The parameter vector can therefore be encoded using modulo differential encoding as described above. In additional embodiments, the parameter vector may be encoded using conventional differential encoding. In yet another embodiment, encoding component 1106 is configured to encode each value separately using fixed rate encoding of the true quantization value, i.e. not differentially encoded, each value.

The following examples of average data rates are observed for typical content. The bit rates are measured for the case in which M=5, the number of audio objects to be reconstructed at the decoder side is 11, the number of frequency bands is 12, and the step size of the parameter quantizer is 0.1 and has 192 levels. For the case in which all five elements per row in the upmix matrix were encoded, the following average data rates are observed:

Fixed bit rate encoding: 165 Kbps,

Differential encoding: 51 Kbps,

Differential modulo encoding: 51 Kbps, but with half the size of the probability table or encoding table as described above.

For the case in which only one element is selected for each row in the upmixing matrix, i.e. sparse coding, through selection component 1104, the following average data rates are observed:

Fixed rate encoding (using 8 bits for value and 3 bits for position): 45 Kbps,

Differential encoding modulo for both element value and element position: 20 Kbps.

Encoding component 1106 may be configured to encode a position in the upmix matrix of each element in a subset of elements in a similar manner as a value. Encoding component 1106 may also be configured to encode the position in the upmix matrix of each element in a subset of elements in a different manner compared to encoding the value. In the case of position encoding using differential encoding or differential modulo encoding, for each row in the upmix matrix and for a plurality of frequency bands or multiple time frames, the element positions of the selected subsets of elements form one or more parameter vectors. Each parameter in the parameter vector corresponds to one of a plurality of frequency bands or a plurality of time frames. The parameter vector is therefore encoded using differential encoding or modulo differential encoding as described above.

It may be noted that the encoder 102' may be combined with the encoder 102 in FIG. 2 to achieve differential encoding modulo the sparse upmix matrix according to the above.

It may be further noted that the method for encoding a row in a sparse matrix was illustrated above for encoding a row in a sparse upmix matrix, but the method can be used for encoding other types of sparse matrices known to those skilled in the art.

Next, a method for encoding a sparse upmix matrix will be explained in detail in connection with FIG. 13-15.

The upmix matrix is received, for example, by the receiving component 1102 in FIG. 11. For each row 1402, 1502 in the upmix matrix, the method comprises selecting a subset S1302 of M, for example 5, row elements in the upmix matrix. Each element in the selected subset of elements is then represented by S1304 by a value and position in the upmix matrix. In fig. 14, one element is selected by S1302 as a subset, for example, element number 3 having a value of 2.34. The representation may therefore be a vector 1404 having two fields. The first field in vector 1404 represents a value, for example, 2.34, and the second field in vector 1404 represents a position, for example, 3. In FIG. 15, two items are selected by S1302 as a subset, for example, item number 3 having a value of 2.34 and item number 5 having a value of -1.81. The representation may therefore be a vector 1504 having four fields. The first field in vector 1504 represents the value of the first element, such as 2.34, and the second field in vector 1504 represents the position of the first element, such as 3. The third field in vector 1504 represents the value of the second element, such as -1.81, and the fourth a field in vector 1504 represents the position of the second element, for example, 5. Representations 1404, 1504 are then encoded by S1306 as above.

Fig. 12 is a general block diagram of an audio decoding system 1200 in accordance with an exemplary embodiment. Decoder 1200 includes a receiving component 1206 configured to receive a downmix signal 1210 comprising M channels and at least one coded element 1204 representing a subset of the M row elements in the upmix matrix. Each of the coded elements contains a value and a row position in the upmix matrix, a position indicating one of the M channels of the downmix signal 1210 to which the coded element corresponds. At least one coded element 1204 is decoded by an upmix matrix element decoding component 1202. The upmix matrix element decoding component 1202 is configured to decode at least one encoded element 1204 according to an encoding strategy used to encode the at least one encoded element 1204. Examples for such encoding strategies are disclosed above. The at least one decoded element 1214 is then sent to a recovery component 1208, which is configured to recover a time-frequency tile of an audio object from the downmix signal 1210 by generating a linear combination of downmix channels that correspond to the at least one encoded element 1204 When forming a linear combination, each downmix channel is multiplied by the value of its corresponding coded element 1204.

For example, if decoded element 1214 contains a value of 1.1 and a position of 2, the time-frequency tile of the second downmix channel is multiplied by 1.1 and this is then used to reconstruct the audio object.

The audio decoding system 500 further includes a rendering component 1216 that outputs an audio signal based on the reconstructed audio object 1218. The type of audio signal depends on what type of playback module is connected to the audio decoding system 1200. For example, if a pair of headphones is connected to the audio decoding system 1200, a stereo signal may be output by rendering component 1216.

Equivalents, additions, alternatives, as well as various aspects

Additional embodiments of the present disclosure will become apparent to those skilled in the art upon examination of the foregoing description. Even though the present description and drawings disclose embodiments and examples, the disclosure is not limited to these specific examples. Many modifications and changes may be made without departing from the scope of the present disclosure, which is set forth by the appended claims. All reference numbers contained in the claims should not be construed as limiting its scope.

Additionally, variations in the disclosed embodiments may be understood and carried out by those skilled in the art in practicing the claimed invention from examination of the drawings, disclosure, and appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain agents are mentioned in various dependent claims does not mean that a combination of these agents cannot be advantageously used.

The systems and methods disclosed above may be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the division of concerns between functional modules, as denoted in the above description, does not necessarily correspond to the division into physical modules; conversely, a single physical component can have multiple functionalities, and a single task can be performed by multiple physical components together. Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or implemented as hardware or an application specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or nonvolatile media) and communications media (or volatile media). As is known to those skilled in the art, the term "computer storage media" includes both volatile and non-volatile, removable and non-removable media, implemented in any method or technology for storing information, for example, through machine-readable instructions, data structures, software modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic storage disks or other magnetic storage devices, or any other medium that can be used to store the required information and that can be accessed by a computer. Additionally, those skilled in the art will recognize that communications media typically implement machine-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier or other transport mechanism, and include any information delivery media.

Claims (7)

1. A method for encoding a vector of parameters in an audio coding system, wherein each parameter corresponds to an aperiodic quantity, the vector has a first element and at least one second element, the method comprising: representing each parameter in the vector by an index value that may take N values; calculating one or more symbols, the calculation including: calculating the difference between the index value of the second element and the index value of its previous element in the vector: and applying a modulo N operation to the difference; associating each of the at least one second element with a corresponding symbol of the one or more symbols, and encoding each of the at least one second element by entropy encoding the symbol associated with the at least one second element based on the probability table , containing the probabilities of the symbols. 2. The method of claim 1, wherein the first element and at least one second element of the parameter vector correspond to different frequency bands used in the audio coding system in a particular time interval. 3. The method of claim 1, wherein the first element and at least one second element of the parameter vector correspond to different time frames used in the audio coding system in a particular frequency band. 4. The method of claim 1, wherein the probability table is converted to a Huffman encoding table, wherein a symbol associated with an element in the vector is used as an index of the encoding table, and wherein the encoding step comprises encoding each of at least one the second element by representing the second element by a codeword in a codebook, which is indexed by a codebook index associated with the second element. 5. The method of claim 4, wherein the encoding step comprises encoding the first element in the vector using an identical Huffman encoding table that is used to encode the at least one second element by representing the first element with a codeword in the table Huffman encoding, which is indexed by the codebook index associated with the first element. 6. The method of claim 1, wherein the parameter vector corresponds to an element in the upmix matrix determined by the audio coding system. 7. A computer-readable storage medium containing machine code instructions adapted to carry out the method of claim 1 when executed on a device having processing capabilities.

Images