KR20080037105A - Apparatus for encoding and decoding audio signal and method thereof - Google Patents

Abstract

An apparatus for encoding and decoding an audio signal and a method thereof are provided to represent the location of a time slot, to which a first parameter set is applied, as an absolute value and represent the location of a time slot, to which a second or more parameter sets are applied, as a difference value, thereby reducing the amount of transmission data. A bsFramingType field(1301) indicates whether the spatial frame of an audio signal is a fixed frame type or a variable frame type, and the fixed frame indicates a frame, of which parameter sets are applied to a predetermined time slot. A bsNumParamSets field(1302) indicates the number of the parameter sets within a spatial frame(numParamSets=bsNumparaSets+1). A bsParamSlot field(1303) indicates the location information of the time slot to which the parameter sets are applied. The bsParamSlot field is represented as the variable number of bits using numSlots, the number of the time slots within a spatial frame.

Description

Apparatus and method for encoding and decoding audio signals

The present invention mainly relates to audio signal processing.

A new approach for recognizing the coding of multichannel audio, commonly called Spatial Audio Coding (SAC), is under research and development. SAC enables multi-channel audio to be transmitted at low bit rates, making it suitable for many audio applications (eg, internet streaming, music downloads, etc.).

SAC obtains the spatial image of a multi-channel audio signal with a simpler set of parameters than distributed coding individual audio input channels. These parameters are sent to the decoder and used to synthesize or reconstruct the spatial characteristics of the audio signal.

In some SAC applications, spatial parameters are transmitted to the decoder as part of the bitstream. This bitstream includes spatial frames, which include a set of time slots arranged such that spatial parameters can be applied. The bitstream also includes positional information, which the decoder can use to identify the exact time slot to which a given set of parameters is applied.

Some SAC applications use conceptual elements in the encoding / decoding path. One such element is commonly referred to as one-to-two (OTT), and the other is commonly referred to as two-to-three (TTT), each of which is called the input channel of the corresponding decoder element. It means the number of output channels. The OTT encoder element extracts two spatial parameters to produce a downmix signal and a residual signal. The TTT element downmixes three audio signals into one downmix signal and one residual signal. These elements can be combined to provide a variety of configurations of spatial audio environment (eg, surround sound, etc.).

Some SAC applications may operate in a non-guided mode of operation, in which only stereo downmix signals are transmitted from the encoder to the decoder without the need for spatial parameter transmission. The decoder synthesizes spatial parameters from the downmix signal used to generate a multi-channel audio signal.

Spatial information associated with an audio signal is encoded into a bitstream that can be transmitted to a decoder or recorded on a recording medium. Such bitstreams may include different syntaxes relating to time, frequency and spatial domains. In some embodiments, the bitstream may include one or more data structures (eg, frames) that include an ordered set of slots so that parameters can be applied. This data structure may be fixed or variable. A data structure type indicator may be included in the bitstream that allows the decoder to determine the data structure type and invoke the appropriate decoding process. The data structure can include location information, and the decoder can use this location information to identify the correct slot to which a given set of parameters is applied. The slot position information may be encoded into a fixed number of bits or a variable number of bits according to the data structure type indicated by the data structure type indicator. For a variable data structure type, slot position information may be encoded into a variable number of bits depending on the position of the slot in the aligned set of slots.

In some embodiments, a method of encoding an audio signal comprises: generating a parameter set of an audio signal corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And inserting said parameter set and said corresponding first or second information into a bitstream representing said audio signal, wherein said first or second information is represented by a variable number of bits. .

In some embodiments, a method of decoding an audio signal comprises: receiving a bitstream comprising a parameter set corresponding to a first information or a second information of an audio signal or corresponding to a range of the first information or the second information. step; And decoding the audio signal based on the parameter set and the first information or the second information, wherein the first information or the second information is represented by a variable number of bits.

Other embodiments of time slot position coding with respect to systems, methods, apparatus, data structures, and computer readable media are also disclosed.

It is to be understood that both the foregoing general description and the detailed description of the following examples are intended to be illustrative, for the purpose of understanding the invention as claimed in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to aid the understanding of the present invention, illustrate embodiments of the present invention, and together with the present specification illustrate the principles of the present invention.

1 is a diagram illustrating a winry for generating spatial information according to an embodiment of the present invention;

2 is a block diagram of an encoder for encoding an audio signal according to an embodiment of the present invention;

3 is a block diagram of a decoder for decoding an audio signal according to an embodiment of the present invention;

4 is a block diagram of a channel converter included in an upmixing unit of a decoder according to an embodiment of the present invention;

5 is a block diagram illustrating a method of constructing a bitstream of an audio signal according to an embodiment of the present invention;

6A and 6B are diagrams and time / frequency graphs for explaining the relationship between parameter sets, time slots and parameter bands, respectively, according to one embodiment of the present invention;

7A is a diagram showing a syntax for displaying configuration information of a spatial information signal according to an embodiment of the present invention;

7B is a table showing the number of parameter bands of a spatial information signal according to an embodiment of the present invention;

FIG. 8A is a diagram illustrating syntax representing the number of parameter bands applied to an OTT box as a fixed number of bits, in accordance with an embodiment of the present invention; FIG.

FIG. 8B is a diagram illustrating syntax representing the number of parameter bands applied to an OTT box with a variable number of bits, in accordance with an embodiment of the present invention; FIG.

FIG. 9A is a diagram illustrating syntax representing the number of parameter bands applied to a TTT box with a fixed number of bits, in accordance with an embodiment of the present invention; FIG.

FIG. 9B is a diagram illustrating syntax representing the number of parameter bands applied to a TTT box with a variable number of bits, in accordance with an embodiment of the present invention; FIG.

10A is a diagram illustrating the syntax of spatial extension configuration information for a spatial extension frame according to an embodiment of the present invention;

10B and 10C are diagrams illustrating syntax of spatial extension configuration information for a residual signal when the residual signal is included in the spatial extension frame according to an embodiment of the present invention;

10D is a diagram illustrating syntax for a method for indicating the number of parameter bands for a residual signal according to an embodiment of the present invention;

11A is a block diagram of a decoding apparatus using non-guided coding according to an embodiment of the present invention;

FIG. 11B is a diagram illustrating a method of grouping the number of parameter bands according to an embodiment of the present invention; FIG.

12 is a diagram illustrating syntax of configuration information of a space frame according to an embodiment of the present invention;

13A is a diagram illustrating syntax of position information of a time slot to which a parameter set is applied according to an embodiment of the present invention;

FIG. 13B is a diagram illustrating syntax for expressing position information of a time slot to which a parameter set is applied as an absolute value and a difference value according to an embodiment of the present invention; FIG.

FIG. 13C is a diagram illustrating a plurality of position information of time slots to which parameter sets are applied as a group according to an embodiment of the present invention; FIG.

14 is a flowchart of an encoding method according to an embodiment of the present invention;

15 is a flowchart of a decoding method according to an embodiment of the present invention;

FIG. 16 is a block diagram illustrating an apparatus structure for implementing the encoding and decoding process described with reference to FIGS. 1 to 15.

1 is a diagram illustrating a principle of generating spatial information according to an embodiment of the present invention. The concept of a coding scheme for multi-channel audio signals is based on the fact that humans perceive audio signals three-dimensionally. The three-dimensional space of the audio signal may be represented using spatial information, which includes channel level difference (CLD), inter channel correlation / coherence (ICC), and channel time difference (ICD). Channel Time Difference (CTD), Channel Prediction Coefficients (CPC), and the like, but are not limited thereto. CLD means energy (level) difference between two audio channels, ICC means the amount of correlation or coherence between two audio channels, and CTD means time difference between two channels.

The generation of the CTD and CLD parameters is shown in FIG. 1. From the far sound source 101, the first direct sound wave 103 reaches the human left ear 107, and the second direct sound wave 102 is diffracted around the human head and then the human right ear 106. ) The two sound waves 102 and 103 are mutually different in arrival time and energy level. The CTD parameter and the CLD parameter are generated based on the arrival time and energy level difference of the sound waves 102 and 103. Also, reflected sound waves 104 and 105 reach both ears 106 and 107, respectively, which are irrelevant. ICC parameters may be generated based on the correlation between sound waves 104 and 105.

In the encoder, spatial information (eg, spatial parameters, etc.) is extracted from the multi-channel audio signal, and a downmix signal is generated. The downmix signal and spatial parameters are sent to the decoder. Although not limited thereto, any number of audio channels may be used for the downmix signal including a mono signal, a stereo signal, or a multi-channel audio signal. At the decoder, a multi-channel upmix signal is generated from the downmix signal and the spatial parameters.

2 is a block diagram of an encoder for encoding an audio signal according to an embodiment of the present invention. The encoder includes a downmixing unit 202, a spatial information generating unit 203, a downmix signal encoding unit 207, and a multiplexing unit 209. Other configurations of the encoder are also possible. The encoder can be implemented in hardware or software, or a combination of hardware and software. The encoder can be implemented as an integrated circuit chip, a chip set, a System on Chip (SoC), a digital signal processor, a general purpose processor, and various digital and analog devices.

The downmixer 202 generates the downmix signal 204 from the multi-channel audio signal 201. In FIG. 2, x1, and xn represent input audio channels. As described above, the downmix signal 204 can be a mono signal, a stereo signal or an audio signal. In the example shown, x'1, x'm represent the channel number of the downmix signal 204. In some embodiments, the encoder processes externally supplied downmix signal 205 (eg, sophisticated downmix, etc.) instead of downmix signal 204.

The spatial information generator 203 extracts spatial information from the multi-channel audio signal 201. In this case, the spatial information means information associated with an audio signal channel used for upmixing the downmix signal 204 into a multichannel audio signal at the decoder. The downmix signal 204 is generated by downmixing the multichannel audio signal. The spatial information is encoded to provide an encoded spatial information signal 206.

The downmix signal encoding unit 207 encodes the downmix signal 204 generated by the downmixing unit 202 to generate an encoded downmix signal 208.

The multiplexer 209 generates the bitstream 210 including the encoded downmix signal 208 and the encoded spatial information signal 206. Bitstream 210 is transmitted to a downstream decoder and / or recorded on a recording medium.

3 is a block diagram of a decoder for decoding an encoded audio signal according to an embodiment of the present invention. The decoder includes a demultiplexer 302, a downmix signal decoder 305, a spatial information decoder 307, and an upmixer 309. The decoder may be implemented in hardware or software, or a combination of hardware and software. The decoder may be implemented as an integrated circuit chip, chipset, system on chip (SoC), digital signal processor, general purpose processor, and various digital devices or devices.

In some embodiments, demultiplexer 302 receives a bitstream 301 representing an audio signal and encodes the downmix signal 303 encoded from the bitstream 301 and the encoded spatial information signal 304. ). In FIG. 3, x'1, x'm represents the channel of the downmix signal 303. The downmix signal decoding unit 305 decodes the encoded downmix signal 303 and outputs the decoded downmix signal 306. If the decoder cannot output the multi-channel audio signal, the downmix signal decoding unit 305 may directly output the downmix signal 306. In FIG. 3, y'1, and y'm represent direct output channels of the downmix signal decoding unit 305.

The spatial information signal decoding unit 307 extracts configuration information of the spatial information signal from the encoded spatial information signal 304, and decodes the spatial information signal 304 using the extracted configuration information.

The upmixing unit 309 may upmix the downmix signal 306 into the multi-channel audio signal 310 using the extracted spatial information 308. In FIG. 3, y1, and y'n denote output channel numbers of the upmixing unit 309. In FIG.

4 is a block diagram of a channel conversion module that may be included in the upmixing unit 309 of the decoder illustrated in FIG. 3. In some embodiments, the upmixing unit 309 may include a plurality of channel conversion modules. The channel conversion module is a conceptual device that can distinguish between the number of input channels and the number of output channels by using specific information.

In some embodiments, the channel conversion module includes a one-to-two box for converting one channel into two channels, a device, and the like, and a TTT (Two for converting two channels into three channels, a device, etc.). -To-Three) box. OTT boxes and / or TTT boxes can be arranged in various useful configurations. For example, the upmixing unit 309 illustrated in FIG. 3 may include a 5-1-5 configuration, a 5-2-5 configuration, a 7-2-7 configuration, a 7-5-7 configuration, and the like. In the 5-1-5 configuration, downmixing five channels into one channel produces a downmix signal with one channel, which can be upmixed into five channels later. Other configurations using various combinations of OTT boxes and TTT boxes can be created in the same way.

Referring to FIG. 4, an example 5-2-5 of the upmixing unit 400 is illustrated. In the 5-2-5 configuration, the downmix signal 401 having two channels is input to the upmixing unit 400. In the example shown, the left channel L and the right channel R are provided as inputs to the upmixing unit 400. In the present embodiment, the upmixing unit 400 includes one TTT box 402 and three OTT boxes 406, 407, and 408. A downmix signal 401 with two channels is provided as input to the TTT box TTT0, which processes the downmix signal 401 to provide three output channels 403, 404 and 405. . One or more spatial parameters (eg, CPC, CLD, ICC, etc.) may be provided as inputs to the TTT box 402 and may be used to process the downmix signal 401 as described below. In this case, the CPC can be described as a prediction coefficient that produces three channels from two channels.

The channel 403, which serves as the output from the TTT box 402, is provided as an input to the OTT box 406 which generates two output channels using one or more spatial parameters. In the example shown, the two output channels represent, for example, a front left (FL) speaker position and a back left (BL) speaker position in a surround sound environment. Channel 404 is provided as an input to OTT box 407 that generates two output channels using one or more spatial parameters. In the example shown, the two output channels represent a front right (FR) speaker position and a back right (BR) speaker position. Channel 405 is provided as an input to OTT box 408 which generates two output channels. In the example shown, two output channels represent a center (C) speaker position and a low frequency enhancement (LFE) channel. In this case, spatial information (eg CLD, ICC, etc.) is provided as input for each of the OTT boxes. In some embodiments, residual signals Res1 and Res2 may be provided as inputs to OTT boxes 406 and 407. In this embodiment, the residual signal may not be provided as an output to the OTT box 408 which outputs the center channel and the LFE channel.

4 is an example of a configuration for a channel conversion module. Other configurations are also possible for the channel conversion module, including various combinations of OTT boxes and TTT boxes. Since each of the channel conversion modules may operate in the frequency domain, the number of parameter bands applied to each of the channel conversion modules may be defined. The parameter band means at least one frequency band that can be applied to one parameter. The number of parameter bands is described with reference to FIG. 6B.

5 is a diagram illustrating a method of constructing a bitstream of an audio signal according to an embodiment of the present invention. FIG. 5A illustrates a bitstream of an audio signal including only a spatial information signal, and FIGS. 5B and 5C illustrate a bitstream of an audio signal including a downmix signal and a spatial information signal. do.

Referring to FIG. 5A, a bitstream of an audio signal may include configuration information 501 and a frame 503. The frame 503 may be repeated bitstrings, and in some embodiments includes one spatial frame 502 that contains spatial audio information.

In some embodiments, configuration information 501 may include the total number of time slots in one spatial frame 502, the total number of parameter bands extending the frequency range of the audio signal, and the parameter bands in the OTT box. Information indicating the number, the number of parameter bands in the TTT box, and the number of parameter bands in the residual signal. The configuration information 501 may include other information as desired.

In some embodiments, spatial frame 502 may include one or more spatial parameters (eg, CLD, ICC, etc.), a frame type, and the number and parameter sets of parameter sets within one frame. It includes a time slot. As desired, the spatial frame 502 may include other information. The meaning and use of the information included in the configuration information 501 and the space frame 502 will be described using FIGS. 6 to 10.

Referring to FIG. 5B, the bitstream of the audio signal includes configuration information 504, a downmix signal 505, and a space frame 506. In this case, one frame 507 includes a downmix signal 505 and a spatial frame 506, which may be repeated in the bitstream.

Referring to FIG. 5C, the bitstream of the audio signal includes a downmix signal 508, configuration information 509, and a space frame 510. In this case, one frame 511 includes configuration information 509 and a space frame 510, and the frame 511 may be repeated in the bitstream. When the configuration information 509 is inserted into each frame 511, the audio signal can be reproduced at any position by the reproduction device.

Although FIG. 5C illustrates that the configuration information 509 is inserted into the bitstream every frame 511, the configuration information 509 is inserted into the bitstream for each of a plurality of frames that are periodically or aperiodically repeated. It will be obvious.

6A and 6B are diagrams showing a relationship between a parameter set, a time slot, and parameter bands according to an embodiment of the present invention. The parameter set refers to one or more spatial parameters applied to one time slot. Spatial parameters may include spatial information such as CLD, ICC, CPC, and the like. A time slot means a time interval of an audio signal to which spatial parameters can be applied. One spatial frame may include one or more time slots.

Referring to FIG. 6A, a plurality of parameter sets 1, P may be used in a spatial frame, and each parameter set may include one or more data fields 1, Q-1. One parameter set may be applied to the entire frequency range of the audio signal, and each spatial parameter in this parameter set may be applied to one or more positions of the corresponding frequency band. For example, if the parameter set includes 20 spatial parameters, the entire frequency band of the audio signal may be divided into 20 regions (hereinafter referred to as parameter bands), and the 20 spatial parameters of the parameter set are 20 Can be applied to two parameter bands. The parameters can be applied to parameter bands as desired. For example, spatial parameters may be intensively applied to the low frequency parameter band and sparsely applied to the high frequency parameter band.

Referring to FIG. 6B, a time / frequency graph is shown that illustrates the relationship between parameter sets and time slots. In the example shown, three parameter sets (parameter set 1, parameter set 2, parameter set 3) are applied to the set arranged in 12 time slots in one spatial frame. In this case, the entire frequency range of the audio signal is divided into nine parameter bands. Thus, the horizontal axis represents the number of time slots and the vertical axis represents the number of parameter bands. Each of the three parameter sets applies to a particular time slot. For example, the first parameter set (parameter set 1) applies to time slot # 1, the second parameter set (parameter set 2) applies to time slot # 5, and the third parameter set (parameter set 3) applies to time slot # 1. Applies to 9. By interpolating and / or copying these parameter sets in time slots, these parameter sets can also be applied to the remaining time slots. In general, the number of parameter sets may be equal to or less than the number of time slots and the number of parameter bands may be equal to or less than the number of frequency bands of the audio signal. By encoding spatial information for some time-frequency regions of the audio signal instead of the entire time-frequency region of the audio signal, it is possible to reduce the amount of spatial information sent from the encoder to the decoder. This data reduction is possible because, according to known principles of audio coding recognition, spatial information in the time-frequency domain is usually sufficient for human auditory recognition.

An important feature of the disclosed embodiment is the encoding and decoding of the time slot positions to which the parameter sets apply, using a fixed number of bits or a variable number of bits. The number of parameter bands may also be represented by a fixed number of bits or by a variable number of bits. As other information used for spatial audio coding, but not limited thereto, a variable coding scheme may be applied to information including information related to the time domain, the spatial domain, and / or the frequency domain (eg, output from a filter bank). Applied to multiple frequency subbands).

7A illustrates syntax representing configuration information of spatial information according to an embodiment of the present invention. This configuration information includes a plurality of fields 701 through 718 to which a plurality of bits can be assigned.

The bsSamplingFrequencyIndex field 701 indicates a sampling frequency obtained from the sampling process of the audio signal. To indicate the sampling frequency, four bits are allocated to the bsSamplingFrequencyIndex field 701. If the value of the bsSamplingFrequencyIndex field 701 is 15, that is, binary 1111, a bsSamplingFrequency field 702 is added to indicate the sampling frequency. In this case, 24 bits are allocated to the bsSamplingFrequency field 702.

The bsFrameLength field 703 indicates the total number of time slots (hereinafter, referred to as numSlots) in one spatial frame, and a relationship of numSlots = bsFrameLength + 1 may exist between numSlots and bsFrameLength field 703.

The bsFreqRes field 704 indicates the total number of parameter bands that extend the entire frequency domain of the audio signal. The bsFreqRes field 704 will be described in FIG. 7B.

The bsTreeConfg field 705 represents information for a tree structure including a plurality of channel conversion modules as described with reference to FIG. 4. The information for constructing the tree includes information such as the type of the channel conversion module, the number of channel conversion modules, the type of spatial information used in the channel conversion module, the number of input / output channels of the audio signal, and the like.

The tree configuration may be any one of 5-1-5 configuration, 5-2-5 configuration, 7-2-7 configuration, 7-5-7 configuration, and the like, depending on the type of channel conversion module or the number of channels. . The 5-2-5 configuration of the tree configuration is shown in FIG.

The bsQuantMode field 706 represents quantization mode information of spatial information.

The bsOneIcc field 707 indicates whether one ICC parameter sub-set is used for all OTT boxes. In this case, the parameter sub-set means a parameter set applied to a specific time slot and a specific channel conversion module.

The bsArbitraryDownmix field 708 indicates whether any downmix gain is present.

The bsFixedGainSur field 709 represents a gain applied to surround channels such as LS (left surround) and RS (right surround).

bsFixedGainLFE represents a gain applied to the LFE channel.

bsFixedGainDM represents the gain applied to the downmix signal.

The bsMatrixMode field 712 indicates whether a matrix is generated from the encoder that is compatible with the stereo downmix signal.

The bsTempShapeConfig field 713 indicates an operation mode of a temporary form (eg, Tempora Envelope Shaping (TES) and / or Temporal Shaping (TP)) at the decoder.

The bsDecorrConfig field 714 indicates an operation mode of the decorrelator of the decoder.

Finally, the bs3DaudioMode field 715 indicates whether the downmix signal is encoded into the 3D signal and whether reverse HRTF processing is used.

After the information of each field is determined / extracted by the encoder / decoder, information about the number of parameter bands applied to the channel conversion module is determined / extracted by the encoder / decoder. The number of parameter bands applied to the OTT box is first determined / extracted (716), and then the number of parameter bands applied to the TTT box is determined / extracted (717). The number of parameter bands for the OTT box and / or TTT box will be described in detail below with reference to FIGS. 8A-9B.

If there is an extension frame, spatialExtensionConfig block 718 includes configuration information for the extension frame. The information contained in spatialExtensionConfig block 718 will be described below with reference to FIGS. 10A-10D.

7B is a table illustrating the number of parameter bands of a spatial information signal according to an embodiment of the present invention. numBands represents the number of parameter bands for the entire frequency domain of the audio signal, and bsFreqRes represents index information for the number of parameter bands. For example, the entire frequency region of the audio signal may be divided into the number of parameter bands (for example, 4, 5, 7, 10, 14, 20, 28, etc.) as desired.

In some embodiments, one parameter may be applied to each parameter band. For example, when numBands is 28, the entire frequency region of the audio signal is divided into 28 parameter bands, and 28 parameters may be applied to each of these 28 parameter bands, respectively. In another example, when numBands is 4, the entire frequency region of a given audio signal is divided into four parameter bands, and four parameters may be applied to each of these four parameter bands, respectively. In FIG. 7B, Reserve means that the number of parameter bands for the entire frequency region of the predetermined audio signal is not determined.

It should be noted that the human auditory organ is not sensitive to the number of parameter bands used in the coding scheme. Thus, using fewer parameter bands can provide similar spatial audio effects to the listener as compared to the case where more parameter bands are used.

Unlike numBands, numSlots indicated by the bsFramelength field 703 shown in FIG. 7A may represent all values. However, if the number of samples in one spatial frame is explicitly divided by numSlots, the value of numSlots will be limited. Therefore, if the maximum value of numSlots to be substantially represented is 'b', all values of the bsFrameLength field 703 may be represented by ceil {log ₂ (b)} bit (s). In this case, 'ceil (x)' means a maximum integer greater than or equal to the value 'x'. For example, if one spatial frame includes 72 time slots, ceil {log ₂ (72)} = 7 bits may be allocated to the bsFrameLength field 703 and that of the parameter bands applied to the channel conversion module The number can be determined in numBands.

FIG. 8A illustrates a syntax for representing a number of parameter bands applied to an OTT box as a fixed number of bits according to an embodiment of the present invention. 7A and 8A, 'i' has a value of 'numOttBoxes 1' to '0', and 'numOttBoxes' is the total number of OTT boxes. That is, the value of 'i' represents each OTT box, and the number of parameter bands applied to each OTT box is expressed according to the value of 'i'. When the OTT box has the LFE channel mode, the number of parameter bands (hereinafter referred to as bsOttBands) applied to the LFE channel of the OTT box may be expressed using a fixed number of bits. In the example shown in FIG. 8A, five bits are allocated to the bsOttBands field 801. If the OTT box does not have the LFE channel mode, the total number of parameter bands (numBands) is assigned to the channel of the OTT box.

FIG. 8B illustrates a syntax for representing the number of parameter bands applied to the OTT box by a variable number of bits according to an embodiment of the present invention. FIG. 8B is similar to FIG. 8A but differs from FIG. 8A in that the bsOttBands field 802 shown in FIG. 8B is represented by a variable number of bits. In detail, the bsOttBands field 802 having a value less than or equal to numBands may be represented by a variable number of bits using numBands.

If numBands is greater than or equal to 2 ^ (n-1) and less than 2 ^ (n), the bsOttBands field 802 may be represented by variable n bits.

For example: (a) when numBands is 40, the bsOttBands field 802 is represented by 6 bits; (b) when numBands is 28 or 20, the bsOttBands field 802 is represented by 5 bits; (c) when numBands is 14 or 10, the bsOttBands field 802 is represented by 4 bits; (d) When numBands is 7, 5 or 4, the bsOttBands field 802 is represented by 3 bits.

If numBands is greater than 2 ^ (n-1) and less than or equal to 2 ^ (n), the bsOttBands field 802 may be represented by variable n bits.

For example: (a) when numBands is 40, the bsOttBands field 802 is represented by 6 bits; (b) when numBands is 28 or 20, the bsOttBands field 802 is represented by 5 bits; (c) when numBands is 14 or 10, the bsOttBands field 802 is represented by 4 bits; (d) when numBands is 7 or 5, the bsOttBands field 802 is represented by 3 bits; (e) When numBands is 4, the bsOttBands field 802 is 2 bits.

The bsOttBands field 802 may be represented by a variable number of bits through a function (hereinafter, referred to as a ceil function) that takes numBands as a variable and rounds it to the nearest integer.

Specifically, i) when 0 <bsOttBands numBands or 0 bsOttBands <numBands, the bsOttBands field 802 is represented by the number of bits corresponding to the value of ceil (log ₂ (numBands)), or ii) 0 bsOttBands numBands. In this case, the bsOttBands field 802 may be represented by ceil (log ₂ (numBands + 1)).

When a value less than or equal to numBands (hereinafter referred to as numberBands) is arbitrarily determined, the bsBands field 802 may be represented by a variable number of bits through a rounding function by taking numberBands as a variable.

Specifically, i) if 0 <bsOttBands numberBands or 0 bsOttBands <numberBands, the bsOttBands field 802 is represented by ceil (log ₂ (numberBands)) bits, or ii) the bsOttBands numberBands, bsOttBands field 802 Can be expressed as ceil (log ₂ (numberBands + 1)).

If more than one OTT box is used, the combination of bsOttBands can be represented by the following equation:

Here, bsOttBands _i represents the i-th bsOttBands. For example, suppose there are three OTT boxes and three values (n = 3) for the bsOttBands field 802. In this example, three values (hereinafter, respectively, a1, a2, and a3) of the bsOttBands field 802 applied to three OTT boxes may be represented by 2 bits, respectively. Therefore, a total of six bits are required to represent the a1, a2, and a3 values. However, when a1, a2, and a3 values are represented in groups, 27 cases (= 3 * 3 * 3) may occur, which can be represented by 5 bits, thereby saving 1 bit. When numBands is 3 and the group value represented by 5 bits is 15, the group value may be represented by 15 = 1 x (3 ^ 2) + 2 * (3 ^ 1) + 0 * (3 ^ 0). Accordingly, the decoder may determine the three values a1, a2, and a3 of the bsOttBands field 802 as 1, 2, and 0 from the group value 15 by applying the inverse of Equation 1.

For multiple OTT boxes, the combination of bsOttBands may be represented as one of Equations 2 to 4 (described below) using numberBands. Since the expression of bsOttBands using numberBands is similar to the expression using numBands in Equation 1, the detailed description will be omitted and only the expression will be described below.

FIG. 9A illustrates a syntax for representing the number of parameter bands applied to a TTT box as a fixed number of bits, according to an embodiment of the present invention. 7A and 9A, 'i' has a value of 'numTttBoxes 1' to '0', and 'numTttBoxes' is the total number of TTT boxes. That is, the value of 'i' represents each TTT box. The number of parameter bands applied to each TTT box is expressed according to the value of 'i'. In some embodiments, the TTT box may be divided into a low frequency band range and a high frequency band range, and different processing may be applied to the low frequency band range and the high frequency band range. Other divisions are possible.

The bsTTTDualMode field 901 indicates whether a given TTT box operates in different modes (hereinafter, referred to as dual mode) for the low frequency band range and the high frequency band range. For example, when the value of the bsTTTDualMode field 901 is 0, one mode is used for the entire band range without distinguishing the low frequency band range and the high frequency band range. When the value of the bsTTTDualMode field 901 is 1, different modes are used for the low frequency band range and the high frequency band range, respectively.

The bsTttModeLow field 902 indicates an operation mode of a given TTT box, which may have various operation modes. For example, the TTT box may have a prediction mode using CPC parameters and ICC parameters, an energy based mode using CLD parameters, and the like. If the TTT box has dual mode, additional information on the high frequency band range will be needed.

The bsTttModeHigh field 903 indicates an operating mode in the high frequency band range when the TTT box has a dual mode.

The bsTttBandsLow field 904 indicates the number of parameter bands applied to the TTT box.

The bsTttBandsHigh field 905 has numBands.

If the TTT box has dual mode, the low band range will be greater than 0 and less than bsTttBandsLow, and the high band range will be greater than bsTttBandsLow and less than bsTttBandsHigh.

If the TTT box does not have dual mode, the number of parameter bands applied to the TTT box will be greater than 0 and less than numBands (907).

The bsTttBandsLow field 904 may be represented by a fixed number of bits. For example, as shown in FIG. 9A, five bits may be allocated to represent the bsTttBandsLow field 904.

FIG. 9B illustrates a syntax for representing the number of parameter bands applied to the TTT box as a variable number of bits according to an embodiment of the present invention. FIG. 9B is similar to FIG. 9A, but in FIG. 9B the bsTttBandsLow field 907 is represented by a variable number of bits and in FIG. 9A the bsTttBandsLow field 904 is represented by a fixed number of bits. 9B is different. Specifically, since the bsTttBandsLow field 907 has a value less than or equal to numBands, the bsTttBandsLow field 907 may be represented by a variable number of bits using numBands.

In detail, when numBands is greater than or equal to 2 ^ (n-1) and less than 2 ^ (n), the bsTttBandsLow field 907 may be represented by n bits.

For example: (i) when numBands is 40, the bsTttBandsLow field 907 is represented by 6 bits; (ii) when numBands is 28 or 20, the bsTttBandsLow field 907 is represented by 5 bits; (iii) when numBands is 14 or 10, the bsTttBandsLow field 907 is represented by 4 bits; (iv) When numBands is 7, 5 or 4, the bsTttBandsLow field 907 is represented by 3 bits.

If numBands is greater than 2 ^ (n-1) and less than or equal to 2 ^ (n), the bsTttBandsLow field 907 may be represented by variable n bits.

For example: (i) when numBands is 40, the bsTttBandsLow field 907 is represented by 6 bits; (ii) when numBands is 28 or 20, the bsTttBandsLow field 907 is represented by 5 bits; (iii) when numBands is 14 or 10, the bsTttBandsLow field 907 is represented by 4 bits; (iv) when numBands is 7 or 5, the bsTttBandsLow field 907 is represented by 3 bits; (v) When numBands is 4, the bsOttBands field 802 is 2 bits.

The bsTttBandsLow field 907 may be represented by a number of bits determined by taking numBands as a variable and rounding to the nearest integer.

For example: i) If 0 <bsOttBandsLow numBands or 0 bsOttBandsLow <numBands, the bsTttBandsLow field 907 is represented by the number of bits corresponding to the value of ceil (log ₂ (numBands)), or ii) 0 bsOttBandsLow numBands In this case, the bsTttBandsLow field 907 may be represented by ceil (log ₂ (numBands + 1)).

When a value less than or equal to numBands, that is, numberBands, is arbitrarily determined, the bsTttBandsLow field 907 may be represented by a variable number of bits using numberBands.

Specifically, i) when 0 <bsOttBandsLow numberBands or 0 bsOttBandsLow <numberBands, the bsTttBandsLow field 907 is represented by the number of bits corresponding to the value of ceil (log ₂ (numberBands)), or ii) 0 bsOttBandsLow numberBands In this case, the bsTttBandsLow field 907 may be represented by a number of bits corresponding to the value of ceil (log ₂ (numberBands + 1)).

If multiple OTT boxes are used, the combination of bsOttBandsLow can be represented by Equation 5 below:

Here, bsTttBandsLow _i represents the i-th bsTttBandsLow. Since the meaning of Equation 5 is the same as that of Equation 1, detailed description of Equation 5 is omitted.

For multiple OTT boxes, the combination of bsOttBandsLow can be represented as one of equations (6) to (8) using numberBands. Since the meanings of Equations 6 to 8 are the same as the meanings of Equations 2 to 4, detailed descriptions of Equations 6 to 8 are omitted.

The number of parameter bands applied to the channel conversion module (eg, OTT box and / or TTT box, etc.) may be expressed as a split value of numBands. In this case, the above-mentioned split value uses the result of dividing numBands by half of numBands or a specific value.

Once the number of parameter bands applied to the OTT and / or TT box is determined, parameter sets that can be applied to each OTT box and / or each TTT box are determined within the range of the number of parameter bands. Each parameter set may be applied to each OTT box and / or each TTT box in units of time slots. That is, one parameter set may be applied to one time slot.

As mentioned in the foregoing description, one spatial frame may include a plurality of time slots. When the space frame is a fixed frame type, parameter sets may be applied to the plurality of time slots at equal intervals. When the spatial frame is a variable frame type, position information of a time slot to which a parameter is applied is required. This will be described later with reference to FIGS. 13A-13C.

FIG. 10A illustrates syntax representing spatial extension configuration information for a spatial extension frame according to an embodiment of the present invention. FIG. The spatial extension configuration information includes a bsSacExtType field 1001, a bsSacExtLen field 1002, a bsSacExtLenAdd field 1003, a bsSacExtLenAddAdd field 1004, and a bsFillBits field 1007. Other fields are also possible.

The bsSacExtType field 1001 indicates a data type of a spatial extension frame. For example, the spatial extension frame may be filled with zeros, residual signal data, any downmix residual dual signal data, or any tree data.

The bsSacExtLen field 1002 indicates the number of bytes of spatial extension configuration information.

The bsSacExtLenAdd field 1003 indicates the number of additional bytes of the space extension configuration information when the number of bytes of the space extension configuration information is 15 or more, for example.

The bsSacLenAddAdd field 1004 indicates the number of additional bytes of the space extension configuration information when the number of bytes of the space extension configuration information is 270 or more, for example.

After each field is determined / extracted in the encoder / decoder, configuration information for the data type included in the spatial extension frame is determined (1005).

As mentioned in the foregoing description, the spatial extension frame may include residual signal data, arbitrary downmix residual signal data, tree configuration data, and the like.

Subsequently, the number of unused bits in the length of the spatial extension configuration information is calculated (1006).

The bsFillBits field 1007 indicates the number of bits of data that may be overlooked to fill unused bits.

10B and 10C illustrate syntaxes indicating spatial extension information for a residual signal when a residual signal is included in a spatial extension frame according to an embodiment of the present invention.

Referring to FIG. 10B, a bsResidualSamplingFrequencyIndex field 1008 indicates a sampling frequency of a residual signal.

The bsResidualFramesPerSpatialFrame field 1009 indicates the number of residual frames per spatial frame. For example, one, two, three or four residual frames may be included in one spatial frame.

The ResidualConfig field 1010 indicates the number of parameter bands for the residual signal applied to each OTT and / or TTT box.

Referring to FIG. 10C, the bsResidualPresent field 1011 indicates whether a residual signal is applied to each OTT and / or TTT box.

The bsResidualBands field 1012 indicates the number of parameter bands of the residual signal present in each OTT and / or TTT box when the residual signal exists in each OTT and / or TTT box. The number of parameter bands of the residual signal may be represented by a fixed number of bits or a variable number of bits. When the number of parameter bands is represented by a fixed number of bits, the residual signal may have a value less than or equal to the total number of parameter bands of the audio signal. Therefore, the number of bits necessary to represent the number of all parameter bands (eg, 5 bits in FIG. 10C, etc.) can be allocated.

FIG. 10D illustrates a syntax for representing the number of parameter bands of the residual signal into a variable number of bits, according to an embodiment of the present invention. The beResidualBands field 1014 may be represented by a variable number of bits using numBands.

If numBands is greater than or equal to 2 ^ (n-1) and less than 2 ^ (n), the beResidualBands field 1014 may be represented by n bits.

For example: (i) when numBands is 40, the beResidualBands field 1014 is represented by 6 bits; (ii) when numBands is 28 or 20, the beResidualBands field 1014 is represented by 5 bits; (iii) when numBands is 14 or 10, the beResidualBands field 1014 is represented by 4 bits; (iv) When numBands is 7, 5 or 4, the beResidualBands field 1014 is represented by 3 bits.

If numBands is greater than 2 ^ (n-1) and less than or equal to 2 ^ (n), the beResidualBands field 1014 may be represented by variable n bits.

For example: (i) when numBands is 40, the beResidualBands field 1014 is represented by 6 bits; (ii) when numBands is 28 or 20, the beResidualBands field 1014 is represented by 5 bits; (iii) when numBands is 14 or 10, the beResidualBands field 1014 is represented by 4 bits; (iv) when numBands is 7 or 5, the beResidualBands field 1014 is represented by 3 bits; (v) When numBands is 4, the beResidualBands field 1014 is 2 bits.

In addition, the beResidualBands field 1014 may be represented by the number of bits determined by a rounding function determined by taking numBands as a variable and rounding to the nearest integer.

Specifically, i) when 0 <beResidualBands numBands or 0 beResidualBands <numBands, the beResidualBands field 1014 is represented by ceil (log ₂ (numBands)) bits, or ii) when 0 beResidualBands numBands, a beResidualBands field 1014). May be represented by ceil (log ₂ (numBands + 1)) bits.

In some embodiments, the beResidualBands field 1014 may be represented using a numberBands that is less than or equal to numBands.

Specifically, i) when 0 <beResidualBands numberBands or 0 beResidualBands <numberBands, the beResidualBands field 1014 is represented by ceil (log ₂ (numberBands)) bits, or ii) when 0 beResidualBands numberBands, the beResidualBands field 1014 is a ceil. It can be expressed as a value of (log ₂ (numberBands + 1)).

When there are a plurality of residual signals N, a combination of beResidualBands may be represented by Equation 9 below:

In this case, bsResidualBands _i represents the i-th bsResidualBands. Since the meaning of Equation 9 is the same as that of Equation 1, a detailed description of Equation 9 is omitted.

If there are multiple residual signals, the combination of bsResidualBands may be represented as one of Equations 10 to 12 using numberBands. Representing bsResidualBands using numberBands is similar to Equation 2 to Equation 4, and thus a detailed description thereof will be omitted.

The number of parameter bands of the residual signal may be expressed as a division value of numBands. In this case, the above-mentioned split value uses the result of dividing numBands by half of numBands or a specific value.

The residual signal may be included in the bitstream of the audio signal along with the downmix signal and the spatial information signal, which may be transmitted to the decoder. The decoder may extract the downmix signal, the spatial information signal, and the residual signal from the bitstream.

Subsequently, the downmix signal is upmixed using spatial information. On the other hand, the residual signal is applied to the downmix signal in the upmix process. Specifically, the downmix signal is upmixed in a plurality of channel conversion modules using spatial information. In this process, the residual signal is applied to the channel conversion module. As mentioned in the foregoing description, the channel conversion module has a plurality of parameter bands, and a set of parameters is applied to the channel conversion module in units of time slots. When the residual signal is applied to the channel conversion module, the residual signal will be needed to update the inter-channel correlation information of the audio signal to which the residual signal is applied. The updated channel correlation information is used for the upmixing process.

11A is a block diagram illustrating a decoder for non-guided coding according to an embodiment of the present invention. Non-guided coding means that the spatial information is not included in the bitstream of the audio signal.

In some embodiments, the decoder includes an analysis filter bank 1102, an analysis unit 1104, a spatial synthesis unit 1006, and a synthesis filter bank 1108. Although the downmix signal of the stereo signal type is shown in FIG. 11A, other types of downmix signals may be used.

In operation, the decoder receives the downmix signal 1101, and the analysis filter bank 1102 converts the received downmix signal 1101 into a frequency domain signal 1103. The analyzer 1104 generates spatial information from the converted downmix signal 1103. The analyzer 1104 may perform the processing in units of slots to generate spatial information 1105 for each of a plurality of slots. In this case, the slot includes a time slot.

Spatial information can be generated in two steps. First, downmix parameters are generated from the downmix signal. Second, the downmix parameters are converted into spatial information such as spatial parameters. In some embodiments, the downmix parameter may be generated through matrix operations of the downmix signal.

The spatial synthesizer 1106 synthesizes the generated spatial information 1105 and the downmix signal 1103 to generate a multi-channel audio signal 1107. The generated multi-channel audio signal 1107 passes through a synthesis filter bank 1108 and is converted into a time domain audio signal 1109.

Spatial information may be generated at predetermined slot locations. The distance between these locations may be the same (ie equidistant). For example, spatial information may be generated every four slots. In addition, spatial information may be generated at variable slot positions. In this case, position information for generating spatial information may be extracted from the bitstream. The location information may be represented by a variable number of bits. The position may be expressed as an absolute value and a difference value from previous slot position information.

When using non-guided coding, the number of parameter bands (hereinafter, referred to as bsNumguidedBlindBands) for each channel of the audio signal may be represented by a fixed number of bits. bsNumguidedBlindBands may be represented by a variable number of bits using numBands. For example, if numBands is greater than or equal to 2 ^ (n-1) and less than 2 ^ (n), bsNumguidedBlindBands may be represented by variable n bits.

Specifically, (a) when numBands is 40, bsNumguidedBlindBands is represented by 6 bits; (b) when numBands is 28 or 20, bsNumguidedBlindBands is represented by 5 bits; (c) when numBands is 14 or 10, bsNumguidedBlindBands is represented by 4 bits; (d) When numBands is 7, 5 or 4, bsNumguidedBlindBands is represented by 3 bits.

If numBands is greater than 2 ^ (n-1) and less than or equal to 2 ^ (n), bsNumguidedBlindBands may be represented by variable n bits.

For example: (a) when numBands is 40, bsNumguidedBlindBands is represented by 6 bits; (b) when numBands is 28 or 20, bsNumguidedBlindBands is represented by 5 bits; (c) when numBands is 14 or 10, bsNumguidedBlindBands is represented by 4 bits; (d) when numBands is 7 or 5, bsNumguidedBlindBands is represented by 3 bits; (e) When numBands is 4, bsNumguidedBlindBands is 2 bits.

In addition, bsNumguidedBlindBands may be represented by a variable number of bits using a rounding function that takes numBands as a variable.

For example, i) 0 <bsNumguidedBlindBands numBands or 0 bsNumguidedBlindBands <If the numBands, bsNumguidedBlindBands is ceil (log ₂ (numBands)) representing bits or, ii) 0 bsNumguidedBlindBands case numBands is, bsNumguidedBlindBands is ceil (log ₂ (numBands + 1 )) Can be expressed in bits.

When a value less than or equal to numBands, that is, numberBands is arbitrarily determined, bsNumguidedBlindBands may be expressed as follows.

Specifically, i) 0 <If bsNumguidedBlindBands numberBands or 0 bsNumguidedBlindBands <numberBands, bsNumguidedBlindBands is represented by a ceil (log ₂ (numberBands)) bits, ii) 0 bsNumguidedBlindBands case numberBands of, bsNumguidedBlindBands is ceil (log ₂ (numberBands + 1)).

When multiple channels N are used, the combination of bsNumguidedBlindBands can be represented by the following equation (13):

Here, bsNumguidedBlindBands _i represents the i-th bsNumguidedBlindBands. Since the meaning of Equation 13 is the same as that of Equation 1, a detailed description of Equation 13 is omitted.

If there are multiple channels, bsNumguidedBlindBands may be represented as one of Equations 14 to 16 using numberBands. Since the expression of bsNumguidedBlindBands using numberBands is the same as the expression of Equations 2 to 4, detailed descriptions of Equations 14 to 16 are omitted.

11B is a diagram illustrating a method of expressing the number of parameter band bands as a group according to an embodiment of the present invention. The number of parameter bands includes information on the number of parameter bands applied to the channel conversion module, the number of parameter bands applied to the residual signal, and the number of parameter bands for each channel of the audio signal when non-guided coding is used. Contains count information. When there are a plurality of number of parameter bands, a plurality of number information (for example, bsOttBands, bsTttBands, bsResidualBands and / or bsNumguidedBlindBands, etc.) may be represented as at least one group.

Referring to FIG. 11B, when there are (kN + L) number of parameter bands and Q bits are required to express the number information of each parameter band, the number information of a plurality of parameter bands may be represented by the following group. have. In this case, 'k' and 'N' are any integer other than '0' and 'L' is any integer satisfying 0 L <N.

The grouping method includes generating k groups by tying N number information of parameter bands and generating final group by tying L number information of last parameter bands. K groups may be represented by M bits, and the final group may be represented by p bits. In this case, the M bits are preferably smaller than the N * Q bits used when representing each of the number information of the parameter bands without grouping them. The P bits are preferably less than or equal to the L * Q bits used when representing each of the number information of the parameter bands without grouping them.

For example, assume that two pieces of information of the number of parameter bands are b1 and b2, respectively. If b1 and b2 each have five values, three bits are required to represent each of b1 and b2. In this case, although three bits can represent eight values, substantially five values are required. Thus, each of b1 and b2 has three spares. However, when b1 and b2 are grouped together and represented as a group, 5 bits will be used instead of 6 bits (= 3 bits + 3 bits). Specifically, since all combinations of b1 and b2 have 25 types (= 5 * 5), groups of b1 and b2 can be represented by 5 bits. Since five bits can represent 32 values, seven spares are generated for the grouping representation. However, in the case where b1 and b2 are represented by grouping, the excess is smaller than that when b1 and b2 are represented by 3 bits, respectively. The method of expressing the number information of the parameter bands as a group may be implemented in various ways as follows.

When the number information of the number of parameter bands each has 40 values, k groups are generated using 2, 3, 4, 5 or 6 as N. These k groups can be represented as 11 bits, 16 bits, 22 bits, 27 bits, and 32 bits, respectively. Alternatively, these k groups are represented by combining each case.

When the number information of the number of parameter bands each has 28 values, k groups are generated using 6 as N, and k may be represented by 29 bits.

When the number information of the number of parameter bands each has 20 values, k groups are generated using 2, 3, 4, 5, 6 or 7 as N. These k groups can be represented as 9 bits, 13 bits, 18 bits, 22 bits, 26 bits and 31 bits, respectively. Alternatively, these k groups can be represented in combination with each case.

When the number information of the number of parameter bands each has 14 values, k groups are generated using 6 as N. These k groups can be represented by 23 bits.

When the number information of the number of parameter bands each has ten values, k groups are generated using 2, 3, 4, 5, 6, 7, 8 or 9 as N. These k groups can be represented by 7, 10, 14, 17, 20, 24, 27 and 30 bits, respectively. Alternatively, these k groups can be represented in combination with each case.

When the number information of the number of parameter bands each has seven values, k groups are generated using 6, 7, 8, 9, 10 or 11 as N. These k groups can be represented by 17, 20, 23, 26, 29 and 31 bits, respectively. Alternatively, these k groups can be represented in combination with each case.

If the number information of the number of parameter bands each has five values, k groups are generated using 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, or 13 as N. Can be. These k groups can be represented by 5, 7, 10, 12, 13, 17, 19, 21, 24, 26, 28 and 31 bits, respectively. Alternatively, these k groups can be represented in combination with each case.

In addition, the number information of the number of parameter bands may be configured to be represented as the above-described group, or may be configured to be continuously expressed with each of the number information of the parameter bands as an independent bit sequence.

12 illustrates syntax representing configuration information of a spatial frame according to an embodiment of the present invention. The spatial frame includes a FraminInfo block 1201, a bsIndependencyFlag block 1201, an Ottdata block 1203, a Tttdata block 1204, a SmgData block 1205, and a TempShapeData block 1206.

FramingInfo block 1201 includes information on the number of parameter sets and information on the time slot to which each parameter is applied. FramingInfo block 1201 is described in detail in FIG. 13A.

The bsIndependencyFlag field 1202 indicates whether the current frame can be decoded without knowledge of the previous frame.

OttData block 1203 contains all spatial parameter information for all OTT boxes.

TttData block 1204 contains all spatial parameter information for all TTT boxes.

SmgData block 1205 includes information about temporal smoothing applied to dequantized spatial parameters.

TempShapeData block 1206 includes information about temporal envelope shaping that is applied to the uncorrelated signal.

FIG. 13A illustrates syntax representing time slot position information to which a parameter set is applied according to an embodiment of the present invention. The bsFramingType field 1301 indicates whether a spatial frame of an audio signal is a fixed frame type or a variable frame type. The fixed frame refers to a frame in which a parameter set is applied to a preset time slot. For example, a parameter set is applied to time slots preset at equal intervals. The variable frame refers to a frame for separately receiving position information of a time slot to which a parameter set is applied.

The bsNumParamSets field 1302 indicates the number of parameter sets in one spatial frame (hereinafter referred to as numParamSets), and a relationship of numParamSets = bsNumparaSets + 1 is established between numParamSets and bsNumParamSets.

For example, if three bits are allocated to the bsNumParasets field 1302 of FIG. 13A, up to eight parameter sets may be provided in one spatial frame. Since there is no limit on the number of bits allocated, more parameter sets can be provided within the spatial frame.

When the spatial frame is a fixed frame type, the position information of the time slot to which the parameter set is applied may be determined according to a preset rule, and additional position information of the time slot to which the parameter set is applied is unnecessary. However, when the spatial frame is a variable frame type, position information to which a parameter set is applied is necessary.

The bsParamSlot field 1303 indicates location information of a time slot to which a parameter set is applied. The bsParamSlot field 1303 may be represented by a variable number of Vito using the number of time slots in one spatial frame, that is, numSlots. Specifically, when numSlots is in a range of 2 ^ (n-1) or more and less than 2 ^ (n), the bsParamSlot field 1303 may be represented by n bits.

For example: (i) if numSlots is in the range between 64 and 127, the bsParamSlot field 1303 may be represented by 7 bits; (ii) when numSlots is in the range between 32 and 63, the bsParamSlot field 1303 may be represented by 6 bits; (iii) when numSlots is in the range between 16 and 31, the bsParamSlot field 1303 is represented by 5 bits; (iv) when numSlots is in the range between 8 and 15, the bsParamSlot field 1303 may be represented by 4 bits; (v) when numSlots is in the range between 4 and 7, the bsParamSlot field 1303 may be represented by 3 bits; (vi) when numSlots is in the range between 2 and 3, the bsParamSlot field 1303 may be represented by 2 bits; (vii) when numSlots is 1, the bsParamSlot field 1303 is represented by 1 bit; (viii) When numSlots is 0, the bsParamSlot field 1303 may be represented by 0 bits. Similarly, when numSlots is in the range between 64 and 127, the bsParamSlot field 1303 may be represented by 7 bits.

If there are multiple parameter sets N , bsParamSlot may be represented according to equation (17).

In this case, bsParamSlots _i represents time slots to which the I-th parameter set is applied. For example, assume that numSlots is 3 and the bsParamSlot field 1303 can have 10 values. In this case, three pieces of information about the bsParamSlot field 1303 (hereinafter, referred to as c1, c2, and c3, respectively) are required. Since four bits are required to represent each of c1, c2, and c3, a total of 12 bits are required. When c1, c2, and c3 are grouped together and represented, 1,000 (= 10 * 10 * 10) cases may occur, which may be represented by 10 bits, thereby saving 2 bits. When numSlots is 3 and the group value represented by 5 bits is 31, the group value may be represented by 31 = 1 x (3 ^ 2) + 5 * (3 ^ 1) + 7 * (3 ^ 0). Accordingly, the decoder device may determine c1, c2, and c3 as 1, 5, and 7 by applying the inverse of equation (17).

FIG. 13B illustrates a syntax representing position information of a time slot to which a parameter set is applied as an absolute value and a difference value according to an embodiment of the present invention. When the spatial frame is a variable frame type, the bsParamSlot field 1301 in FIG. 13A may be expressed as an absolute value and a difference value using the fact that the bsParamSlot information is monotonically increased.

For example: (i) the position of the time slot to which the first parameter set applies is generated as an absolute value, i.e. bsParamSlot [0]; (ii) The position of the time slot to which the second or more parameter set is applied may be generated as an absolute value, that is, a difference value or difference value 1 between bsParamSlot [ps] and bsParamSlot [ps-1] (hereinafter, bsDiffParamSlot [ps). ]. In this case, ps means a parameter set.

The bsParamSlot [0] field 1304 may be represented by a number of bits calculated using numSlots and numParamSets (hereinafter, referred to as nBitsParamSlot (0)).

The bsDiffParamSlot [ps] field 1305 may be represented by a number of bits calculated using numSlots, numParamSetst, and a position of a time slot to which the previous parameter set is applied (hereinafter referred to as nBitParamSlot (ps)).

Specifically, in order to represent bsParamSlot [ps] with the minimum number of bits, the number of bits representing bsParamSlot [ps] may be determined according to the following rules: (i) The plurality of bsParamSlot [ps] is increased in ascending order. (BsParamSlot [ps]> bsParamSlot [ps-1]); (ii) the maximum value of bsParamSlot [0] is numSlots NumParamSets; (iii) When 0 <ps <numParamSets, bsParamSlot [ps] has only a value between bsParamSlot [ps-1] + 1 and numSlots numParamSets + ps.

For example, if numSlots is 10 and numParamSets is 3, since bsParamSlot [ps] increases in ascending order, the maximum value of bsParamSlot [0] is 10 3 = 7. In other words, bsParamSlot [0] should be selected from a value between 0 and 7. This is because when bsParamSlot [0] has a value of 7 or more, the number of time slots for the remaining parameter sets is insufficient.

If bsParamSlot [0] is 5, the time slot position bsParamSlot [1] for the second parameter set should be selected from values between 5 + 1 = 6 and 10 3 + 1 = 8.

If bsParamSlot [1] is 7, bsParamSlot [2] may be 8 or 9. If bsParamSlot [1] is 8, bsParamSlot [2] may be 9.

Thus, bsParamSlot [ps] may be represented as a variable number of bits using the above feature instead of being represented as a fixed bit.

In configuring bsParamSlot [ps] in the bitstream, when ps is 0, bsParamSlot [0] may be represented as an absolute value by the number of bits corresponding to nBitsParamSlot (0). When ps is greater than 0, bsParamSlot [ps] may be represented as an absolute value by the number corresponding to nBitsParaSlot (ps). When reading bsParamSlot [ps] configured as described above from the bitstream, the length of the bitstream, that is, nBitsParamSlot [ps] for each data may be expressed using Equation 18.

Specifically, nBitsParamSlot [ps] may be expressed as nBitsParamSlot [0] = f _b (numSlots numParaSets + 1). If 0 <ps <numParamSets, nBitsParamSlot [ps] may be expressed as nBitsParamSlot [ps] = f _b (numSlots numParaSets + ps bsParamSlot [ps-1]). nBitsParamSlot [ps] may be determined using Equation 19, which is extended from Equation 18 to 7 bits.

An example of the function fb (x) is described below. If numSlots is 15 and numParamSets is 3, the above function can obtain nBitsParamSlot [1] = fb (15 3 + 1 7) = 3 bits. In this case, the bsDiffParamSlot [1] field 1305 may be represented by 3 bits.

When the value represented by 3 bits is 3, bsParamSlot [1] becomes 7 + 3 = 10. Thus, nBitsParamSlot [2] = f _b (15 3 + 2 10) = 2 bits. In this case, the bsDiffParamSlot [1] field 1305 may be represented by 2 bits. If the number of residual time slots is equal to the number of residual parameter sets, 0 bits will be allocated to the bsDiffParamSlot [ps] field. In other words, no additional information is needed to represent the position of the time slot to which the parameter set is applied.

Therefore, the number of bits for bsParamSlot [ps] may be variably determined. The number of bits for bsParamSlot [ps] can be read from the bitstream using the function f _b (x) at the decoder. In some embodiments, the function f _b (x) may comprise a function ceil (log ₂ (x)).

When reading information about bsParamSlot [ps] expressed in absolute and difference values from the bitstream at the decoder, first bsParamSlot [0] is read from the bitstream and then bsDiffParamSlot [ps] for 0 <ps <numParamSets. Will be read. And, bsParamSlot [ps] for the interval 0 ps <numParamSets can be obtained using bsParamSlot [0] and bsDiffParamSlot [ps]. For example, as illustrated in FIG. 13B, bsParamSlot [ps] may be obtained by adding bsDiffParamSlot [ps] +1 to bsParamSlot [ps-1].

FIG. 13C is a diagram illustrating syntax indicating position information of a time slot to which a parameter set is applied according to an embodiment of the present invention. When there are a plurality of parameter sets, the plurality of bsParamSlots 1307 for the plurality of parameter sets may be represented as at least one group.

If the number of bsParamSlots 1307 is (kN + L) and Q bits are required to represent each of the bsParamSlots 1307, bsParamSlots 1307 may be represented as the following group. In this case, 'k' and 'N' are any integer other than '0' and 'L' is any integer satisfying 0 L <N.

The grouping method includes grouping N bsParamSlots 1307 to generate k groups, and grouping the last L bsParamSlots 1307 to generate a final group. The k groups can be represented by M bits, and the final group can be represented by p bits. In this case, the M bits are preferably smaller than the N * Q bits used when representing each of the bsParamSlots 1307 without grouping them. The P bits are preferably less than or equal to the L * Q bits used when representing each of the bsParamSlots 1307 without grouping them.

For example, assume that a pair of bsParamSlots 1307 for two parameter sets is d1 and d2, respectively. If d1 and d2 each have five values, three bits are required to represent d1 and d2, respectively. In this case, although three bits can represent eight values, substantially five values are required. Thus, each of d1 and d2 has three spares. However, when d1 and d2 are grouped together and represented as a group, 5 bits will be used instead of 6 bits (= 3 bits + 3 bits). Specifically, since all combinations of d1 and d2 have 25 types (= 5 * 5), the group of d1 and d2 can be represented by 5 bits. Since five bits can represent 32 values, seven spares are generated for the grouping representation. However, in the case where d1 and d2 are represented by grouping, the excess is smaller than that in the case where d1 and d2 are represented by 3 bits, respectively.

When constructing a group, the data for the group may be constructed using the difference between pairs of bsParamSlot [0] for the initial value and bsParamSlot [ps] for the second or more value.

When configuring a group, if the number of parameter sets is 1, the bits may be directly assigned without grouping. If the number of parameter sets is 2 or more, the bits may be allocated after the grouping is completed.

14 is a flowchart of an encoding method according to an embodiment of the present invention. The method of encoding an audio signal and the operation of an encoder according to the invention are described below.

First, the total number (numSlots) of time slots and the total number (numBands) of parameter bands of an audio signal are determined in one spatial frame (S1401).

Then, the number of parameters and / or the residual signal applied to the channel conversion module is determined (S1402).

When the OTT box has the LFE channel mode, the number of parameter bands applied to the OTT box is determined separately.

If the OTT box does not have the LFE channel mode, numBands is used as the number of parameter bands applied to the OTT box.

Subsequently, the type of the space frame is determined. In this case, the spatial frame is classified into a fixed frame type and a variable frame type.

When the spatial frame is a variable frame type (S1403), the number of parameter sets used in one spatial frame is determined (S1406). In this case, the parameter set may be applied to the channel conversion module in units of time slots.

Subsequently, the position of the time slot to which the parameter set is applied is determined (S1407). In this case, the position of the time slot to which the parameter set is applied may be expressed as an absolute value and a difference value. For example, the position of the time slot to which the first parameter set is applied may be represented as an absolute value, and the position of the time slot to which the second or more parameter set is applied may be represented as a difference value from the position of the previous time slot. In this case, the position of the time slot to which the parameter set is applied may be represented as a variable number of bits.

In detail, the position of the time slot to which the first parameter set is applied may be expressed by the number of bits calculated using the total number of time slots and the total number of parameter sets. The position of the time slot to which the second or more parameter set is applied may be expressed by the total number of time slots, the total number of parameter sets, and the number of bits calculated using the position of the time slot to which the previous parameter set is applied. .

When the space frame is a fixed frame type, the number of parameter sets used in one space frame is determined (S1404). In this case, the position of the time slot to which the parameter set is applied is determined using a preset rule. For example, the position of the time slot to which the parameter set is applied may be determined to have an equal interval from the position of the time slot to which the previous parameter set is applied (S1405).

Subsequently, the downmixing unit and the space generating unit determine the total number of time slots determined before, the total number of parameter bands, the total number of parameter bands to be applied to the channel converting unit, and the parameter set in one spatial frame. A downmix signal and spatial information are generated using the total number and time slot position information to which the parameter set is applied (S1408).

Finally, the multiplexer generates a bitstream including the downmix signal and spatial information (S1409) and transmits the generated bitstream to the decoder (S1409).

15 is a flowchart of a decoding method according to an embodiment of the present invention. The operation of the decoding method and the decoder of the audio signal according to the present invention will be described below.

First, the decoder receives a bitstream of an audio signal (S1501). The demultiplexer separates the downmix signal and the spatial information signal from the received bitstream (S1502). Subsequently, the spatial information signal decoding unit extracts information on the total number of time slots, the total number of parameter bands, and the number of parameter bands applied to the channel conversion module from the configuration information of the spatial information signal. (S1503).

When the spatial frame is a variable frame type (S1504), the number of parameter sets in one spatial frame and the position information of the time slot to which the parameter set is applied are extracted to the spatial frame (S1505). The location information of the time slot may be represented by a fixed number of bits or a variable number of bits. In this case, the position information of the time slot to which the first parameter set is applied may be represented as an absolute value, and the position information of the time slot to which the second or more parameter set is applied may be represented as a difference value. The actual position information of the time slot to which the second or more parameter set is applied may be obtained by adding a difference value to the position information of the time slot to which the previous parameter set is applied.

Finally, the downmix signal is converted into a multi-channel audio signal using the extracted information (S1506).

Embodiments disclosed herein provide several advantages over conventional audio coding schemes.

First, in coding of a multi-channel audio signal, the amount of data to be transmitted can be reduced by representing the position of a time slot to which a parameter set is applied by a variable number of bits.

Second, by expressing the position of the time slot to which the first parameter set is applied as an absolute value and the position of the time slot to which the second or more parameter set is applied as a difference value, the amount of transmission data can be reduced.

Third, the amount of transmission data can be reduced by expressing the number of parameter bands applied to the OTT box and / or the TTT box to a fixed number of bits or a variable number of bits. In this case, the position of the time slot to which the parameter set is applied can be expressed using the principle described above, where the parameter set is within the number range of parameter bands.

FIG. 16 is a block diagram illustrating an example of an apparatus structure 1600 for implementing the audio encoder / decoder described with reference to FIGS. 1 to 15. The device structure 1600 can decode personal computers, server computers, consumer electronics, mobile phones, PDAs, electronic tablets, television systems, television set-top boxes, game consoles, media players, music players, navigation systems, and audio signals. Applicable to various devices, including, but not limited to, any other device that is present. Some of these devices can use a combination of hardware and software to implement the modified architecture.

The architecture 1600 may include one or more processors 1602 (eg, PowerPC, Intel Pentium 4, etc.), one or more display devices 1604 (eg, CRTs, LCDs, etc.), audio subsystems ( 1606 (e.g., audio hardware / software), one or more network interfaces 1608 (e.g., Ethernet, FireWire, USB, etc.), and input device 1610 (e.g., keyboard, mouse, etc.) And one or more computer readable media 1612 (eg, RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components may communicate and exchange data over one or more buses 1614 (eg, EISA, PCI, PCI Express, etc.).

The term computer readable media refers to any medium that provides instructions to the processor 1602 for execution. Non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory, etc.) ) And a transmission medium. Transmission media include, but are not limited to, coaxial cables, copper wiring, and fiber optics. The transmission medium may also take the form of acoustic waves, light waves or radio frequency waveforms.

Computer-readable medium 1612 includes an operating system 1616 (eg, Mac OS, Windows, Linux, etc.), network communication module 1618, audio codec 1620 and one or more applications 1622. It includes more.

Operating system 1616 can be multi-user, multiprocessing, multitasking, multithreading, real time, and the like. Operating system 1616 recognizes input from input device 1610; Send output to display device 1604 and subsystem 1606; Maintain files and directories on computer readable medium 1612 (eg, memory or recording device, etc.); Control peripheral devices (eg, disk drives, printers, etc.); Perform basic tasks of managing traffic on one or more buses 1614, but are not limited to such.

The network communication module 1618 includes various components for establishing and maintaining a network connection (eg, software for implementing a communication protocol such as TCP / IP, HTTP, Ethernet, etc.). The network communication module 1618 may include a browser that enables an operator of the device structure 1600 to find information (eg, audio content, etc.) and search the network (eg, the Internet, etc.).

The audio codec 1620 is responsible for implementing all or part of the encoding and / or decoding process described with reference to FIGS. 1-15. In some embodiments, the audio codec includes encoding and / or decoding an audio signal in accordance with the invention described herein in conjunction with hardware (eg, processor 1602, audio subsystem 1606, etc.). To process the audio signal.

Application 1622 may include any software related to the audio content, the audio content being encoded in a media player, music player (eg, MP3 player, etc.), mobile phone application, PDA, television system, set top box, etc. And / or decoded, but are not limited to such. In one embodiment, the audio codec may be used by an application service provider to provide encoding / decoding services over a network (eg, the Internet, etc.).

In the foregoing description, for purposes of explanation, several specific details are set forth in order to provide a thorough understanding of the present invention. However, one skilled in the art will understand that the invention may be practiced without these specific details. Further, structures and devices are shown in block diagram form in order to avoid obscuring the present invention.

In particular, those skilled in the art will appreciate that other structural and graphical environments may be used and that the present invention may be implemented using other graphical tools and products than those described above. In particular, the client / server approach is merely one example of a structure that provides the dashboard functionality of the present invention; Those skilled in the art will appreciate that other than client / server approaches may be used.

Some of the details have been presented in terms of algorithms and symbolic representations of operations on data bits in computer memory. These algorithmic descriptions and representations are the means by which those skilled in the data processing arts most effectively convey the substance of their work to others skilled in the art. In general, and herein, an algorithm is recognized as a sequence of steps leading to a desired result. This step requires manipulation of physical quantities. Generally, but not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared or otherwise manipulated. Primarily for normal use, it is convenient to refer to these signals as bits, values, elements, symbols, characters, predicates, numbers, and the like.

However, these terms and similar terms will all be associated with appropriate physical quantities and are merely convenient naming applied to these quantities. Unless specifically stated otherwise, as will be apparent from the discussion, in this specification, the use of terms such as processing or computing or calculating or determining or displaying is intended to be a computer system. A computer system or similar electronic computing device that manipulates data expressed as physical (electrical) quantities in registers and memories of a computer and converts it to data represented as physical quantities in a computer system memory or register or other information storage, transmission, or display device. It refers to the action and processing of.

The invention also relates to an apparatus for performing such an operation. The apparatus may be specially configured according to the required purpose or may include a general purpose computer which is selectively activated or reconfigured by a computer program stored in the computer. Such computer programs include, but are not limited to, floppy disks, optical disks, CD-ROMs, disks of any type, including magnetic optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards or optical cards, Or may be stored in a computer storage medium including any type of medium suitable for storing electronic instructions and coupled to the computer system.

The algorithms and modules disclosed herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in the programs according to the techniques disclosed herein, and it may be convenient to configure more specialized apparatus to perform the method steps. The structures required for these various systems will appear in the description below. Various programming languages may be used to implement the teachings of the invention described herein. In addition, as will be apparent to those skilled in the art, the modules, features, attributes, methodologies, and other aspects of the present invention may be implemented in software, hardware, firmware, or any combination of the three. Of course, when the components of the present invention are implemented as software, these components are independent programs, as part of larger programs, as a plurality of individual programs, as statically or dynamically linked libraries, as kernel loaded modules. It may be implemented as a device driver, or in any manner currently known or known to those skilled in the art of computer programming. In addition, the present invention is not limited to being implemented in any particular operating system or environment.

It will be apparent to those skilled in the art that various modifications and changes can be made to the embodiments disclosed herein without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover all such modifications and variations of the disclosed embodiments provided they come within the scope of the appended claims and their equivalents.

Claims (20)

Generating a parameter set of an audio signal corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Inserting the parameter set and the corresponding first or second information into a bitstream representing the audio signal, The first information or the second information is a method of encoding an audio signal, characterized in that represented by a variable number of bits. Receiving a bitstream including a parameter set corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Decoding the audio signal based on the parameter set and the first or second information, And the first information or the second information is represented by a variable number of bits. The method of claim 2, The first information is location information of a time domain or time slot to which the parameter set is applied, and the second information is location information of a frequency domain, subband or parameter band to which the parameter set is applied. Decoding method. The method of claim 3, And the first information or the second information indicates a fixed position or a variable position. The method of claim 4, wherein The first information is represented by a variable number of bits determined using the number information of the time domain or the time slot, and the second information is determined using the number information of the frequency domain, the sub band, or the parameter band. The decoding method of an audio signal, characterized in that represented by a variable number of bits. The method of claim 5, And when the number information is 2 ^ (n-1) or more and less than 2 ^ (n), the variable number of bits is determined as n bits. The method of claim 5, And when the number information is greater than 2 ^ (n-1) and less than or equal to 2 ^ (n), the variable number of bits is determined to be n bits. The method of claim 2, And applying said parameter set to a channel conversion module for generating n channels from m channels. The method of claim 8, The range of the second information is a low frequency range or a high frequency range decoding method of an audio signal. The method of claim 2, Wherein the first information is expressed as a sum of a previous value and a difference value, the previous value indicates position information to which the first parameter set is applied, and the difference value indicates position information to which the second parameter set is applied. Method of decoding the signal. The method of claim 2, If the second information is position information of a parameter band, a plurality of the second information corresponding to the parameter set is represented by a combination using the following equation:

Here, numSlot ⁱ and bsTttBands _i represent the number of parameter bands and the i th second information, respectively. The method of claim 2, And a plurality of the first information or the second information is divided into groups, and a plurality of the first information or the second information is divided into groups and represented for each group. The method of claim 12, When the number of the first information or the second information is (kN + L), the group is generated by grouping N first information or second information together and represented by M bits, and the last group is L first information. Or generating second bits by tying together the second information to be represented by P bits. Means for generating a parameter set of an audio signal corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Means for inserting the parameter set and the corresponding first or second information into a bitstream representing the audio signal, And the first information or the second information is represented by a variable number of bits. Means for receiving a bitstream comprising a parameter set corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Means for decoding the audio signal based on the parameter set and the first or second information, The first information or the second information is a decoding apparatus of an audio signal, characterized in that represented by a variable number of bits. A first field comprising first information or second information; A second field corresponding to the first information or the second information or corresponding to the range of the first information or the second information, And the first information or the second information is represented by a variable number of bits. The method of claim 2, Determining mode information of two or more ranges of the first information or the second information; Using the mode information for decoding the audio information. A computer readable medium having stored thereon instructions which, when executed by a processor, cause the processor to: The operation is, Receiving a bitstream including a parameter set corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Decoding the audio signal based on the parameter set and the first or second information, And the first information or the second information is represented by a variable number of bits. A processor; And A computer readable medium having stored thereon instructions which, when executed by the processor, cause the processor to: The operation is, Receiving a bitstream including a parameter set corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Decoding the audio signal based on the parameter set and the first or second information, And wherein the first information or the second information is represented by a variable number of bits. Means for receiving a bitstream comprising a parameter set corresponding to the first information or the second information of the audio signal or corresponding to the range of the first information or the second information; And Means for decoding the audio signal based on the parameter set and the first or second information, And wherein the first information or the second information is represented by a variable number of bits.

Images