TWI732403B - Decoding audio bitstreams with enhanced spectral band replication metadata in at least one fill element - Google Patents

Abstract

Embodiments relate to an audio processing unit that includes a buffer, bitstream payload deformatter, and a decoding subsystem. The buffer stores at least one block of an encoded audio bitstream. The block includes a fill element that begins with an identifier followed by fill data. The fill data includes a first flag identifying whether a base form of spectral band replication processing or an enhanced form of spectral band replication processing is to be performed on audio content of the at least one block of the encoded audio bitstream, and if the first flag identifies the enhanced form of spectral band replication processing, a second flag identifying whether signal adaptive frequency domain oversampling is enabled or disabled. A corresponding method for decoding an encoded audio bitstream is also provided.

Description

Decoding audio bitstream with enhanced spectrum band replication metadata in at least one padding element

The present invention relates to audio signal processing. Some embodiments are related to encoding and decoding audio bitstreams (for example, bitstreams in MPEG-4 AAC format), which include meta-data for controlling Enhanced Spectrum Band Copy (eSBR). Other embodiments are related to decoding such a bitstream by a conventional decoder that is not configured to perform eSBR processing and ignoring such metadata, or for decoding by generating eSBR control data in response to the bitstream, does not include such Audio bitstream of metadata.

A typical audio bitstream includes both audio data indicating one or more channels of audio content (for example, encoded audio data), and metadata indicating audio data or at least one feature of the audio content. One well-known format for generating encoded audio bitstreams is the MPEG-4 Advanced Audio Coding (AAC) format, which is described in the MPEG standard ISO/IEC 14496-3:2009. In the MPEG-4 standard, AAC stands for "Advanced Audio Coding "Advanced audio coding" and HE-AAC stand for "high-efficiency advanced audio coding".

The MPEG-4 AAC standard defines several audio profiles, which determine which components and coding tools exist in compatible encoders and decoders. Three of these audio specifications are (1) AAC specifications, (2) HE-AAC specifications, and (3) HE-AAC v2 specifications. The AAC specification includes AAC low-complexity (or "AAC-LC") object types. The AAC-LC object system, with some adjustments, corresponds to the MPEG-2 AAC low-complexity specification, and does not include the spectral band copy ("SBR") object type nor the parametric stereo ("PS") object Type. The HE-AAC specification is a super-collection of the AAC specification and also includes SBR object types. The HE-AAC v2 specification is a super set of the HE-AAC specification, and also includes PS object types.

The SBR object type includes a spectrum band copy tool, which is an important coding tool that significantly improves the compression efficiency of perceptual audio codecs. SBR reconstructs the high frequency components of the audio signal at the receiver (for example, in the decoder). Therefore, the encoder only needs to encode and transmit low frequency components, allowing higher audio quality at low data rates. SBR replicates the sequence of harmonics based on the available limited bandwidth signal and the control data obtained from the encoder. The sequence of harmonics is truncated in advance to reduce the data rate. The ratio of tonal components and noise-like components is maintained by adaptive inverse filtering and optional addition of noise and sinusoidal signals. In the MPEG-4 AAC standard, the SBR tool performs spectral patching, in which a number of adjacent quadrature mirror filter (QMF) subbands are copied from the low-band part of the audio signal transmission to the audio signal In the high frequency band, the audio signal is generated in the decoder.

Spectrum patching is not ideal for certain audio types, such as music content with relatively low crossover frequencies. Therefore, techniques for improving spectrum band replication are needed.

The first type of embodiment relates to an audio processing unit, which includes a memory, a bitstream payload deformatter, and a decoding subsystem. The memory is configured to store at least one block of the encoded audio bitstream (for example, MPEG-4 AAC bitstream). The bitstream payload deformatter is configured to demultiplex the encoded audio block. The decoding subsystem is configured to decode the audio content of the encoded audio block. The encoded audio block includes a padding element, which has an identifier indicating the start of the padding element, and padding data included after the identifier. The filling data includes a first flag that identifies whether to perform a basic form of spectral band copy processing or an enhanced form of spectral band copy processing on the audio content of the at least one block of the encoded audio bit stream, and if the first One flag identifies the enhanced form of the spectrum band copy processing, and the second flag identifies whether to enable or disable the signal adaptive frequency domain oversampling.

The second type of embodiment relates to a method for decoding an encoded audio bitstream. The method includes receiving at least one block of an encoded audio bitstream, demultiplexing at least some parts of the at least one block of the encoded audio bitstream, and decoding the encoded audio bitstream At least some part of the at least one block. The at least one block of the encoded audio bit stream includes a padding element, which has a start indicating the padding element The identifier of and the padding information included after the identifier. The filling data includes a first flag that identifies whether to perform a basic form of spectral band copy processing or an enhanced form of spectral band copy processing on the audio content of the at least one block of the encoded audio bit stream, and if the first One flag identifies the enhanced form of the spectrum band copy processing, and the second flag identifies whether to enable or disable the signal adaptive frequency domain oversampling.

Other types of embodiments relate to encoding and transcoding an audio bitstream, the audio bitstream including metadata identifying whether enhanced spectral band replication (eSBR) processing will be performed.

1: encoder

2: Delivery subsystem

3: decoder

4: Post-processing unit

100: encoder

105: encoder

106: Metadata Generator Level

107: Filler/Formatter level

109: Buffer memory

200: decoder

201: Buffer memory

202: Decoding Subsystem

203: eSBR processing level

204: Control bit generator stage

205: Bitstream payload formatter (parser)

210: Audio Processing Unit (APU)

213: SBR processing level

215: Bitstream payload deformatter (parser)

300: post processor

301: Buffer memory (buffer)

400: eSBR decoder

401: eSBR control data generation subsystem

500: Audio Processing Unit (APU)

Figure 1 is a block diagram of an embodiment of a system configured to perform an embodiment of the method of the present invention.

Figure 2 is a block diagram of an encoder, which is an embodiment of the audio processing unit of the present invention.

Figure 3 is a block diagram of a system including a decoder, which is an embodiment of the audio processing unit of the present invention, and optionally has a post-processor coupled to it.

Figure 4 is a block diagram of a decoder, which is an embodiment of the audio processing unit of the present invention.

Fig. 5 is a block diagram of a decoder, which is another embodiment of the audio processing unit of the present invention.

Fig. 6 is a block diagram of another embodiment of the audio processing unit of the present invention.

Fig. 7 is a diagram of the blocks of the MPEG-4 AAC bitstream, including the segments into which the bitstream is divided.

Symbols and terminology

Throughout the present disclosure, including in the scope of the patent application, the description of performing operations on ("on") a signal or data (for example, filtering, scaling, converting, or applying gain to the signal or data) is used in a broad sense to indicate Perform operations on the signal or data directly, or perform operations on a processed version of the signal or data (for example, on a version of the signal that has been preliminarily filtered or preprocessed before the operation is performed).

Throughout the present disclosure, including in the scope of the patent application, the expression "audio processing unit" is used in a broad sense to indicate a system, device, or device configured to process audio data. Examples of audio processing units include, but are not limited to, encoders (e.g., transcoders), decoders, codecs, pre-processing systems, post-processing systems, and bit stream processing systems (sometimes referred to as bit stream processing systems). Metastream processing tools). Almost all consumer electronics, such as mobile phones, TVs, laptops, and tablet computers, include an audio processing unit.

Throughout the present disclosure, including in the scope of patent application, the term "coupled" or "coupled" is used in a broad sense to refer to one of direct or indirect connection. Therefore, if the first device is coupled to the second device, the connection may be through a direct connection, or through an indirect connection through other devices or connections. In addition, components integrated into or integrated with other components are also coupled to each other.

Detailed description of embodiments of the present invention

The MPEG-4 AAC standard considers that the encoded MPEG-4 AAC bitstream includes metadata, which instructs the decoder to use (if any) to decode the various types of SBR processing of the audio content of the bitstream, and/or It controls at least one feature or parameter of at least one SBR tool that controls the SBR process and/or indicates that it will be used to decode the audio content of the bitstream. In this article, the expression "SBR metadata" is used to represent this type of metadata described or mentioned in the MPEG-4 AAC standard.

The top level of the MPEG-4 AAC bitstream is a sequence of data blocks ("raw_data_block" elements). Each data block contains audio data (typically used for a time period of 1024 or 960 samples) and related information and/or other The data section of the data (referred to as "block" in this article). In this article, the term "block" is used to refer to the section of the MPEG-4 AAC bitstream, which contains audio data (and corresponding metadata and optional data) that determine or indicate one (but no more than one) "raw_data_block" element. Other relevant information).

Each block of the MPEG-4 AAC bitstream may include some syntax elements (each syntax element is also embodied as a section of data in the bitstream). Seven types of this syntax element are defined in the MPEG-4 AAC standard. Each syntax element is identified by a different value of the data element "id_syn_ele". Examples of syntax elements include "single_channel_element()", "channel_pair_element()", and "fill_element()". The mono element is a container that includes audio data of a mono audio channel (mono audio signal). The binaural element includes audio data of two audio channels (ie, stereo audio signal).

The filling element is a container of information, and the information includes an identifier (for example, the value of the above-mentioned element "id_syn_ele") followed by data (which is called "filling data"). The stuffing element has traditionally been used to adjust the instantaneous bit rate of the bit stream to be transmitted on a fixed rate channel. By adding an appropriate amount of padding data to each block, a fixed data rate can be achieved.

According to an embodiment of the present invention, the padding data may include one or more extension payloads, which extend the type of data (for example, metadata) that can be transmitted in the bit stream. A decoder that receives a bit stream with padding data containing a new data type can optionally be used by a device (for example, a decoder) receiving the bit stream to expand the functionality of the device. Therefore, as understood by those skilled in the art, the filling element is a special type of data structure and is different from the data structure typically used to transmit audio data (for example, an audio load containing channel data).

In some embodiments of the present invention, the identifier used to identify the padding element may be composed of a three-bit most significant bit transmitted prior to the unsigned integer ("uimsbf"), which has a value of 0x6. In a block, multiple instances of the same type of syntax element (for example, multiple padding elements) may appear.

Another standard for encoding audio bitstreams is the MPEG Unified Speech and Audio Coding (USAC) standard (ISO/IEC 23003-3: 2012). The MPEG USAC standard describes the encoding and decoding of audio content using spectral band copy processing (including the SBR process described in the MPEG-4 AAC standard, and also other enhanced forms of spectral band copy processing). This process is described in the MPEG-4 AAC standard An expansion of the collection of SBR tools and an enhanced version of the spectrum band replication tool (sometimes referred to as "enhanced SBR tool" or "eSBR tool" in this article). Therefore, eSBR (as defined in the USAC standard) is an improvement of SBR (as defined in the MPEG-4 AAC standard).

In this article, the expression "enhanced SBR processing" (or "eSBR processing") is used to indicate the use of at least one eSBR tool not described or mentioned in the MPEG-4 AAC standard (for example, described or mentioned in the MPEG USAC standard). And at least one eSBR tool) to copy the spectrum band. Examples of such eSBR tools are harmonic transposition, QMF-patching additional pre-processing or "pre-flattening", and Temporal Envelope Shaping between sub-band samples. ) Or "inter-TES".

The bitstream generated in accordance with the MPEG USAC standard (sometimes referred to herein as "USAC bitstream") includes the encoded audio content, and typically includes the audio content that will be used by the decoder to decode the USAC bitstream At least one of the various types of metadata for the spectral band copy processing and/or at least one SBR tool and/or eSBR tool that controls the spectral band copy processing and/or represents the audio content that will be used to decode the USAC bitstream Metadata of features or parameters.

In this article, the expression "enhanced SBR metadata" (or "eSBR metadata") is used to indicate the frequency spectrum of the audio content that will be used by the decoder to decode the encoded audio bitstream (for example, USAC bitstream) Various types of metadata with copy processing, and/or metadata that controls copy processing of this spectrum band, and/or instructions will be used to decode the audio content, but not Metadata of at least one feature or parameter of at least one SBR tool and/or eSBR tool described or mentioned in the MPEG-4 AAC standard. An example of eSBR metadata is metadata that is described or mentioned in the MPEG USAC standard but not described or mentioned in the MPEG-4 AAC standard (indicating spectrum band copy processing, or used to control spectrum band copy processing) . Therefore, eSBR metadata refers to the metadata of non-SBR metadata in this article, and SBR metadata refers to the metadata of non-eSBR metadata in this article.

The USAC bitstream may include both SBR metadata and eSBR metadata. More specifically, the USAC bitstream may include eSBR metadata that controls the eSBR processing performance of the decoder, and SBR metadata that controls the SBR processing performance of the decoder. According to an exemplary embodiment of the present invention, eSBR metadata (for example, eSBR specific configuration data) is contained in (according to the present invention) MPEG-4 AAC bitstream (for example, in the sbr_extension() container at the end of the SBR payload) .

During the decoding of an encoded bit stream using the eSBR tool set (including at least one eSBR tool), the decoder performs eSBR processing, and reproduces the high frequency band of the audio signal according to the copy of the harmonic sequence that was truncated during the encoding process . Such eSBR processing typically adjusts the spectral envelope of the generated high frequency band, and applies inverse filtering, and adds noise and sinusoidal components to re-establish the spectral characteristics of the original audio signal.

According to an exemplary embodiment of the present invention, one or more metadata sections of an encoded audio bitstream (for example, MPEG-4 AAC bitstream) include eSBR metadata (for example, including eSBR metadata). Few control bits), the encoded audio bit stream also contains the encoded audio data in In other sections (audio data section). Typically, at least one such metadata section of each section of the bitstream is (or includes) a padding element (including an identifier indicating the start of the padding element), and the eSBR metadata is included in In the filled element, after the identifier.

FIG. 1 is a block diagram of an exemplary audio processing chain (audio data processing system) in which one or more components of the system can be configured according to embodiments of the present invention. The system includes the following elements, which are coupled together as shown in the figure: an encoder 1, a transmission subsystem 2, a decoder 3, and a post-processing unit 4. In variants of the system shown, one or more of these components are omitted, or additional audio data processing units are included.

In some embodiments, the encoder 1 (which optionally includes a preprocessing unit) is configured to accept PCM (time domain) samples containing audio content as input, and output an encoded audio bitstream representing the audio content ( It has a format that conforms to the MPEG-4 AAC standard). Bitstream data representing audio content is sometimes referred to herein as "audio data" or "encoded audio data." If the encoder is configured according to the exemplary embodiment of the present invention, the audio bitstream output from the encoder includes eSBR metadata (and typically other metadata) and audio data.

One or more encoded audio bit streams output from the encoder 1 can be asserted to the encoded audio delivery subsystem 2. The sub-system 2 is configured to store and/or transmit each encoded bit stream output from the encoder 1. The encoded bit stream output from Encoder 1 can be stored by Subsystem 2 (for example, in the form of DVD or Blu-ray Disc), or transmitted by Subsystem 2 (which can realize a transmission link or network), or by Subsystem 2 Store and transmission.

The decoder 3 is configured to decode the encoded MPEG-4 AAC audio bitstream (generated by the encoder 1), which is received via the subsystem 2. In some embodiments, the decoder 3 is configured to extract eSBR metadata from each block of the bitstream, and decode the bitstream (including performing eSBR processing by using the extracted eSBR metadata) to generate Decoded audio data (for example, a stream of decoded PCM audio samples). In some embodiments, the decoder 3 is configured to extract SBR metadata from the bitstream (but ignore the eSBR metadata contained in the bitstream), and decode the bitstream (including by using the extracted SBR metadata is used to perform SBR processing to generate decoded audio data (for example, a stream of decoded PCM audio samples). Typically, the decoder 3 includes a buffer that stores (e.g., in a non-transitory manner) sections of the encoded audio bitstream received from the subsystem 2.

The post-processing unit 4 of FIG. 1 is configured to receive a stream of decoded audio data (for example, decoded PCM audio samples) from the decoder 3 and perform post-processing on it. The post-processing unit 4 can also be configured to present post-processed audio content (or decoded audio received from the decoder 3) for playback by one or more speakers.

Figure 2 is a block diagram of an encoder (100), which is an embodiment of the audio processing unit of the present invention. Any component or element of the encoder 100 can be implemented as one or more processing processes and/or one or more circuits (for example, ASICs, FPGAs, or other products) in hardware, software, or a combination of hardware and software. Body circuit). Encoder 100 includes encoder 105, filler/format The converter stage 107, the metadata generator stage 106, and the buffer memory 109 are connected as shown. Typically, the encoder 100 also includes other processing elements (not shown). The encoder 100 is configured to convert the input audio bitstream into an encoded output MPEG-4 AAC bitstream.

The metadata generator stage 106 is coupled and configured to generate (and/or through the filler/formatter stage 107) metadata (including eSBR metadata and SBR metadata), which will be filled/formatted The stage 107 is included in the encoded bit stream to be output from the encoder 100.

The encoder 105 is coupled and configured to encode the input audio data (for example, by performing compression on it), and the generated encoded audio judgment is prompted to the filler/formatter stage 107 for inclusion in the to-be It is output from the encoded bit stream of stage 107.

The filler/formatter stage 107 is configured to multiplex the encoded audio from the encoder 105 and the metadata (including eSBR metadata and SBR metadata) from the metadata generator stage 106 to generate output from The encoded bit stream of the filler/formatter stage 107 preferably has a format as specified by one of the embodiments of the present invention.

The buffer memory 109 is configured to store (for example, in a non-transitory manner) at least one block of the encoded audio bit stream output from the filler/formatter stage 107, and the encoded audio bit A sequence of blocks of the metastream will then be judged and prompted from the buffer memory 109 as the output from the encoder 100 to the delivery system.

Fig. 3 is a block diagram of a system including a decoder (200), which is an embodiment of the audio processing unit of the present invention, and optionally also has a post-processor (300) coupled to it. Any component or element of the decoder 200 and the post-processor 300 can be implemented as one or more processing processes and/or one or more circuits (for example, ASICs, FPGAs, or other integrated circuits). The decoder 200 includes a buffer memory 201, a bit stream load formatter (parser) 205, an audio decoding subsystem 202 (sometimes called a "core" decoding stage or a "core" decoding subsystem), and eSBR The processing stage 203 and the control bit generator stage 204 are connected as shown in the figure. Typically, the decoder 200 also includes other processing elements (not shown).

The buffer memory (buffer) 201 stores (for example, in a non-transitory manner) at least one block of the encoded MPEG-4 AAC audio bitstream received by the decoder 200. In the operation of the decoder 200, a sequence of blocks of the bit stream is judged and prompted by the buffer 201 to the formatter 205.

In a variation of the FIG. 3 embodiment (or the FIG. 4 embodiment to be described), the APU that is not a decoder (for example, the APU 500 of FIG. 6) includes a buffer memory (for example, a buffer equivalent to the buffer 201). Memory), which stores (e.g., in a non-transitory manner) the same form of encoded audio bitstream (e.g., MPEG-4 AAC audio bitstream) received by the buffer 201 of FIG. 3 or FIG. 4 ) At least one block (ie, an encoded audio bitstream including eSBR metadata).

Referring again to FIG. 3, the deformatter 205 is coupled and configured to demultiplex each block of the bit stream to extract SBR metadata (including quantized envelope data) and eSBR metadata (and usually also Including other yuan Data) for at least the eSBR metadata and the SBR metadata judgment and prompting to the eSBR processing stage 203, and typically other extracted metadata judgment and prompting to the decoding subsystem 202 (and optionally also for judging and prompting To control bit generator 204). The formatter 205 is also coupled and configured to extract audio data from each block of the bit stream, and judge and prompt the extracted audio data to the decoding subsystem (decoding stage) 202.

The system of FIG. 3 optionally also includes a post-processor 300. The post-processor 300 includes a buffer memory (buffer) 301 and other processing elements (not shown), which includes at least one processing element coupled to the buffer 301. The buffer 301 stores (for example, in a non-transitory manner) at least one block (or frame) of the decoded audio data received from the decoder 200 by the post-processor 300. The processing elements of the post-processor 300 are coupled and configured to receive and adaptively process a sequence of blocks (or blocks) of the decoded audio output from the buffer 301, which uses the self-decoding subsystem 202 (and/or The meta-data output by the formatter 205) and/or the control bits output from the stage 204 of the decoder 200.

The audio decoding subsystem 202 of the decoder 200 is configured to decode the audio data extracted by the parser 205 (this kind of decoding may be referred to as a "core" decoding operation) to generate decoded audio data, and determine to prompt the decoded audio data The audio data to the eSBR processing stage 203. Decoding is performed in the frequency domain and usually includes inverse quantization followed by spectral processing. Typically, the final stage of processing in the subsystem 202 applies a frequency domain to time domain conversion to the decoded frequency domain audio data so that the output of the subsystem is time domain decoded data. Stage 203 is configured to apply the SBR element to the decoded audio data Data and eSBR metadata (extracted by the parser 205) indicated by the SBR tool and eSBR tool (that is, SBR and eSBR metadata are used to perform SBR and eSBR processing on the output of the decoding subsystem 202) to generate fully decoded audio Data, which is output from the decoder 200 (for example, to the post-processor 300). Typically, the decoder 200 includes a memory (accessible by the subsystem 202 and the stage 203) that stores the deformatted audio data and metadata output from the deformatter 205, and the stage 203 is configured Complete access to audio data and metadata (including SBR metadata and eSBR metadata) required during SBR and eSBR processing. The SBR processing and eSBR processing in the stage 203 can be regarded as post-processing of the output of the core decoding subsystem 202. Optionally, the decoder 200 also includes a final upmixing (upmixing) subsystem (which can use the parametric stereo ("PS") tool defined in the MPEG-4 AAC standard, using the de-formatter 205 PS metadata and/or control bits generated in subsystem 204), which are coupled and configured to perform upmixing on the output of stage 203 to generate fully decoded and upmixed audio, which is self-decoding器200 output. Alternatively, the post-processor 300 is configured to perform upmixing on the output of the decoder 200 (for example, using the PS metadata extracted by the formatter 205 and/or the control bits generated in the subsystem 204) .

In response to the metadata extracted by the formatter 205, the control bit generator 204 can generate control data, and the control data can be used in the decoder 200 (for example, in the final upmix subsystem) and/ Or it may be judged as the output of the decoder 200 (for example, to the post-processor 300 for use in post-processing). Respond to the metadata extracted from the output bitstream (with And optionally also in response to control data), stage 204 can generate (and determine prompts to post-processor 300) control bits that indicate that the decoded audio data output from eSBR processing stage 203 should undergo a specific type of post-processing . In some embodiments, the decoder 200 is configured to judge the metadata extracted by the reformatter 205 from the input bit stream to the post-processor 300, and the post-processor 300 is configured to use the metadata, The decoded audio data output from the decoder 200 performs post-processing.

Figure 4 is a block diagram of an audio processing unit ("APU") (210), which is another embodiment of the audio processing unit of the present invention. The APU 210 is a conventional decoder, which is not configured to perform eSBR processing. Any component or element of the APU 210 can be implemented as one or more processing processes and/or one or more circuits (for example, ASICs, FPGAs, or other integrated circuits) in hardware, software, or a combination of hardware and software Circuit). The APU 210 includes a buffer memory 201, a bitstream load formatter (parser) 215, an audio decoding subsystem 202 (sometimes called a "core" decoding stage or a "core" decoding subsystem), and SBR The processing stage 213 is connected as shown. Typically, the APU 210 also includes other processing elements (not shown).

The elements 201 and 202 of the APU 210 are the same as the same numbered elements of the decoder 200 (of FIG. 3), and their above description will not be repeated. In the operation of the APU 210, a sequence of blocks of the encoded audio bitstream (MPEG-4 AAC bitstream) received by the APU 210 is judged and prompted from the buffer 201 to the formatter 215.

The de-formatter 215 is coupled and configured to demultiplex each block of the bit stream to extract SBR metadata (including quantized envelope data) to And typically other metadata is extracted from it, but the eSBR metadata that may be included in the bitstream according to other embodiments of the present invention is ignored. The formatter 215 is configured to prompt at least the SBR metadata determination to the SBR processing stage 213. The de-formatter 215 is also coupled and configured to extract audio data from each block of the bit stream, and to judge and prompt the extracted audio data to the decoding subsystem (decoding stage) 202.

The audio decoding subsystem 202 of the decoder 200 is configured to decode the audio data extracted by the formatter 215 (this type of decoding may be referred to as a "core" decoding operation) to generate the decoded audio data, and the The decoded audio data is judged and prompted to the SBR processing stage 213. The decoding is performed in the frequency domain. Typically, the final stage of processing in the subsystem 202 applies a frequency domain to time domain conversion to the decoded frequency domain audio data so that the output of the subsystem is time domain decoded data. The stage 213 is configured to apply the SBR tool (but not the eSBR tool) indicated by the SBR metadata (extracted by the formatter 215) to the decoded audio data (ie, use the SBR metadata to the decoding subsystem 202 The output performs SBR processing) to generate fully decoded audio data, which is output from the APU 210 (for example, to the post-processor 300). Typically, the APU 210 includes a memory (accessible by the subsystem 202 and the stage 213) that stores the deformatted audio data and metadata output from the deformatter 215, and the stage 213 is configured as Access audio data and metadata (including SBR metadata) required during SBR processing. The SBR processing in stage 213 can be regarded as a post-processing of the output of the core decoding subsystem 202. Optionally, APU 210 also includes a final upmixing subsystem (which can use the parameters defined in the MPEG-4 AAC standard A stereophonic ("PS") tool, which uses the PS metadata extracted by the formatter 215), which is coupled and configured to perform upmixing on the output of stage 213 to produce a fully decoded, upmixed audio , Which is output from APU 210. Alternatively, a post-processor is configured to perform upmixing on the output of the APU 210 (for example, using the PS metadata extracted by the formatter 215 and/or the control bits generated in the APU 210).

The various embodiments of the encoder 100, the decoder 200, and the APU 210 are configured to perform different embodiments of the method of the present invention.

According to some embodiments, the encoded audio bitstream (e.g., MPEG-4 AAC bitstream) contains eSBR metadata (e.g., contains a small number of control bits that are eSBR metadata), so that a traditional decoder ( It is not configured to analyze eSBR metadata, or is not configured to use any eSBR tool that the eSBR metadata belongs to.) The eSBR metadata can be ignored, but eSBR metadata or any eSBR metadata to which the eSBR metadata belongs is still not used as much as possible. The eSBR tool is used to decode the bit stream, usually without any major loss in the quality of the decoded audio. However, an eSBR decoder configured to analyze the bitstream to identify eSBR metadata and use at least one eSBR tool in response to the eSBR metadata will enjoy the benefits of using at least one such eSBR tool. Therefore, the embodiments of the present invention provide a mechanism for effectively transmitting Enhanced Spectrum Band Copy (eSBR) control data or metadata in a backward compatible manner.

Typically, the eSBR metadata in the bitstream represents one or more of the following eSBR tools (which are described in the MPEG USAC standard, and which may or may not be used by the encoder during the generation of the bitstream) ( example For example, it represents at least one characteristic or parameter of one or more of them):

●Harmonic modulation;

●QMF-repair) additional pre-processing (pre-planarization); and

●Sample time envelope forming or "inter-TES" between sub-bands.

For example, the eSBR metadata included in the bitstream can represent the value of a parameter (described in the MPEG USAC standard and in this disclosure): harmonicSBR[ch], sbrPatchingMode[eh], sbrOversamplingFlag[ch], sbrPitchInBins[ch] , SbrPitchInBins[ch], bs_interTes, bs_temp_shape[ch][env], bs_inter_temp_shape_mode[ch][env], and bs_sbr_preprocessing.

Here, the symbol X[ch], where X is a parameter, indicates that the parameter belongs to the channel ("ch") of the audio content of the encoded bit stream to be decoded. For simplicity, sometimes the expression of [ch] is omitted, and it is assumed that the relevant parameters belong to the channel of the audio content.

In this article, the symbol X[ch][env], where X is a parameter, indicates that the parameter belongs to the SBR envelope ("env") of the channel ("ch") of the audio content of the encoded bitstream to be decoded. ). For simplicity, sometimes the expressions of [env] and [ch] are omitted, and it is assumed that the relevant parameters belong to the channel SBR envelope of the audio content.

As mentioned, the MPEG USAC standard considers that the USAC bitstream includes eSBR metadata, which controls the performance of the eSBR processing performed by the decoder. The eSBR metadata includes the following one-bit metadata parameters: harmonicSBR; bs_intcrTES; and bs_pvc.

The parameter "harmonicSBR" indicates that harmonic repair (harmonic modulation) is used for SBR. Specifically, harmonicSBR=0 means non-harmonic, spectrum repair, as described in section 4.6.18.6.3 of the MPEG-4 AAC standard; and harmonicSBR=1 means harmonic SBR repair (with the form used in eSBR, such as MPEG As described in section 7.5.3 or 7.5.4 of the USAC standard). According to non-eSBR spectrum band duplication (ie, SBR that is not eSBR), harmonic SBR repair is not used. According to this disclosure, spectrum inpainting is called the basic form of spectrum band duplication, and harmonic shifting is called the enhanced form of spectrum band duplication.

The value of the parameter "bs_interTES" indicates that the inter-TES tool of eSBR is used.

The value of the parameter "bs_pvc" indicates the use of eSBR PVC tools.

During the decoding of the encoded bit stream, the performance of the harmonic shift during the eSBR processing stage (for each channel "ch" of the audio content indicated by the bit stream) decoded is determined by the following eSBR metadata Parameter controlled: sbrPatchingMode[ch]; sbrOversamplingFlag[ch]; sbrPitchInBinsFlag[ch]; and sbrPitchInBins[ch].

The value "sbrPatchingMode[ch]" indicates the type of transposer used in eSBR: sbrPatchingMode[ch]=1 indicates non-harmonic patching, as described in section 4.6.18.6.3 of the MPEG-4 AAC standard; sbrPatchingMode [ch]=0 means harmonic SBR repair, as described in section 7.5.3 or 7.5.4 of the MPEG USAC standard.

The value "sbrOversamplingFlag[ch]" means the use of information in eSBR No. adaptive frequency domain oversampling, combined with DFT-based harmonic SBR repair, as described in section 7.5.3 of the MPEG USAC standard. This flag controls the size of the DFT used in the pitch shifter: 1 means to allow signal adaptive frequency domain oversampling, as described in section 7.5.3.1 of the MPEG USAC standard; 0 means to prohibit signal adaptive frequency domain oversampling, As described in section 7.5.3.1 of the MPEG USAC standard.

The value "sbrPitchInBinsFlag[ch]" controls the interpretation of the sbrPitchInBins[ch] parameter: 1 means that the value in sbrPitchInBins[ch] is valid and greater than zero; 0 means that the value of sbrPitchInBins[ch] is set to zero.

The value "sbrPitchInBins[ch]" controls the increase in the cross product term in the SBR harmonic shifter. The value sbrPitchinBins[ch] is an integer value in the range [0,127] and represents the distance measured in the frequency bins of the 1536-line DFT acting on the sampling frequency of the core encoder.

In the case of an MPEG-4 AAC bitstream indicating two SBR channels whose channels are not coupled (instead of a single SBR channel), the bitstream indicates two instances of the above syntax (for harmonic or anharmonic Wave shift), an instance is used for one channel of sbr_channel_pair_element().

The harmonic shifting of the eSBR tool generally improves the quality of the decoded music signal at relatively low crossover frequencies. Harmonic shifting should be implemented in the decoder via DFT-based or QMF-based harmonic shifting. Non-harmonic transposition (ie, traditional spectrum inpainting or copying) usually improves the voice signal. Therefore, it is important to decide which type of transposition is better for encoding specific audio content. The starting point is to select the transposition method based on voice/music detection. Harmonic shift is used for music content, and spectrum patching is used for voice content.

The performance of pre-flattening during eSBR processing is controlled by the value of a one-bit eSBR metadata parameter called "bs_sbr_preprocessing", which means that pre-flattening is performed or not performed based on this single-bit value. When using the SBR QMF patching algorithm (as described in section 4.6.18.6.3 of the MPEG-4 AAC standard), the pre-flattening step can be performed (when indicated by the "bs_sbr_preprocessing" parameter), trying to avoid being input to Discontinuities in the shape of the spectrum envelope of the high-frequency signal of the subsequent envelope adjuster (the envelope adjuster performs other stages of the eSBR processing). Pre-flattening generally improves the operation of subsequent envelope adjuster stages, producing high-band signals that are considered more stable.

During the eSBR processing in the decoder, the performance of the inter-subband sample time envelope shaping ("inter-TES" tool) is determined by the following for each channel ("ch") of the audio content of the USAC bitstream to be decoded Controlled by the eSBR metadata parameters of the SBR envelope ("env"): bs_temp_shape[ch][env]; and bs_inter_temp_shape_mode[ch][env].

The inter-TES tool processes the QMF subband samples after the envelope adjuster. This processing step uses a finer time granularity than the envelope adjuster to shape the time envelope of the higher frequency band. By applying a gain factor to each QMF subband sample in the SBR envelope, inter-TES reshapes the time envelope between the QMF subband samples.

The parameter "bs_temp_shape[ch][env]" is a flag, which is sent and used Inter-TES signal. The parameter "bs_inter_temp_shape_mode[ch][env]" represents (as defined in the MPEG USAC standard) the value of the parameter γ in the inter-TES.

According to some embodiments of the present invention, for the overall bit rate requirement included in the MPEG-4 AAC bitstream, the eSBR metadata representing the eSBR tools (harmonic transposition, pre-flattening, and inter_TES) is expected to be On the order of hundreds of bits per second, because only the differential control data required to perform eSBR processing is transmitted. Traditional decoders can ignore this information because it is included in a backward compatible manner (to be explained later). Therefore, for some reasons, the adverse effect on the bit rate associated with the eSBR metadata is negligible. These reasons include the following:

●The bitrate penalty (caused by the inclusion of the eSBR metadata) is a very small part of the total bit rate, because only the differential control data required to perform eSBR processing is transmitted (not the SBR control data) Hookup);

●SBR-related control information tuning (tuning) usually does not rely on the details of transposition; and

●The inter-TES tool (used during eSBR processing) performs single ended post-processing of the transposed signal.

Therefore, the embodiments of the present invention provide a mechanism for efficiently transmitting enhanced spectrum band replication (eSBR) control data or metadata in a backward compatible manner. Such efficient transmission of eSBR control data reduces the memory requirements of decoders, encoders, and transcoders that adopt the aspect of the present invention, and at the same time has no obvious adverse effect on the bit rate. In addition, it also reduces the This embodiment implements the complexity and processing requirements associated with eSBR, because SBR data only needs to be processed once, not simulcast. This can be if eSBR is treated as a completely independent object in MPEG-4 AAC, rather than backwards. The compatible way is integrated into the MPEG-4 AAC codec.

Next, referring to FIG. 7, the elements of the block ("raw_data_block") of the MPEG-4 AAC bitstream according to some embodiments of the present invention will be described. The MPEG-4 AAC bitstream includes eSBR metadata. Figure 7 is a diagram of a block ("raw_data_block") of the MPEG-4 AAC bitstream, showing some of its blocks.

A block of the MPEG-4 AAC bitstream may include at least one "single_channel_element()" (for example, the mono element shown in Figure 7), and/or at least one "channel_pair_element()" (although it may exist , But not explicitly shown in Figure 7), which includes audio data for audio programs. The block may also include some "fill_elements" (for example, fill element 1 and/or fill element 2 in FIG. 7), which includes information about the program (for example, metadata). Each "single_channel_element()" includes an identifier (for example, "ID1" in FIG. 7), which indicates the start of a mono element, and may include audio data indicating a different channel of a multi-channel audio program. Each "channel_pair_element" includes an identifier (not shown in FIG. 7), which indicates the start of a two-channel element, and may include audio data indicating two channels of the program.

One of the fill_element of the MPEG-4 AAC bitstream (referred to as a fill element in this document) includes an identifier ("ID2" in FIG. 7), which indicates the fill element At the beginning, and the padding data is after the identifier. The identifier ID2 can be composed of a three-bit most significant bit transmitted prior to an unsigned integer ("uimsbf"), which has a value of 0×6. The stuffing data can include an extension_payload() element (sometimes referred to as extended payload in this document), the syntax of which is shown in Table 4.57 of the MPEG-4 AAC standard. There are several types of extended payloads, and they are identified through the "extension_type" parameter, which is a four-bit most significant bit transmitted prior to unsigned integer ("uimsbf").

The padding data (for example, its extended payload) may include a header or an identifier (for example, "Header 1" in FIG. 7), which indicates the section of the padding data of the SBR object (that is, the header initializes a " The "SBR object" type is called sbr_extension_data() in the MPEG-4 AAC standard. For example, the extended payload of Spectrum Band Copy (SBR) is marked as the value '1101' or '1110', which is used in the extension_type field in the header, where the identifier '1101' identifies the extended payload with SBR data, and '1110' 'Identify the expanded load with SBR data and use cyclic redundancy check (CRC) to verify the correctness of the SBR data.

When the header (for example, the extension_type field) initializes an SBR object type, the SBR metadata (sometimes referred to as "spectral band copy data" in this text" and called sbr_data() in the MPEG-4 AAC standard) ) Follows the header, and at least one spectrum band duplication extension element (for example, the "SBR extension element" of fill element 1 in FIG. 7) can follow the SBR metadata. This spectrum band copy extension element (a section of the bit stream) is called the "sbr_extension()" container in the MPEG-4 AAC standard. Spectrum The extension element with copy optionally includes a header (for example, the "SBR extension header" of padding element 1 in FIG. 7).

The MPEG-4 AAC standard considers that a spectrum band replication extension element can include PS (parametric stereo) data for audio data of a program. The MPEG-4 AAC standard takes into account that when the header of the stuffing element (for example, its extended payload) initializes an SBR object type (like "Header 1" in Figure 7) and the spectrum band of the stuffing element duplicates the extended element In the case of PS data, the padding element (for example, its extended payload) includes spectrum band copy data and the "bs_extension_id" parameter. The parameter value (ie, bs_extension_id=2) indicates that the PS data is included in the spectrum band copy of the padding element Expansion element.

According to some embodiments of the present invention, eSBR metadata (for example, a flag indicating whether to perform enhanced spectral band replication (eSBR) processing on the audio content of the block) is included in the spectral band replication extension element of the filler element. For example, this flag is indicated in the padding element 1 of FIG. 7, where the flag appears after the header of the "SBR extension element" of the padding element 1 (the "SBR extension header" of the padding element 1). Optionally, this flag and additional eSBR metadata are also included in the spectrum band replication extension element, which is after the header of the spectrum band replication extension element (for example, in the SBR extension element of the padding element in FIG. 7 , After the SBR extension header). According to some embodiments of the present invention, the padding element including eSBR metadata also includes the "bs_extension_id" parameter. The parameter value (for example, bs_extension_id=3) indicates that the eSBR metadata is included in the padding element and indicates that the relevant The audio content of the block is processed by eSBR.

According to some embodiments of the present invention, the eSBR metadata is contained in the padding element of the MPEG-4 AAC bitstream (for example, padding element 2 in FIG. 7), instead of duplicating the extended element (SBR) in the spectrum band of the padding element. Extension element). This is because the padding element of extension_payload() containing SBR data or SBR data with CRC does not contain any other expansion payload of any other expansion type. Therefore, in an embodiment where the eSBR metadata stores its own expansion load, a separate padding element is used to store the eSBR metadata. This filling element includes an identifier (for example, "ID2" in FIG. 7), which indicates the start of the filling element, and the filling data is after the identifier. The stuffing data includes an extension_payload() element (sometimes referred to as the extended payload in this article), the syntax of which is shown in Table 4.57 of the MPEG-4 AAC standard. The padding data (e.g., its extended payload) includes a header (e.g., "Header 2" of padding element 2 in FIG. 7), which represents an eSBR object (ie, the header initializes an enhanced spectrum band copy ( eSBR) object type), and the padding data (for example, its extended payload) includes eSBR metadata after the header. For example, the padding element 2 of FIG. 7 includes this header ("header 2") and also includes the eSBR metadata after the header (ie, the "flag" in the padding element 2, which indicates whether to The audio content of this block performs enhanced spectral band replication (eSBR) processing. Optionally, after header 2, additional eSBR metadata is also included in the padding data of padding element 2 in Figure 7. As described in this paragraph In the described embodiment, the header (for example, header 2 in FIG. 7) has an identification value, which is not one of the conventional values defined in Table 4.57 of the MPEG-4 AAC standard, but instead represents An eSBR extension payload (such that the extension_type field of the header indicates that the padding data includes eSBR metadata).

In the first type of embodiment, the present invention is an audio processing unit (for example, a decoder), including:

The memory (for example, the buffer 201 of FIG. 3 or 4) is configured to store at least one block of the encoded audio bitstream (for example, at least one block of the MPEG-4 AAC bitstream);

The bit stream load de-formatter (for example, the element 205 of FIG. 3 or the element 215 of FIG. 4) is coupled to the memory and is configured to demultiplex at least a part of the block of the bit stream ;as well as

The decoding subsystem (for example, the elements 202 and 203 of FIG. 3, or the elements 202 and 213 of FIG. 4) is coupled and configured to decode at least a portion of the audio content of the block of the bit stream, wherein the area The blocks include:

A padding element, which includes an identifier indicating the start of the padding element (for example, the "id_syn_ele" identifier has a value of 0×6 in Table 4.85 of the MPEG-4 AAC standard), and the padding data is after the identifier, where the The filling information includes:

The first flag identifies whether to perform the basic form of spectral band copying processing or the enhanced form of spectral band copying processing on the audio content of the at least one block of the encoded audio bitstream (for example, using all the blocks in the block). The spectrum band copy data and eSBR metadata included), and if the first flag identifies the enhanced form of the spectrum band copy processing, the second flag identifies whether to enable or disable the signal adaptive frequency domain oversampling.

The first flag is eSBR metadata, and an example of the flag is the sbrPatchingMode flag. Another example of this flag is the harmonicSBR flag. Both of these flags indicate whether to perform the basic form of spectrum band copy or the enhanced form of spectrum copy for the audio data of the block. formula. The basic form of spectrum copying is spectrum inpainting, and the enhanced form of spectrum copying is harmonic shifting.

In some embodiments, the padding data also includes additional eSBR metadata (ie, eSBR metadata other than the flag).

The memory may be a buffer memory (for example, the embodiment of the buffer 201 in FIG. 4) that stores (for example, in a non-transitory manner) at least one block of the encoded audio bitstream.

It is estimated that during the decoding of MPEG-4 AAC bitstreams including eSBR metadata (representing these eSBR tools), the eSBR processing performed by the eSBR decoder (using eSBR harmonic transposition, pre-flattening, and inter_TES tools) The complexity of the performance can be as follows (for typical decoding using the indicated parameters):

●Harmonic shift (16kbps, 14400/28800Hz)

○Based on DFT: 3.68 WMOPS (weighted million operations per second);

○Based on QMF: 0.98 WMOPS;

●QMF repair pre-processing (pre-flattening): 0.1WMOPS; and

●Inter-subband sample time envelope shaping (inter-TES): up to 0.16 WMOPS.

It is known that for transients, DFT-based permutation generally performs better than QMF-based permutation.

According to some embodiments of the present invention, the padding element (of the encoded audio bitstream) containing eSBR metadata also includes a parameter (for example, the "bs_extension_id" parameter), and the parameter value (for example, bs_extension_id=3) sends out the eSBR Metadata is the signal contained in the filling element and the signal that will perform eSBR processing on the audio content of the relevant block, and/or A parameter (for example, the same "bs_extension_id" parameter) whose value (for example, bs_extension_id=2) signals that the sbr_extension() container of the filling element includes PS data. For example, as shown in Table 1 below, such a parameter with the value bs_extension_id=2 can signal that the sbr_extension() container of the filling element includes PS data, and this parameter with the value bs_extension_id=3 can send the filling element The sbr_extension() container includes the signal of eSBR metadata:

According to some embodiments of the present invention, the syntax of each spectrum band copy extension element including eSBR metadata and/or PS data is shown in Table 2 below (where "sbr_extension()" represents a container, which is a spectrum band Copy the extension element, "bs_extension_id" is as described in Table 1 above, "ps_data" means PS data, and "esbr_data" means eSBR metadata):

In an exemplary embodiment, esbr_data() mentioned in Table 2 above indicates the value of the following metadata parameters:

1. Each of the one-bit metadata parameter "harmonicSBR"; "bs_interTES"; and "bs_sbr_preprocessing";

2. For each channel ("ch") of the audio content of the encoded bitstream to be decoded, each of the above parameters: "sbrPatchingMode[ch]"; "sbrOversamplingFlag[ch]"; "sbrPitchInBinsFlag[ch] "; and "sbrPitchInBins[ch]"; and

3. For each SBR envelope ("env") of each channel ("eh") of the audio content of the encoded bitstream to be decoded, each of the above parameters: "bs_temp_shape[ch][env]"; And "bs_inter_temp_shape_mode[ch][env]".

For example, in some embodiments, esbr_data() may have the syntax shown in Table 3 to indicate these metadata parameters:

In Table 3, the number in the middle row represents the bit number of the corresponding parameter in the left row.

In some embodiments, the present invention is a method including the step of encoding audio data to generate an encoded bitstream (for example, MPEG-4 AAC bitstream), the step including by including eSBR metadata In at least one section of at least one block of the encoded bit stream, and audio data is included in at least one other section of the block. In a typical embodiment, the method includes the step of multiplexing the audio data with the eSBR metadata in each block of the encoded bit stream. In a typical decoding of an encoded bit stream in an eSBR decoder, the decoder extracts eSBR metadata from the bit stream (including by parsing and demultiplexing eSBR metadata and audio data), and uses the eSBR Metadata is used to process the audio data to generate a stream of decoded audio data.

Another aspect of the present invention is an eSBR decoder, which is configured to perform eSBR processing during decoding of an encoded audio bitstream (for example, MPEG-4 AAC bitstream) that does not include eSBR metadata ( For example, use at least one of eSBR tools called harmonic shifting, pre-flattening, or inter_TES). An example of such a decoder will be described with reference to FIG. 5.

The eSBR decoder (400) of FIG. 5 includes a buffer memory 201 (which is equivalent to the memory 201 of FIGS. 3 and 4), and a bitstream load de-formatter 215 (which is equivalent to the de-formatter 215 of FIG. 4). ), audio decoding subsystem 202 (sometimes called "core" decoding stage or "core" decoding subsystem, and it is equivalent to core decoding subsystem 202 in Figure 3), eSBR control data generation subsystem 401, and eSBR The processing stage 203 (which is equivalent to the stage 203 of FIG. 3) is connected as shown in the figure. Typically, the decoder 400 also includes other processing Elements (not shown).

In the operation of the decoder 400, a sequence of blocks of the encoded audio bitstream (MPEG-4 AAC bitstream) received by the decoder 400 is judged and prompted from the buffer 201 to the formatter 215 .

The deformatter 215 is coupled and configured to demultiplex each block of the bitstream to extract SBR metadata (including quantized envelope data), and usually also extract other metadata from it. The formatter 215 is configured to prompt at least the SBR metadata judgment to the eSBR processing stage 203. The de-formatter 215 is also coupled and configured to extract audio data from each block of the bit stream, and to judge and prompt the extracted audio data to the decoding subsystem (decoding stage) 202.

The audio decoding subsystem 202 of the decoder 400 is configured to decode the audio data extracted by the formatter 215 (this kind of decoding may be referred to as a "core" decoding operation) to generate decoded audio data, and the The decoded audio data is judged and prompted to the eSBR processing stage 203. The decoding is performed in the frequency domain. Typically, the final stage of processing in the subsystem 202 applies a frequency domain to time domain conversion to the decoded frequency domain audio data so that the output of the subsystem is time domain decoded audio data. The stage 203 is configured to apply the SBR tool (and eSBR tool) indicated by the SBR metadata (extracted by the deformatter 215) and the eSBR metadata generated in the subsystem 401 to the decoded audio data (ie, use The SBR and eSBR metadata performs SBR and eSBR processing on the output of the decoding subsystem 202 to generate fully decoded audio data, which is output from the decoder 400. Typically, the decoder 400 includes a memory (accessible by the subsystem 202 and the stage 203), which The memory stores the de-formatted audio data and metadata output from the de-formatter 215 (and optionally also from the subsystem 401), and the stage 203 is configured to access what is needed during SBR and eSBR processing Audio data and metadata. The SBR processing in stage 203 can be regarded as a post-processing of the output of the decoding subsystem 202. Optionally, the decoder 400 also includes a final upmixing subsystem (which can use the parametric stereo ("PS") tool defined in the MPEG-4 AAC standard, using the PS extracted by the formatter 215 Metadata), which is coupled and configured to perform upmixing on the output of stage 203 to generate fully decoded and upmixed audio, which is output from APU 210.

The control data generation subsystem 401 of FIG. 5 is coupled and configured to detect at least one attribute of the encoded audio bit stream to be decoded, and respond to at least one result of the detection step to generate eSBR control data (according to this Other embodiments of the invention may be or may include any type of eSBR metadata included in the encoded audio bitstream). The eSBR control data is judged and prompted to level 203 to trigger the application of individual eSBR tools or a combination of eSBR tools when a specific attribute (or combination of attributes) of the bit stream is detected, and/or to control Application of this eSBR tool. For example, in order to control the performance of eSBR processing using harmonic shifting, certain embodiments of the control data generation subsystem 401 may include: a music detector (for example, a simplified version of a traditional music detector) for responding to detection Set the sbrPatchingMode[ch] parameter to the bitstream expressing or not expressing music (and determine the parameter prompting the setting to level 203); the transient detector is used to respond to the detection of the audio indicated by the bitstream Presence or absence of content Set the sbrOversamplingFlag[ch] parameter (and judge the parameter that prompts the setting to level 203) for transient change; and/or the pitch detector to respond to the detection of the audio content indicated by the bit stream The sbrPitchInBinsFlag[ch] and sbrPitchInBins[ch] parameters are set based on the pitch (and the parameter that determines the setting prompts to level 203). Another aspect of the present invention is the audio bitstream decoding method implemented by any embodiment of the decoder of the present invention described in this paragraph and the previous paragraph.

Aspects of the invention include encoding or decoding methods, having the type in which any embodiment of the APU, system, or device of the invention is configured (eg, programmed) to execute. Other aspects of the invention include systems or devices that are configured (e.g., programmed) to perform any embodiment of the method of the invention, and computer-readable media (e.g., optical discs) that store program codes (e.g., with Non-transitory mode) is used to perform any embodiment of the method of the invention or its steps. For example, the system of the present invention may be or may include a programmable general-purpose processor, a digital signal processor, or a microprocessor, which is programmed with software or firmware and/or is otherwise configured to perform any multiple operations on data, including An embodiment of the method of the invention or its steps. Such a general-purpose processor may be or may include a computer system, which includes an input device, a memory, and a processing circuit, and is programmed (and/or otherwise configured) to perform the method of the present invention in response to the data prompted to it by the judgment ( Or its steps).

The embodiments of the present invention can be implemented in hardware, firmware, or software, or a combination of both (for example, a programmable logic array). Unless otherwise specified, the algorithms or processes included as part of the present invention are not inherently related to any particular computer or other device. Especially, various general-purpose machines It can be used together with the code written according to the teachings of this article, or it can be more convenient to construct a more specialized device (for example, an integrated circuit) to execute the required method steps. Therefore, the present invention can be implemented in one or more computer programs that are executed on one or more programmable computer systems (for example, any of the elements in FIG. 1 or the encoder 100 in FIG. 2 (Or its component), or the decoder 200 (or its component) of FIG. 3, or the decoder 210 (or its component) of FIG. 4, or the implementation of the decoder 400 (or its component) of FIG. 5), the One or more programmable computer systems each include at least one processor, at least one data storage system (including volatile or non-volatile memory and/or storage elements), at least one input device or port, and at least one output device Or port. The code is applied to the input data to perform the functions described in this article and generate output information. The output information is applied to one or more output devices in a known manner.

Each such program can be implemented in any desired computer language (including machine language, assembly language, or high-level programming language, logic language, or object-oriented programming language) to communicate with the computer system. In any case, the language can be a compiled language or an interpreted language.

For example, when implemented by a computer software instruction sequence, various functions and steps of the embodiments of the present invention can be implemented by a multi-threaded software instruction sequence running in an appropriate digital signal processing hardware. In this case, the embodiment The various devices, steps, and functions of the software can correspond to the part of the software instructions.

Each such computer program is preferably stored or downloaded to a general-purpose or special-purpose programmable computer-readable storage medium or device (for example, solid-state memory or media, or magnetic or optical media) for use as the storage Media or device When read by the computer system to execute the program described herein, the computer is configured and operated. The system of the present invention can also be implemented as a computer-readable storage medium configured with (ie, storing) a computer program, wherein the storage medium so configured allows the computer system to operate in a specific and predetermined manner to perform the functions described herein.

A number of embodiments of the invention have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the invention. According to the above teachings, many modifications and variations of the present invention are possible. It should be understood that within the scope of the appended patent application, the present invention can be implemented in a manner different from that specifically described herein. Any reference numbers included in the scope of the following patent applications are for illustrative purposes only, and should not be used to interpret or limit the scope of the patent applications in any way.

200: decoder

201: Buffer memory

202: Audio Decoding Subsystem

203: eSBR processing level

204: Control bit generation stage

205: Bitstream payload formatter (parser)

300: post processor

301: Buffer memory (buffer)

Claims (8)

An audio processing unit, comprising: a bit stream load de-formatter, configured to demultiplex the blocks of the encoded audio bit stream; a decoding subsystem, coupled to the bit stream load de-formatter, And is configured to decode at least a part of the block of the encoded audio bitstream, wherein the block of the encoded audio bitstream includes: a padding element having an identification indicating the start of the padding element And the padding data after the identifier, wherein the padding data includes: at least one flag identifying whether to perform an enhanced form of spectral band copy processing on the audio content of at least one block of the encoded audio bitstream , And enhanced spectrum band replication metadata, which does not include one or more of the parameters used for harmonic transposition and spectrum repair, wherein the enhanced spectrum band replication metadata is the metadata that is configured to enable at least one eSBR tool, the The eSBR tool is described or mentioned in the MPEG USAC standard and not described or mentioned in the MPEG-4 AAC standard, where the enhanced spectrum band replication metadata includes a parameter indicating whether to perform pre-flattening, and if the parameter indicates To perform pre-flattening, the decoding subsystem is further configured to perform additional preprocessing to avoid discontinuities in the shape of the spectral envelope of the high-frequency signal input to the envelope adjuster. For example, the audio processing unit of item 1 in the scope of patent application, wherein the encoded audio bit stream is an MPEG-4 AAC bit stream. For example, the audio processing unit of item 1 in the scope of patent application, wherein the identifier is a three-digit unsigned integer, the most significant bit is transmitted first and has a value of 0×6. For example, the audio processing unit of item 1 in the scope of the patent application, wherein the filling data includes an extended load, and the extended load includes spectrum band copying extended data, and uses the most significant bit to transmit first and has a value of '1101' or '1110' The four-bit unsigned integer of to identify the extended load, and the spectrum band copy extended data includes: a spectrum band copy header, the spectrum band copy data after the header, and the spectrum band copy data after the spectrum band copy data The spectrum band replicates the extended element of, and the flag is included in the spectrum band replicated extended element. A method for decoding an encoded audio bitstream, the method comprising: demultiplexing a block of the encoded audio bitstream; and demultiplexing the block of the encoded audio bitstream At least a part of the decoding, wherein the block of the encoded audio bit stream includes: a padding element having an identifier indicating the start of the padding element, and padding data after the identifier, wherein the padding data includes: A flag to identify whether to perform an enhanced form of spectral band copy processing on the audio content of at least one block of the encoded audio bitstream, and Enhanced spectral band replication metadata, which does not include one or more parameters used for harmonic transposition and spectrum repair, wherein the enhanced spectral band replication metadata is the metadata configured to enable at least one eSBR tool, the eSBR tool Is described or mentioned in the MPEG USAC standard and not described or mentioned in the MPEG-4 AAC standard, and wherein the enhanced spectrum band replication metadata includes a parameter indicating whether to perform pre-flattening, and if the parameter indicates that it will To perform pre-flattening, additional pre-processing is performed to avoid discontinuities in the shape of the spectral envelope of the high-frequency signal input to the envelope adjuster. For example, the method of item 5 of the scope of patent application, wherein the identifier is a three-digit unsigned integer, the most significant bit is transmitted first and has a value of 0×6. For example, the method of item 5 of the scope of the patent application, wherein the filling data includes an extended load, the extended load includes the spectrum band copying extended data, and the most significant bit is used to transmit the first four with a value of '1101' or '1110' A bit unsigned integer to identify the extended load, and the spectrum band copy extended data includes: a spectrum band copy header, the spectrum band copy data after the header, and the spectrum after the spectrum band copy data With a copy extension element, and the flag is included in the spectrum band copy extension element. Such as the method of item 5 of the scope of patent application, wherein the encoded audio bit stream is an MPEG-4 AAC bit stream.

Images