TW201941186A - Decoding audio bitstreams with enhanced spectral band replication metadata in at least one fill element - Google Patents

Abstract

Embodiments relate to an audio processing unit that includes a buffer, bitstream payload deformatter, and a decoding subsystem. The buffer stores at least one block of an encoded audio bitstream. The block includes a fill element that begins with an identifier followed by fill data. The fill data includes a first flag identifying whether a base form of spectral band replication processing or an enhanced form of spectral band replication processing is to be performed on audio content of the at least one block of the encoded audio bitstream, and if the first flag identifies the enhanced form of spectral band replication processing, a second flag identifying whether signal adaptive frequency domain oversampling is enabled or disabled. A corresponding method for decoding an encoded audio bitstream is also provided.

Description

   Decode an audio bitstream with enhanced spectral band replication metadata in at least one padding element   

The present invention relates to audio signal processing. Some embodiments relate to encoding and decoding audio bitstreams (e.g., bitstreams having the MPEG-4 AAC format), which include metadata for controlling enhanced spectrum band replication (eSBR). Other embodiments relate to decoding such a bit stream by a conventional decoder that is not configured to perform eSBR processing and ignore such metadata, or to decode such a bit stream by generating eSBR control data in response to the bit stream. Audio bit stream for metadata.

A typical audio bitstream includes both audio data (eg, encoded audio data) indicating one or more channels of audio content, and metadata indicating at least one characteristic of the audio data or audio content. One well-known format for generating an encoded audio bit stream is the MPEG-4 Advanced Audio Coding (AAC) format, which is described in the MPEG standard ISO / IEC 14496-3: 2009. In the MPEG-4 standard, AAC stands for "advanced audio coding" and HE-AAC stands for "high-efficiency advanced audio coding".

The MPEG-4 AAC standard defines several audio profiles that determine which components and encoding tools exist in compatible encoders and decoders. Three of these audio specifications are (1) AAC specifications, (2) HE-AAC specifications, and (3) HE-AAC v2 specifications. The AAC specification includes AAC low complexity (or "AAC-LC") object types. AAC-LC object series, with some adjustments, corresponds to the MPEG-2 AAC low complexity specification, and does not include the Spectral Band Replication ("SBR") object type nor the parametric stereo ("PS") object Pattern. The HE-AAC specification is a superset of the AAC specification and also includes SBR object types. The HE-AAC v2 specification is a superset of the HE-AAC specification, and also includes PS object types.

The SBR object type includes a spectral band copying tool, which is an important coding tool that significantly improves the compression efficiency of a perceptual audio codec. The SBR reconstructs the high-frequency components of the audio signal at the receiver (eg, in a decoder). Therefore, the encoder only needs to encode and transmit low-frequency components, allowing higher audio quality at low data rates. SBR replicates a sequence of harmonics based on the available limited bandwidth signals and control data obtained from the encoder. The sequence of the harmonics is truncated beforehand to reduce the data rate. The ratio of tonal and noise-like components is maintained by adaptive inverse filtering and optional addition of noise and sine signals. In the MPEG-4 AAC standard, the SBR tool performs spectrum patching, in which several adjacent Quadrature Mirror Filter (QMF) subbands are copied from the low-band portion of the audio signal transmission to the audio signal. In the high frequency band, the audio signal is generated in the decoder.

Spectrum repair is not ideal for certain audio formats, such as music content with relatively low crossover frequencies. Therefore, there is a need for techniques for improving spectral band replication.

The first type of embodiment relates to an audio processing unit, which includes a memory, a bit stream payload deformatter, and a decoding subsystem. The memory is configured to store at least one block of an encoded audio bit stream (eg, an MPEG-4 AAC bit stream). The bitstream payload deformatter is configured to demultiplex the encoded audio block. The decoding subsystem is configured to decode the audio content of the encoded audio block. The encoded audio block includes a padding element having an identifier indicating the start of the padding element, and padding information included after the identifier. The filling data includes a first flag, identifying whether to perform a basic form of the spectrum band copy process or an enhanced form of the spectrum band copy process on the audio content of the at least one block of the encoded audio bit stream, and if the first A flag identifies the enhanced form of the spectral band replication process, and a second flag identifies whether the enabled or disabled signal is adaptive frequency domain oversampling.

The second type of embodiment relates to a method for decoding an encoded audio bit stream. The method includes receiving at least one block of an encoded audio bit stream, demultiplexing at least some portions of the at least one block of the encoded audio bit stream, and decoding the encoded audio bit stream At least some parts of the at least one block. The at least one block of the encoded audio bitstream includes a padding element having an identifier indicating the start of the padding element, and padding data included after the identifier. The filling data includes a first flag, identifying whether to perform a basic form of the spectrum band copy process or an enhanced form of the spectrum band copy process on the audio content of the at least one block of the encoded audio bit stream, and if the first A flag identifies the enhanced form of the spectral band replication process, and a second flag identifies whether the enabled or disabled signal is adaptive frequency domain oversampling.

Other types of embodiments are related to encoding and transcoding audio bitstreams that contain metadata identifying whether an enhanced spectrum band replication (eSBR) process will be performed.

1‧‧‧ Encoder

2‧‧‧ delivery subsystem

3‧‧‧ decoder

4‧‧‧ post-processing unit

100‧‧‧ encoder

105‧‧‧ Encoder

106‧‧‧ Metadata generation level

107‧‧‧ Filler / Formatter Level

109‧‧‧Buffer memory

200‧‧‧ decoder

201‧‧‧Buffer memory

202‧‧‧Audio decoding subsystem

203‧‧‧eSBR processing level

204‧‧‧Control bit generation stage

205‧‧‧ bit stream payload deformatter (profiler)

210‧‧‧ Audio Processing Unit (APU)

213‧‧‧SBR processing level

215‧‧‧bit stream payload deformatter (profiler)

300‧‧‧ post processor

301‧‧‧Buffer memory (buffer)

400‧‧‧eSBR decoder

401‧‧‧eSBR control data generation subsystem

500‧‧‧ Audio Processing Unit (APU)

FIG. 1 is a block diagram of an embodiment of a system configured to perform an embodiment of the method of the present invention.

FIG. 2 is a block diagram of an encoder, which is an embodiment of an audio processing unit of the present invention.

FIG. 3 is a block diagram of a system including a decoder, which is an embodiment of the audio processing unit of the present invention, and optionally has a post-processor coupled thereto.

FIG. 4 is a block diagram of a decoder, which is an embodiment of an audio processing unit of the present invention.

FIG. 5 is a block diagram of a decoder, which is another embodiment of the audio processing unit of the present invention.

FIG. 6 is a block diagram of another embodiment of an audio processing unit according to the present invention.

FIG. 7 is a block diagram of an MPEG-4 AAC bitstream, including a segment into which the bitstream is divided.

    Symbols and terms    

Throughout this disclosure, including in the scope of patent applications, descriptions of operations (e.g., filtering, scaling, transforming, or applying gain to a signal or data) on ("on") a signal or data are used in a broad sense to represent Perform an operation directly on the signal or data, or on a processed version of the signal or data (eg, a version of the signal that has been preliminarily filtered or pre-processed before the operation is performed).

Throughout this disclosure, including in the scope of patent applications, the expression "audio processing unit" is used in a broad sense to indicate a system, device, or device configured to process audio data. Examples of audio processing units include, but are not limited to, encoders (e.g., transcoders), decoders, codecs, pre-processing systems, post-processing systems, and bit-stream processing systems (sometimes referred to as bit Meta Stream Processing Tool). Almost all consumer electronics, such as mobile phones, televisions, laptops, and tablets, include an audio processing unit.

Throughout this disclosure, including in the scope of patent applications, the terms "coupled" or "coupled" are used broadly to refer to one of directly or indirectly connected. Therefore, if the first device is coupled to the second device, the connection may be via a direct connection, or via an indirect connection through another device or connection. In addition, components integrated into or integrated with other components are also coupled to each other.

    Detailed description of an embodiment of the present invention    

The MPEG-4 AAC standard considers the various types of SBR processing of the encoded MPEG-4 AAC bitstream including metadata that instructs the decoder to apply (if any) to decode the audio content of the bitstream, and / or It controls this SBR process, and / or it indicates at least one characteristic or parameter of at least one SBR tool to be used to decode the audio content of the bitstream. In this text, the expression "SBR metadata" is used to represent this type of metadata described or mentioned in the MPEG-4 AAC standard.

The top layer of the MPEG-4 AAC bitstream is a sequence of data blocks ("raw_data_block" elements). Each data block contains audio data (typically used for time periods of 1024 or 960 samples) and related information and / or other The data section of the data (referred to herein as a "block"). In this article, the term "block" is used to refer to a section of the MPEG-4 AAC bitstream, which contains audio data (and corresponding metadata and optional) that determines or indicates one (but no more than one) "raw_data_block" element Other relevant information).

Each block of the MPEG-4 AAC bitstream may include some syntax elements (each syntax element is also embodied as a section of data in the bitstream). Seven types of this syntax element are defined in the MPEG-4 AAC standard. Each syntax element is identified by a different value of the data element "id_syn_ele". Examples of syntax elements include "single_channel_element ()", "channel_pair_element ()", and "fill_element ()". A mono element is a container that contains audio data (monaural audio signals) for a single audio channel. The two-channel element includes audio data (ie, stereo audio signals) for two audio channels.

The padding element is a container of information that includes an identifier (for example, the value of the above-mentioned element "id_syn_ele") followed by the data (which is referred to as "padding data"). Filler elements have traditionally been used to adjust the instantaneous bit rate of a bit stream to be transmitted on a fixed rate channel. By adding an appropriate amount of padding data to each block, a fixed data rate can be achieved.

According to an embodiment of the present invention, the padding data may include one or more extension payloads, which extend the type of data (eg, metadata) that can be transmitted in the bit stream. A decoder receiving a bitstream with stuffing data containing a new data type may optionally be used by a device (eg, a decoder) receiving the bitstream to extend the functionality of the device. Therefore, as can be understood by those skilled in the art, the padding element is a special type of data structure and is different from a data structure typically used to transmit audio data (eg, an audio payload containing channel data).

In some embodiments of the present invention, the identifier for identifying the padding element may be composed of a three-bit most significant bit transmitted by a previous unsigned integer ("uimsbf"), which has a value of 0 × 6. In a block, multiple instances of the same type of syntax element (eg, multiple padding elements) may occur.

Another standard for encoding audio bit streams is the MPEG Unified Speech and Audio Coding (USAC) standard (ISO / IEC 23003-3: 2012). The MPEG USAC standard describes the encoding and decoding of audio content using spectral band copy processing (including SBR processing as described in the MPEG-4 AAC standard, and also other enhanced forms of spectral band copy processing). This process applies an extended and enhanced version of the spectrum band replication tool (sometimes referred to herein as the "enhanced SBR tool" or "eSBR tool") of the set of SBR tools described in the MPEG-4 AAC standard. Therefore, eSBR (as defined in the USAC standard) is an improvement on SBR (as defined in the MPEG-4 AAC standard).

Herein, the expression "enhanced SBR processing" (or "eSBR processing") is used to indicate the use of at least one eSBR tool (for example, described or provided in the MPEG USAC standard) that is not described or mentioned in the MPEG-4 AAC standard. And at least one eSBR tool) of the frequency band replication process. Examples of such eSBR tools are harmonic transposition, QMF-patching additional pre-processing or "pre-flattening", and temporal envelope shaping between subband samples. ) Or "inter-TES".

The bitstream generated in accordance with the MPEG USAC standard (sometimes referred to herein as the "USAC bitstream") includes encoded audio content, and typically includes audio content that will be used by a decoder to decode the USAC bitstream. Each type of metadata of the spectrum band copy process, and / or at least one SBR tool and / or at least one of the eSBR tools that control the spectrum band copy process and / or represent the audio content that will be used to decode the USAC bitstream Metadata for features or parameters.

Herein, the expression "enhanced SBR metadata" (or "eSBR metadata") is used to indicate the frequency spectrum of audio content indicating that it will be used by a decoder to decode an encoded audio bitstream (e.g., USAC bitstream). Various types of metadata with copy processing, and / or metadata that controls the copy processing of this spectrum band, and / or instructions will be used to decode this audio content, but are not described or mentioned in the MPEG-4 AAC standard Metadata for at least one feature or parameter of at least one SBR tool and / or eSBR tool. An example of eSBR metadata is metadata that is described or mentioned in the MPEG USAC standard but not described or mentioned in the MPEG-4 AAC standard (indicating spectrum band copy processing, or used to control spectrum band copy processing) . Therefore, eSBR metadata in this article represents metadata that is not SBR metadata, and SBR metadata in this article represents metadata that is not eSBR metadata.

The USAC bitstream may include both SBR metadata and eSBR metadata. More specifically, the USAC bitstream may include eSBR metadata that controls the eSBR processing performance of the decoder, and SBR metadata that controls the SBR processing performance of the decoder. According to a typical embodiment of the present invention, eSBR metadata (e.g., eSBR-specific configuration data) is contained in (according to the present invention) an MPEG-4 AAC bitstream (e.g., in the sbr_extension () container at the end of the SBR payload) .

During the use of the eSBR toolset (including at least one eSBR tool) to decode an encoded bit stream, the decoder performs eSBR processing and regenerates the high-frequency band of the audio signal based on the duplication of the harmonic sequences truncated during the encoding process. . Such eSBR processing typically adjusts the generated high-frequency band spectral envelope, applies inverse filtering, and adds noise and sinusoidal components to re-establish the spectral characteristics of the original audio signal.

According to a typical embodiment of the present invention, eSBR metadata is included in one or more metadata sections of an encoded audio bitstream (e.g., MPEG-4 AAC bitstream) (e.g. Few control bits), the encoded audio bit stream also includes the encoded audio data in other sections (audio data section). Typically, at least one such metadata segment of each segment of the bitstream is (or includes) a padding element (containing an identifier indicating the start of the padding element), and the eSBR metadata is contained in Fill the element, after the identifier.

FIG. 1 is a block diagram of an exemplary audio processing chain (audio data processing system), in which one or more elements of the system can be configured according to an embodiment of the present invention. The system includes the following components, coupled together as shown in the figure: encoder 1, transfer subsystem 2, decoder 3, and post-processing unit 4. In a variation of the system shown, one or more of these components are omitted or an additional audio data processing unit is included.

In some embodiments, the encoder 1 (which optionally includes a pre-processing unit) is configured to accept as input an PCM (time domain) sample containing audio content and output a stream of encoded audio bits representing the audio content ( Has a format that complies with the MPEG-4 AAC standard). Bitstream data representing audio content is sometimes referred to herein as "audio data" or "encoded audio data." If the encoder is configured according to a typical embodiment of the present invention, the audio bit stream output from the encoder includes eSBR metadata (and typically also includes other metadata) and audio data.

One or more encoded audio bit streams output from the encoder 1 can be asserted to the encoded audio delivery subsystem 2. Subsystem 2 is configured to store and / or pass individual encoded bit streams output from encoder 1. The encoded bit stream output from the encoder 1 may be stored by the subsystem 2 (for example, in the form of a DVD or Blu-ray disc), or transmitted by the subsystem 2 (which may implement a transmission link or network), or by the subsystem 2 Store and transfer.

The decoder 3 is configured to decode an encoded MPEG-4 AAC audio bit stream (generated by the encoder 1), which is received via the subsystem 2. In some embodiments, the decoder 3 is configured to extract eSBR metadata from each block of the bitstream and decode the bitstream (including by performing eSBR processing by using the extracted eSBR metadata) to generate Decoded audio data (e.g., a stream of decoded PCM audio samples). In some embodiments, the decoder 3 is configured to extract the SBR metadata from the bitstream (but ignore the eSBR metadata contained in the bitstream), and decode the bitstream (including by using the extracted SBR metadata to perform SBR processing) to generate decoded audio data (eg, a stream of decoded PCM audio samples). Typically, the decoder 3 includes a buffer that stores (eg, in a non-transitory manner) a segment of the encoded audio bitstream received from the subsystem 2.

The post-processing unit 4 of FIG. 1 is configured to accept a stream of decoded audio data (eg, decoded PCM audio samples) from the decoder 3 and perform post-processing on it. The post-processing unit 4 may also be configured to present post-processed audio content (or decoded audio received from the decoder 3) for playback by one or more speakers.

FIG. 2 is a block diagram of an encoder (100), which is an embodiment of an audio processing unit of the present invention. Any component or element of the encoder 100 may be implemented as one or more processes and / or one or more circuits (e.g., ASICs, FPGAs, or other integrated circuits) in hardware, software, or a combination of hardware and software. Body circuit). The encoder 100 includes an encoder 105, a filler / formatter stage 107, a metadata generation stage 106, and a buffer memory 109, which are connected as shown in the figure. Typically, the encoder 100 also includes other processing elements (not shown). The encoder 100 is configured to convert an input audio bit stream into an encoded output MPEG-4 AAC bit stream.

Metadata generator 106 is coupled and configured to generate (and / or pass stage 107) metadata (including eSBR metadata and SBR metadata), which will be included by stage 107 in the to-be-outputted encoder 100 Coded bit stream.

The encoder 105 is coupled and configured to encode input audio data (e.g., by performing compression on it), and prompts the resulting encoded audio judgment to stage 107 for inclusion in the to-be-outputted stage 107 Coded bit stream.

Stage 107 is configured to multiplex the encoded audio from encoder 105 and metadata (including eSBR metadata and SBR metadata) from generator 106 to produce an encoded bit stream to be output from stage 107 Preferably, the encoded bit stream has a format as specified by one embodiment of the present invention.

The buffer memory 109 is configured to store (e.g., in a non-transitory manner) at least one block of the encoded audio bit stream output from the stage 107, and a sequence of the encoded audio bit stream The block will then be judged to prompt the buffer memory 109 as an output from the encoder 100 to the delivery system.

FIG. 3 is a block diagram of a system including a decoder (200), which is an embodiment of the audio processing unit of the present invention, and optionally has a post-processor (300) coupled thereto. Any component or element of the decoder 200 and post-processor 300 may be implemented as one or more processes and / or one or more circuits (e.g., ASICs, FPGAs, or other integrated circuits). The decoder 200 includes a buffer memory 201, a bit stream load deformatter (profiler) 205, an audio decoding subsystem 202 (sometimes referred to as a "core" decoding stage or "core" decoding subsystem), and an eSBR The processing stage 203 and the control bit generation stage 204 are connected as shown in the figure. Typically, the decoder 200 also includes other processing elements (not shown).

The buffer memory (buffer) 201 stores (eg, in a non-transitory manner) at least one block of the encoded MPEG-4 AAC audio bit stream received by the decoder 200. In the operation of the decoder 200, a sequence of blocks of the bit stream is judged and presented to the deformatter 205 by the buffer 201.

In a variation of the embodiment of FIG. 3 (or the embodiment of FIG. 4 to be described), an APU that is not a decoder (for example, APU 500 of FIG. 6) includes a buffer memory (for example, a buffer equivalent to buffer 201 Memory), which stores (e.g., in a non-transitory manner) the same form of encoded audio bitstream (e.g., MPEG-4 AAC audio bitstream) received by buffer 201 of FIG. 3 or FIG. ) Of at least one block (ie, an encoded audio bit stream including eSBR metadata).

Referring again to FIG. 3, the deformatter 205 is coupled and configured to demultiplex various blocks of the bitstream to extract SBR metadata (including quantized envelope data) and eSBR metadata (and generally also Including other metadata), to at least prompt the eSBR metadata and the SBR metadata judgment to the eSBR processing level 203, and typically also prompt other extracted metadata judgments to the decoding subsystem 202 (and optionally It is also judged that the prompt is sent to the control bit generator 204). The deformatter 205 is also coupled and configured to extract audio data from each block of the bit stream, and to prompt the extracted audio data to a decoding subsystem (decoding stage) 202.

The system of FIG. 3 optionally also includes a post-processor 300. The post-processor 300 includes a buffer memory (buffer) 301 and other processing elements (not shown) including at least one processing element coupled to the buffer 301. The buffer 301 stores (eg, in a non-transitory manner) at least one block (or frame) of decoded audio data received by the post-processor 300 from the decoder 200. Processing elements of post-processor 300 are coupled and configured to receive and adaptively process a sequence of blocks (or boxes) of decoded audio output from buffer 301, which uses self-decoding subsystem 202 (and / or Metadata output by the deformatter 205) and / or control bits output from the stage 204 of the decoder 200.

The audio decoding subsystem 202 of the decoder 200 is configured to decode the audio data extracted by the parser 205 (such decoding may be referred to as a "core" decoding operation) to generate decoded audio data, and determine to prompt the decoded audio data Audio data to eSBR processing stage 203. Decoding is performed in the frequency domain and usually involves inverse quantization followed by spectral processing. Typically, the final stage of processing in the sub-system 202 applies a frequency-domain to time-domain conversion to the decoded frequency-domain audio data such that the output of the subsystem is time-domain decoded data. The stage 203 is configured to apply to the decoded audio data the SBR tools and eSBR tools (i.e., use the SBR and eSBR metadata to decode the subsystem 202) The output performs SBR and eSBR processing) to generate fully decoded audio data, which is output from the decoder 200 (eg, to the post-processor 300). Typically, the decoder 200 includes a memory (accessible by the subsystem 202 and the stage 203) that stores the deformatted audio data and metadata output from the deformatter 205, and the stage 203 is configured To access audio data and metadata (including SBR metadata and eSBR metadata) required during SBR and eSBR processing. The SBR processing and eSBR processing in stage 203 may be considered as post-processing to the output of the core decoding subsystem 202. Optionally, the decoder 200 also includes a final upmixing subsystem (which can apply a parametric stereo ("PS") tool as defined in the MPEG-4 AAC standard, using the extraction by the deformatter 205 PS metadata and / or control bits generated in subsystem 204), which are coupled and configured to perform upmixing on the output of stage 203 to produce fully decoded, upmixed audio, which is self-decoding Device 200 output. Alternatively, the post-processor 300 is configured to perform upmixing on the output of the decoder 200 (eg, using PS metadata extracted by the deformatter 205 and / or control bits generated in the subsystem 204) .

In response to the metadata extracted by the deformatter 205, the control bit generator 204 may generate control data, and the control data may be used within the decoder 200 (e.g., in the final upmixing subsystem) and / Or it is judged that the prompt is output by the decoder 200 (for example, to the post-processor 300 for use in post-processing). Responsive to the metadata extracted from the output bit stream (and optionally also to the control data), stage 204 may generate (and determine a hint to post-processor 300) a control bit, which instructs the output from eSBR processing stage 203 Decoded audio data for should be post-processed in a certain type. In some embodiments, the decoder 200 is configured to prompt the metadata extracted by the deformatter 205 from the input bit stream to the post-processor 300, and the post-processor 300 is configured to use the metadata to The decoded audio data output from the decoder 200 performs post-processing.

4 is a block diagram of an audio processing unit ("APU") (210), which is another embodiment of the audio processing unit of the present invention. The APU 210 is a conventional decoder, which is not configured to perform eSBR processing. Any component or element of the APU 210 may be implemented as one or more processes and / or one or more circuits (e.g., ASICs, FPGAs, or other integrated circuits) in hardware, software, or a combination of hardware and software. Circuit). APU 210 includes buffer memory 201, bit stream payload deformatter (profiler) 215, audio decoding subsystem 202 (sometimes referred to as the "core" decoding stage or "core" decoding subsystem), and SBR The processing stage 213 is connected as shown. Typically, the APU 210 also includes other processing elements (not shown).

The components 201 and 202 of the APU 210 are the same as the components of the same number of the decoder 200 (of FIG. 3), and the above description of them will not be repeated. In the operation of the APU 210, a sequence of blocks of the encoded audio bit stream (MPEG-4 AAC bit stream) received by the APU 210 is judged and presented to the deformatter 215 from the buffer 201.

The deformatter 215 is coupled and configured to demultiplex various blocks of the bitstream to extract SBR metadata (including quantized envelope data), and typically also extract other metadata from it, but ignore According to other embodiments of the present invention, eSBR metadata may be included in the bitstream. The deformatter 215 is configured to prompt at least the SBR metadata judgment to the SBR processing stage 213. The deformatter 215 is also coupled and configured to extract audio data from each block of the bitstream, and prompt the judgment of the extracted audio data to the decoding subsystem (decoding stage) 202.

The audio decoding subsystem 202 of the decoder 200 is configured to decode the audio data extracted by the deformatter 215 (such decoding may be referred to as a "core" decoding operation) to generate decoded audio data and The decoded audio data is judged and presented to the SBR processing stage 213. The decoding is performed in the frequency domain. Typically, the final stage of processing in the subsystem 202 applies a frequency-domain to time-domain conversion to the decoded frequency-domain audio data such that the output of the subsystem is time-domain decoded data. The stage 213 is configured to apply the SBR tool (but not the eSBR tool) indicated by the SBR metadata (extracted by the deformatter 215) to the decoded audio data (ie, use the SBR metadata to decode the subsystem 202). The output performs SBR processing) to produce fully decoded audio data, which is output from the APU 210 (eg, to the post-processor 300). Typically, the APU 210 includes a memory (accessible by the subsystem 202 and the stage 213) that stores deformatted audio data and metadata output from the deformatter 215, and the stage 213 is configured to Access audio data and metadata (including SBR metadata) required during SBR processing. The SBR processing in stage 213 may be viewed as post-processing of the output of the core decoding subsystem 202. Optionally, the APU 210 also includes a final upmixing subsystem (which can apply a parametric stereo ("PS") tool as defined in the MPEG-4 AAC standard, using PS elements extracted by the deformatter 215 Data), which is coupled and configured to perform upmixing on the output of stage 213 to produce fully decoded, upmixed audio, which is output from APU 210. Alternatively, a post-processor is configured to perform upmixing on the output of the APU 210 (eg, using PS metadata extracted by the deformatter 215 and / or control bits generated in the APU 210).

Various embodiments of the encoder 100, decoder 200, and APU 210 are configured to perform different embodiments of the method of the present invention.

According to some embodiments, the encoded audio bitstream (e.g., MPEG-4 AAC bitstream) contains eSBR metadata (e.g., contains a small number of control bits that are eSBR metadata), such that a traditional decoder ( It is not configured to analyze eSBR metadata, or configured to use any eSBR tool to which the eSBR metadata belongs.) You can ignore eSBR metadata, but still try not to use eSBR metadata or any of the eSBR metadata. The eSBR tool is used to decode the bit stream, usually without any significant loss in decoding audio quality. However, an eSBR decoder configured to parse a bitstream to identify eSBR metadata and respond to eSBR metadata using at least one eSBR tool will enjoy the benefits of using at least one such eSBR tool. Therefore, an embodiment of the present invention provides a mechanism for efficiently transmitting enhanced spectrum band replication (eSBR) control data or metadata in a backward compatible manner.

Typically, the eSBR metadata in a bitstream represents one or more of the following eSBR tools (which are described in the MPEG USAC standard and which may or may not be applied by the encoder during the generation of the bitstream) ( For example, at least one characteristic or parameter representing one or more of them): ˙ harmonic shift; ˙QMF-repair) additional pre-processing (pre-flattening); and sample time envelope shaping or "inter-TES" between ˙ subbands .

For example, the eSBR metadata included in the bitstream may represent the values of the parameters (described in the MPEG USAC standard and this disclosure): harmonicSBR [ch], sbrPatchingMode [ch], sbrOversamplingFlag [ch], sbrPitchInBins [ch] , SbrPitchInBins [ch], bs_interTes, bs_temp_shape [ch] [env], bs_inter_temp_shape_mode [ch] [env], and bs_sbr_preprocessing.

Herein, the symbol X [ch], where X is a parameter, indicates that the parameter belongs to the audio channel ("ch") of the encoded bit stream to be decoded. For simplicity, the [ch] expression is sometimes omitted, and it is assumed that the relevant parameters belong to the channel of the audio content.

Herein, the symbol X [ch] [env], where X is a parameter indicating that the parameter belongs to the SBR envelope ("env") of the channel ("ch") of the audio content of the encoded bit stream to be decoded ). For simplicity, the expressions [env] and [ch] are sometimes omitted, and it is assumed that the relevant parameters belong to the channel SBR envelope of the audio content.

As mentioned, the MPEG USAC standard takes into account that the USAC bitstream includes eSBR metadata, which controls the performance of the eSBR processing performed by the decoder. The eSBR metadata includes the following one-bit metadata parameters: harmonicSBR; bs_interTES; and bs_pvc.

The parameter "harmonicSBR" indicates that harmonic repair (harmonic shift) is used for the SBR. Specifically, harmonicSBR = 0 indicates non-harmonic, spectrum repair, as described in section 4.6.18.6.3 of the MPEG-4 AAC standard; and harmonicSBR = 1 indicates harmonic SBR repair (which has the form used in eSBR, such as MPEG (As described in the USAC Standard Section 7.5.3 or 7.5.4). Based on non-eSBR spectral band replication (ie, SBR that is not eSBR), harmonic SBR patching is not used. From this disclosure, spectrum repair is referred to as the basic form of spectrum band replication, and harmonic shifting is referred to as the enhanced form of spectrum band replication.

The value of the parameter "bs_interTES" indicates that the inter-TES tool of eSBR is used.

The value of the parameter "bs_pvc" indicates that the PVC tool using eSBR is used.

The performance of the harmonic transposition during the decoding of the encoded bit stream during the eSBR processing stage (for each channel "ch" of the audio content indicated by the bit stream) is determined by the following eSBR metadata Controlled by parameters: sbrPatchingMode [ch]; sbrOversamplingFlag [ch]; sbrPitchInBinsFlag [ch]; and sbrPitchInBins [ch].

The value "sbrPatchingMode [ch]" indicates the type of transposer used in eSBR: sbrPatchingMode [ch] = 1 indicates non-harmonic patching, as described in section 4.6.18.6.3 of the MPEG-4 AAC standard; sbrPatchingMode [ch] = 0 means harmonic SBR patching, as described in MPEG USAC Standard Section 7.5.3 or 7.5.4.

The value "sbrOversamplingFlag [ch]" indicates the use of signal adaptive frequency domain oversampling in eSBR, combined with DFT-based harmonic SBR patching, as described in section 7.5.3 of the MPEG USAC standard. This flag controls the size of the DFT used in the transpose: 1 means that the signal adaptive frequency domain oversampling is allowed, as described in the MPEG USAC standard section 7.5.3.1; 0 means the signal adaptive frequency domain oversampling is disabled, As described in MPEG USAC Standard Section 7.5.3.1.

The value "sbrPitchInBinsFlag [ch]" controls the interpretation of the sbrPitchInBins [ch] parameter: 1 means that the value in sbrPitchInBins [ch] is valid and greater than zero; 0 means that the value of sbrPitchInBins [ch] is set to zero.

The value "sbrPitchInBins [ch]" controls the increase of the cross product term in the SBR harmonic transpose. The value sbrPitchinBins [ch] is an integer value in the range [0,127] and represents the distance measured in frequency bins of a 1536-line DFT acting on the sampling frequency of the core encoder.

In the case where the MPEG-4 AAC bit stream indicates an uncoupled SBR dual channel (instead of a single SBR channel), the bit stream indicates two instances of the above syntax (for harmonic or non-harmonic Wave transposition), an instance for one channel of sbr_channel_pair_element ().

The harmonic transposition of eSBR tools generally improves the quality of decoded music signals at relatively low crossover frequencies. Harmonic transposition should be implemented in the decoder via DFT-based or QMF-based harmonic transposition. Non-harmonic transposition (ie, traditional spectrum patching or duplication) generally improves speech signals. Therefore, deciding which type of transposition is better to be used as the starting point for encoding specific audio content is to select the transposition method based on voice / music detection. Harmonic shift is used for music content, and spectrum repair is used for speech content.

The effectiveness of pre-planarization during eSBR processing is controlled by the value of a one-bit eSBR metadata parameter called "bs_sbr_preprocessing", which means that pre-planarization is performed or not performed based on this single bit value. When using the SBR QMF repair algorithm (as described in section 4.6.18.6.3 of the MPEG-4 AAC standard), the step of pre-flattening can be performed (when indicated by the "bs_sbr_preprocessing" parameter) in an effort to avoid being input to Discontinuities in the shape of the spectral envelope of a high-frequency signal of a subsequent envelope adjuster (the envelope adjuster performs other stages of the eSBR processing). Pre-flattening generally improves the operation of subsequent envelope adjuster stages, producing high-band signals that are considered more stable.

During eSBR processing in the decoder, the performance of inter-subband sample time envelope shaping ("inter-TES" tool) is determined by the following channels ("ch") for the audio content of the USAC bitstream to be decoded The eSBR metadata parameters of the SBR envelope ("env") are controlled by: bs_temp_shape [ch] [env]; and bs_inter_temp_shape_mode [ch] [env].

The inter-TES tool processes QMF subband samples after the envelope adjuster. This processing step uses a finer time granularity than the envelope adjuster to shape the time envelope of the higher frequency band. By applying a gain factor to each QMF subband sample in the SBR envelope, inter-TES shapes the time envelope between QMF subband samples.

The parameter "bs_temp_shape [ch] [env]" is a flag that sends a signal using inter-TES. The parameter "bs_inter_temp_shape_mode [ch] [env]" indicates (as defined in the MPEG USAC standard) the value of the parameter γ in inter-TES.

According to some embodiments of the present invention, for the overall bit rate requirements included in the MPEG-4 AAC bit stream, the eSBR metadata representing the eSBR tools (harmonic transposition, pre-flattening, and inter_TES) is expected to be On the order of hundreds of bits per second, because only the differential control data needed to perform eSBR processing is transmitted. Traditional decoders can ignore this information because it is included in a backward compatible manner (to be explained later). Therefore, for some reasons, the adverse effects on the bit rate associated with the inclusion of eSBR metadata can be ignored. These reasons include the following: bitrate penalty (due to the inclusion of the eSBR metadata ) Is a very small part of the total bit rate, because only the differential control data required to perform eSBR processing is transmitted (not the simulcast of SBR control data); ˙ Tuning of SBR-related control information usually does not rely on transposition (transposition) details; and the ˙inter-TES tool (adopted during eSBR processing) performs single ended post-processing of the transposed signal.

Therefore, the embodiments of the present invention provide a mechanism for efficiently transmitting enhanced spectrum band replication (eSBR) control data or metadata in a backward compatible manner. The efficient transmission of such eSBR control data reduces the memory requirements in the decoder, encoder, and transcoder employing the aspects of the present invention, and has no significant adverse effect on the bit rate. In addition, the complexity and processing requirements associated with the implementation of the eSBR according to the embodiment of the present invention are reduced because the SBR data only needs to be processed once instead of simulcast, which can be if eSBR is considered as one of the MPEG-4 AAC Completely separate objects, rather than being integrated into the MPEG-4 AAC codec in a backward compatible manner.

Next, referring to FIG. 7, elements of a block (“raw_data_block”) of an MPEG-4 AAC bitstream according to some embodiments of the present invention will be described. The MPEG-4 AAC bitstream includes eSBR metadata. FIG. 7 is a diagram of one block ("raw_data_block") of the MPEG-4 AAC bit stream, showing some of its sections.

A block of an MPEG-4 AAC bitstream may include at least one "single_channel_element ()" (for example, the mono element shown in Figure 7), and / or at least one "channel_pair_element ()" (although it may exist , But not explicitly shown in FIG. 7), which includes audio data for audio programs. The block may also include some "fill_elements" (e.g., fill element 1 and / or fill element 2 of FIG. 7), which includes information (e.g., metadata) about the program. Each "single_channel_element ()" includes an identifier (eg, "ID1" in FIG. 7), which indicates the start of a mono element, and may include audio data indicating a different channel of a multi-channel audio program. Each "channel_pair_element" includes an identifier (not shown in Fig. 7), which indicates the start of the two-channel element, and may include audio data indicating the two channels of the program.

One fill_element (referred to herein as a fill element) of an MPEG-4 AAC bitstream includes an identifier ("ID2" in FIG. 7), which indicates the beginning of the fill element, and the fill data follows the identifier. The identifier ID2 may be composed of a three-bit most significant bit transmitted by a previous unsigned integer ("uimsbf"), which has a value of 0 × 6. The stuffing information may include an extension_payload () element (sometimes referred to herein as an extension payload), the syntax of which is shown in Table 4.57 of the MPEG-4 AAC standard. There are several types of extension payloads, and they are identified by the "extension_type" parameter, which is a four-bit most significant bit transmitted before the unsigned integer ("uimsbf").

The padding data (e.g., its extended payload) may include a header or identifier (e.g., "Header 1" of FIG. 7) indicating a section representing the padding data of the SBR object (ie, the header initializes a " The "SBR Object" type is called sbr_extension_data () in the MPEG-4 AAC standard. For example, the spectrum band replication (SBR) extension load is labeled as the value '1101' or '1110' for the extension_type field in the header, where the identifier '1101' identifies the extension load with SBR data, and '1110 'Identify the extended load with SBR data using a cyclic redundancy check (CRC) to verify the correctness of the SBR data.

When a header (e.g., extension_type field) initializes an SBR object type, the SBR metadata (sometimes referred to herein as "spectral band copy data" and sbr_data () in the MPEG-4 AAC standard) ) Follows the header, and at least one spectral band copy extension element (for example, the "SBR extension element" of padding element 1 of FIG. 7) may follow the SBR metadata. This spectral band copy extension element (a section of the bit stream) is called the "sbr_extension ()" container in the MPEG-4 AAC standard. The spectral band replication extension element optionally includes a header (eg, "SBR extension header" of padding element 1 of FIG. 7).

The MPEG-4 AAC standard takes into account that a spectral band replication extension element may include PS (Parametric Stereo) data for audio data of a session. The MPEG-4 AAC standard considers that when the header of a padding element (for example, its extended payload) initializes an SBR object type (as in "Header 1" in Figure 7) and the spectral band copy extension element of the padding element includes In PS data, the padding element (for example, its extension load) includes spectrum band replication data, and a "bs_extension_id" parameter whose value (ie, bs_extension_id = 2) indicates that the PS data is a spectrum band replication included in the padding element. Expansion element.

According to some embodiments of the present invention, eSBR metadata (for example, a flag indicating whether to perform enhanced spectrum band copy (eSBR) processing on the audio content of the block) is included in the spectrum band copy extension element of the padding element. For example, this flag is indicated in the padding element 1 of FIG. 7, where the flag appears after the header of the “SBR extension element” of the padding element 1 (“SBR extension header” of the padding element 1). Optionally, this flag and additional eSBR metadata are also included in the spectrum band replication extension element, which is after the header of the spectrum band replication extension element (for example, in the SBR extension element of the padding element in FIG. 7). After the SBR extension header). According to some embodiments of the present invention, the padding element including the eSBR metadata also includes a "bs_extension_id" parameter. The parameter value (for example, bs_extension_id = 3) indicates that the eSBR metadata is included in the padding element and indicates that the related The audio content of the block performs eSBR processing.

According to some embodiments of the present invention, the eSBR metadata is contained in a stuffing element (for example, stuffing element 2 in FIG. 7) of the MPEG-4 AAC bit stream, instead of copying an extension element (SBR) in the frequency band of the stuffing element Expansion element). This is because the stuff element of extension_payload () containing SBR data with SBR data or SBR data with CRC does not contain any other extension payload of any other extension type. Therefore, in the embodiment where the eSBR metadata stores its own extended load, a separate padding element is used to store the eSBR metadata. This filling element includes an identifier (for example, "ID2" of FIG. 7), which indicates the beginning of the filling element, and the filling data is after the identifier. The stuffing information includes an extension_payload () element (sometimes referred to herein as an extension payload) whose syntax is shown in Table 4.57 of the MPEG-4 AAC standard. The padding data (e.g., its extended payload) includes a header (e.g., "Header 2" of padding element 2 of Figure 7), which represents an eSBR object (i.e., the header initiates an enhanced spectrum band copy ( eSBR) object type), and the padding data (eg, its extended payload) includes eSBR metadata after the header. For example, the padding element 2 of FIG. 7 includes this header ("Header 2") and also includes the eSBR metadata after the header (that is, the "flag" in the padding element 2, which indicates whether the The audio content of this block performs enhanced spectrum band replication (eSBR) processing. Optionally, after Header 2, additional eSBR metadata is also included in the padding data of padding element 2 of Figure 7. In this paragraph In the described embodiment, the header (for example, header 2 of FIG. 7) has an identification value, which is not one of the conventional values defined in Table 4.57 of the MPEG-4 AAC standard, but instead represents An eSBR extension payload (so that the extension_type field of the header indicates that the padding data includes eSBR metadata).

In a first type of embodiment, the present invention is an audio processing unit (e.g., a decoder) including: a memory (e.g., buffer 201 of FIG. 3 or 4) configured to store encoded audio bits At least one block of the stream (e.g., at least one block of the MPEG-4 AAC bit stream); the bit stream payload deformatter (e.g., element 205 of FIG. 3 or element 215 of FIG. 4) is coupled to The memory, and configured to demultiplex at least a portion of the block of the bitstream; and a decoding subsystem (e.g., elements 202 and 203 of FIG. 3, or elements 202 and 213 of FIG. 4), Coupled and configured to decode at least a portion of the audio content of the block of the bitstream, wherein the block includes: a padding element that includes an identifier (e.g., "id_syn_ele" identification The identifier has the value 0 × 6) of the table 4.85 of the MPEG-4 AAC standard, and the padding data is after the identifier, wherein the padding data includes: a first flag, identifying whether the coded audio bit stream is Basic form of spectrum band copy processing for audio content of at least one block Enhanced form of spectral band copy processing (for example, using the spectral band copy data and eSBR metadata contained in the block), and if the first flag identifies the enhanced form of the spectral band copy process, the second flag identifies Do you want to enable or disable adaptive frequency domain oversampling?

The first flag is eSBR metadata, and an example of the flag is the sbrPatchingMode flag. Another example of this flag is the harmonicSBR flag. Both of these flags indicate whether the basic form of the spectral band copy or the enhanced form of the spectral copy is performed on the audio data of the block. The basic form of spectrum replication is spectrum repair, and the enhanced form of spectrum replication is harmonic shift.

In some embodiments, the padding data also includes additional eSBR metadata (ie, eSBR metadata in addition to the flag).

The memory may be a buffer memory (eg, the implementation of the buffer 201 of FIG. 4) that stores (eg, in a non-transitory manner) at least one block of the encoded audio bit stream.

It is estimated that eSBR processing (using eSBR harmonic transposition, pre-flattening, and inter_TES tools) performed by the eSBR decoder during decoding of the MPEG-4 AAC bitstream including eSBR metadata (representing these eSBR tools) The complexity of the performance can be as follows (for typical decoding using the indicated parameters): ˙ Harmonic transposition (16kbps, 14400 / 28800Hz) ○ Based on DFT: 3.68 WMOPS (weighted millions of operations per second); ○ Based on QMF: 0.98 WMOPS; ˙QMF repair pretreatment (pre-flattening): 0.1 WMOPS; and inter-TES sample time envelope shaping (inter-TES): up to 0.16 WMOPS.

It is known that for transients, DFT-based permutations generally perform better than QMF-based permutations.

According to some embodiments of the present invention, the padding element (encoded audio bit stream) containing the eSBR metadata also includes a parameter (eg, "bs_extension_id" parameter), and the parameter value (eg, bs_extension_id = 3) issues the eSBR Metadata is a signal contained in the padding element and signals that eSBR processing will be performed on the audio content of the relevant block, and / or a parameter (e.g., the same "bs_extension_id" parameter) whose value (e.g., bs_extension_id = 2) A signal that the sbr_extension () container of the padding element includes PS data is issued. For example, as shown in Table 1 below, such a parameter with a value of bs_extension_id = 2 may issue a signal of the sbr_extension () container of the padding element including PS data, and such a parameter with a value of bs_extension_id = 3 may issue the padding element The sbr_extension () container includes signals for eSBR metadata:

According to some embodiments of the present invention, the syntax of each spectrum band replication extension element including eSBR metadata and / or PS data is shown in Table 2 below (where "sbr_extension ()" represents a container, and the container is a spectrum band Copy the extension element, "bs_extension_id" is as described in Table 1 above, "ps_data" means PS data, and "esbr_data" means eSBR metadata):

In an exemplary embodiment, esbr_data () mentioned in Table 2 above indicates the values of the following metadata parameters: 1. The above-mentioned one-bit metadata parameter "harmonicSBR"; "bs_interTES"; and "bs_sbr_preprocessing" Each; 2. For each channel ("ch") of the audio content of the encoded bit stream to be decoded, each of the above parameters: "sbrPatchingMode [ch]"; "sbrOversamplingFlag [ch]"; "sbrPitchInBinsFlag" [ch] "; and" sbrPitchInBins [ch] "; and 3. SBR envelopes (" env ") for each channel (" ch ") of the audio content of the encoded bit stream to be decoded, the above parameters Each of them: "bs_temp_shape [ch] [env]"; and "bs_inter_temp_shape_mode [ch] [env]".

For example, in some embodiments, esbr_data () may have the syntax shown in Table 3 to indicate these metadata parameters:

In Table 3, the number in the middle row indicates the number of bits of the corresponding parameter in the left row.

In some embodiments, the present invention is a method comprising the step of encoding audio data to produce an encoded bit stream (e.g., an MPEG-4 AAC bit stream), the step including by including eSBR metadata In at least one section of at least one block of the encoded bitstream, and including audio data in at least one other section of the block. In a typical embodiment, the method includes the steps of multiplexing the audio data with the eSBR metadata in each block of the encoded bit stream. In a typical decoding of an encoded bitstream in an eSBR decoder, the decoder extracts eSBR metadata (including by parsing and demultiplexing eSBR metadata and audio data) from the bitstream and uses the eSBR Metadata to process the audio data to produce a stream of decoded audio data.

Another aspect of the present invention is an eSBR decoder configured to perform eSBR processing during decoding of an encoded audio bit stream (e.g., MPEG-4 AAC bit stream) that does not include eSBR metadata. For example, using at least one of the eSBR tools called harmonic transposition, pre-planarization, or inter_TES). An example of such a decoder will be described with reference to FIG. 5.

The eSBR decoder (400) of FIG. 5 includes a buffer memory 201 (which is equivalent to the memory 201 of FIGS. 3 and 4), and a bit stream load deformatter 215 (which is equivalent to the deformatter 215 of FIG. 4). ), Audio decoding subsystem 202 (sometimes referred to as "core" decoding level or "core" decoding subsystem, and it is equivalent to core decoding subsystem 202 of Fig. 3), eSBR control data generation subsystem 401, and eSBR Processing stage 203 (which is equivalent to stage 203 of FIG. 3) is connected as shown. Typically, the decoder 400 also includes other processing elements (not shown).

In the operation of the decoder 400, a sequence of blocks of the encoded audio bit stream (MPEG-4 AAC bit stream) received by the decoder 400 is judged and prompted from the buffer 201 to the deformatter 215. .

The deformatter 215 is coupled and configured to demultiplex various blocks of the bitstream to extract SBR metadata (including quantized envelope data), as well as generally extract other metadata from it. The deformatter 215 is configured to prompt at least the SBR metadata judgment to the eSBR processing stage 203. The deformatter 215 is also coupled and configured to extract audio data from each block of the bit stream, and prompt the judgment of the extracted audio data to the decoding subsystem (decoding stage) 202.

The audio decoding subsystem 202 of the decoder 400 is configured to decode the audio data extracted by the deformatter 215 (such decoding may be referred to as a "core" decoding operation) to produce decoded audio data and The decoded audio data is judged and presented to the eSBR processing stage 203. The decoding is performed in the frequency domain. Typically, the final stage of processing in the subsystem 202 applies frequency-domain to time-domain conversion to the decoded frequency-domain audio data such that the output of the subsystem is time-domain decoded audio data. The stage 203 is configured to apply the SBR tools (and eSBR tools) indicated by the SBR metadata (extracted by the deformatter 215) and the eSBR metadata generated in the subsystem 401 to the decoded audio data (i.e., using the The SBR and eSBR metadata perform SBR and eSBR processing on the output of the decoding subsystem 202) to generate fully decoded audio data, which is output from the decoder 400. Typically, decoder 400 includes a memory (accessible by subsystem 202 and stage 203) that stores deformatted output from deformatter 215 (and optionally also from subsystem 401). Audio data and metadata, and stage 203 is configured to access audio data and metadata required during SBR and eSBR processing. The SBR processing in stage 203 may be considered as post-processing of the output of the decoding subsystem 202. Optionally, the decoder 400 also includes a final upmixing subsystem (which can apply a parametric stereo ("PS") tool defined in the MPEG-4 AAC standard, using the PS extracted by the deformatter 215 (Metadata), which is coupled and configured to perform upmixing on the output of stage 203 to produce fully decoded, upmixed audio, which is output from APU 210.

The control data generation subsystem 401 of FIG. 5 is coupled and configured to detect at least one attribute of the encoded audio bit stream to be decoded, and respond to at least one result of the detection step to generate eSBR control data (according to this Other embodiments of the invention may be or may include any type of eSBR metadata contained in the encoded audio bitstream). The eSBR control data is judged and prompted to level 203 to trigger the application of an individual eSBR tool or a combination of eSBR tools when a specific attribute (or a combination of attributes) of the bit stream is detected, and / or to control Application of this eSBR tool. For example, in order to control the performance of eSBR processing using harmonic shift, some embodiments of the control data generation subsystem 401 may include a music detector (eg, a simplified version of a traditional music detector) for responding to detection Set the sbrPatchingMode [ch] parameter (and the parameter indicating the setting to level 203) to the bit stream to indicate whether or not the music is represented; a transient detector is used to respond to the audio signal indicated by the bit stream Set the sbrOversamplingFlag [ch] parameter (and determine the parameter prompting the setting to level 203) with or without transients in the content; and / or a pitch detector to respond to the detection of the bit stream The sbrPitchInBinsFlag [ch] and sbrPitchInBins [ch] parameters are set for the pitch of the indicated audio content (and it is judged that the parameter prompting the setting is to level 203). Another aspect of the present invention is an audio bit stream decoding method performed by any embodiment of the decoder of the present invention described in this paragraph and the previous paragraph.

Aspects of the present invention include encoding or decoding methods, with a type in which any embodiment of the APU, system, or device of the present invention is configured (eg, programmed) to perform. Other aspects of the invention include a system or device configured (e.g., programmed) to perform any embodiment of the method of the invention, and a computer-readable medium (e.g., an optical disc) that stores program code (e.g., Non-transitory manner) Any embodiment for performing the method of the invention or its steps. For example, the system of the present invention may be or may include a programmable general-purpose processor, digital signal processor, or microprocessor that is programmed in software or firmware and / or is additionally configured to perform any of a variety of operations on data, including Examples of the method of the invention or its steps. Such a general-purpose processor may be or may include a computer system including an input device, a memory, and a processing circuit, which is programmed (and / or otherwise configured) in response to information judged to prompt it to perform the method of the present invention ( Or its steps).

Embodiments of the invention may be implemented in hardware, firmware, or software, or a combination of both (e.g., a programmable logic array). Unless otherwise specified, the algorithms or processes included as part of the invention are not inherently related to any particular computer or other device. In particular, various general-purpose machines may be used with code written in accordance with the teachings herein, or it may be more convenient to construct more specialized equipment (e.g., integrated circuits) to perform the required method steps. Therefore, the present invention can be implemented in one or more computer programs running on one or more programmable computer systems (e.g., any element of FIG. 1, or the encoder 100 of FIG. 2). (Or its elements), or the decoder 200 (or its elements) of FIG. 3, or the decoder 210 (or its elements) of FIG. 4, or the decoder 400 (or its elements) of FIG. 5), One or more programmable computer systems each include at least one processor, at least one data storage system (including volatile or non-volatile memory and / or storage elements), at least one input device or port, and at least one output device Or port. The code is applied to the input data to perform the functions described in this article and generate output information. The output information is applied to one or more output devices in a known manner.

Each such program can be implemented in any desired computer language (including machine language, assembly language, or high-level programming language, logical language, or object-oriented programming language) to communicate with a computer system. In any case, the language can be a compiled language or an interpreted language.

For example, when implemented by a computer software instruction sequence, various functions and steps of the embodiments of the present invention may be implemented by a multi-threaded software instruction sequence running in appropriate digital signal processing hardware. In this case, the embodiment The various devices, steps, and functions of the software may correspond to portions of software instructions.

Each such computer program is preferably stored or downloaded to a general-purpose or special-purpose programmable computer-readable storage medium or device (e.g., solid-state memory or media, or magnetic or optical media) for use in the storage When the media or device is read by the computer system to execute the programs described herein, the computer is configured and operated. The system of the present invention can also be implemented as a computer-readable storage medium configured (ie, stored) with a computer program, wherein the storage medium so configured causes the computer system to operate in a specific and predetermined manner to perform the functions described herein.

A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Many modifications and variations of the present invention are possible in accordance with the above teachings. It should be understood that within the scope of the appended patent application, the present invention may be implemented in a manner different from that specifically described herein. Any reference signs included in the scope of patent applications below are for illustrative purposes only and should not be used to interpret or limit the scope of patent applications in any way.

Claims (15)

    An audio processing unit includes: a buffer configured to store at least one block of an encoded audio bitstream; a bitstream load deformatter, coupled to the buffer, and configured to encode the encoded Demultiplexing at least a portion of the at least one block of the audio bitstream; and a decoding subsystem, coupled to the bitstream load deformatter, and configured to configure the encoded audio bitstream with the At least a portion of at least one block is decoded, wherein the at least one block of the encoded audio bit stream includes: a padding element, having an identifier indicating a start of the padding element, and padding data after the identifier , Where the filling data includes at least one flag, identifying whether to perform an enhanced form of spectrum band copy processing on the audio content of the at least one block of the encoded audio bit stream, and enhanced spectrum band copy metadata, which Excludes one or more parameters for harmonic shifting and spectrum patching, where the enhanced spectrum band replication metadata is a resource configured to enable at least one eSBR tool It is expected that the eSBR tool is described or mentioned in the MPEG USAC standard and not described or mentioned in the MPEG-4 AAC standard, wherein the at least one flag is included in the extended payload and the audio processing device uses the return A function of the number of bits in the extended load.         For example, the audio processing unit in the first patent application scope, wherein the enhanced spectrum band replication metadata does not include parameters for selecting between harmonic shift and spectrum repair.         For example, the audio processing unit in the second patent application range, wherein the enhanced spectrum band replication metadata includes at least one of the following: i) a parameter indicating whether to perform pre-flattening; ii) a parameter indicating whether to perform sub-flattening Interband sample time envelope shaping; and iii) a parameter indicating whether to perform signal adaptive frequency domain oversampling.         For example, the audio processing unit of the first patent application scope, wherein the at least one block of the encoded audio bit stream includes spectrum band replication metadata.         For example, the audio processing unit in the fourth scope of the patent application, wherein the enhanced spectral band replication metadata does not include parameters equivalent to the parameters of the spectral band replication metadata.         For example, the audio processing unit in the patent application scope item 4, wherein the spectral band replication metadata is metadata configured to enable at least one SBR tool, which is described or mentioned in the MPEG-4 AAC standard.         For example, the audio processing unit in the fourth scope of the patent application, wherein the spectral band replication metadata includes one or more parameters for harmonic shift and spectrum repair.         For example, the audio processing unit in the first patent application scope, wherein the enhanced form of the spectrum band copy processing includes harmonic shift and does not include spectrum repair.         For example, the audio processing unit of the scope of patent application, wherein a value of the at least one flag indicates that the enhanced form of the spectrum band copy processing should be performed on the audio content of the at least one block of the encoded audio bit stream, Another value of the at least one flag indicates a basic form of performing a spectral band copy process on audio content of the at least one block of the encoded audio bit stream.         For example, the audio processing unit in the patent application No. 9 range, wherein the basic form of the spectrum band copy processing includes spectrum repair and does not include harmonic shift.         For example, the audio processing unit in the ninth scope of the patent application, wherein the basic form of the spectrum band copy processing is to use the spectrum band copy processing as described in the MPEG-4 AAC standard.         For example, the audio processing unit of the first patent application scope, wherein the enhanced form of the spectral band copy processing is the spectral band copy processing using at least one eSBR tool, which is described or mentioned in the MPEG USAC standard and is not described Or mentioned in the MPEG-4 AAC standard.         For example, the audio processing unit of the first patent application scope, wherein the audio processing unit is an audio decoder, and the identifier is a three-bit unsigned integer, the most significant bit is transmitted first and has a value of 0 × 6.         A method for decoding an encoded audio bit stream, the method comprising: receiving at least one block of an encoded audio bit stream; and at least a portion of the at least one block of the encoded audio bit stream Demultiplexing; and decoding at least a portion of the at least one block of the encoded audio bitstream, wherein the at least one block of the encoded audio bitstream includes: a padding element having an indication of the padding element The starting identifier, and padding data after the identifier, wherein the padding data includes: at least one flag identifying whether to perform spectrum on the audio content of the at least one block of the encoded audio bit stream Enhanced form with copy processing, and enhanced spectrum band copy metadata, which does not include one or more parameters for harmonic shift and spectrum repair, where the enhanced spectrum band copy metadata is configured to enable at least one eSBR Metadata of the tool, the eSBR tool is described or mentioned in the MPEG USAC standard and not described or mentioned in the MPEG-4 AAC standard, wherein the at least one flag is Included in the extended load and the method includes using a function that returns the number of bits in the extended load.         A computer-readable medium including instructions that, when executed by a processor, cause the computer to perform the steps of the method as defined in item 14 of the patent application.