RU2772778C2 - Temporary reconciliation of processing data based on quadrature mirror filter - Google Patents

Abstract

FIELD: data processing.

SUBSTANCE: invention relates to means for temporary reconciliation of processing data based on a quadrature mirror filter. A set of forms of subband signals is generated based on data on a signal form. Decoded metadata is generated based on metadata. Temporary reconciliation of the set of forms of subband signals and decoded metadata is performed. A restored sound signal frame is generated based on the time-reconciled set of forms of subband signals and decoded metadata. In this case, generation of the set of forms of subband signals based on data on a signal form contains application of signal form delay to a form of a signal that is provided in a time domain. At the same time, constant delay is included that is independent of length N of the restored sound signal frame.

EFFECT: decrease in the delay in sound encoding and decoding.

7 cl, 6 dwg, 1 tbl

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of United States Provisional Application No. 61/877.194, filed September 12, 2013, and United States Provisional Patent Application No. 61/909.593, filed November 27, 2013, the contents of each of which are incorporated herein in their entirety. by reference.

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

The present invention relates to the timing of encoded audio encoder data with corresponding metadata, such as spectral band replication (SBR) metadata, in particular High Efficiency (NOT) Advanced Audio Coding (AAC).

BACKGROUND OF THE INVENTION

A technical problem in the field of audio coding is to provide low latency audio coding and decoding systems, for example, to enable real-time applications such as live broadcasts. In addition, it is advantageous to provide audio encoding and decoding systems that exchange encoded bitstreams that can be connected to other bitstreams. In addition, computationally efficient audio encoding and decoding systems should be designed to enable cost-effective implementation of the systems. The present document addresses the technical problem of creating encoded bitstreams that can be connected in an efficient manner while simultaneously maintaining the delay time at an appropriate level for live broadcasting. The present document describes an audio encoding and decoding system that allows bitstreams to be combined with acceptable encoding delays, thereby enabling applications such as live broadcasting, in which a broadcast bitstream can be generated from a plurality of source bitstreams.

SUMMARY OF THE INVENTION

In accordance with one aspect, an audio decoder is described, configured to determine a reconstructed audio frame from an access block of a received data stream. Typically, the data stream contains a sequence of access blocks to determine the corresponding sequence of recovered audio frames. An audio frame typically contains a predetermined number N of audio time domain samples (when N is greater than one). In this regard, the sequence of blocks of access may respectively describe the sequence of frames of the audio signal.

The access block contains waveform data and metadata, wherein the waveform data and metadata are associated with the same reconstructed audio frame. In other words, the waveform data and the metadata for determining the reconstructed audio frame are contained in the same access block. Each of the access blocks in the sequence of access blocks may contain waveform data and metadata for generating a corresponding reconstructed frame in the sequence of reconstructed audio frames. In particular, a particular frame's access block may contain (eg, all of) the data necessary to determine a reconstructed frame for a particular frame.

In one example, a particular frame access block may contain (e.g., all of) the data necessary to implement a high frequency restoration (HFR) circuit to generate a particular frame high band signal based on a particular frame low band signal (contained in the waveform data and block access) and based on the decoded metadata.

Alternatively, or in addition, a particular frame's access unit may contain (eg, all of) the data necessary to implement the dynamic range extension of the particular frame. In particular, the expansion or expansion of the lower band signal of a particular frame may be performed based on the decoded metadata. To this end, the decoded metadata may contain one or more of the deployment parameters. Said one or more of the deployment parameters may be indicative of one or more of the following: whether or not compression/expansion should be applied to a particular frame; Whether or not compression/expansion should be applied uniformly to all channels of a multi-channel audio signal (i.e., should the same deployment gain(s) be applied to all channels of a multi-channel audio signal, or should different factor(s) be applied) deployment amplification for various channels of a multi-channel audio signal); and/or temporary resolution of the deployment gain.

Providing access blocks in sequence with access blocks that each contain the data necessary to generate the corresponding reconstructed audio frame, regardless of the previous or next access block, is useful for solving connection problems, since it allows you to connect the data flow between two adjacent access blocks, without affecting the quality of perception of the restored frame of the audio signal at the connection point (for example, immediately after it).

In one example, the reconstructed audio frame comprises a low band signal and a high band signal, wherein the u waveform data indicates the low band signal, and wherein the metadata indicates the spectrum envelope of the high band signal. The low band signal may correspond to an audio signal component spanning a relatively low frequency range (eg, containing frequencies below a predetermined crossover frequency). The high band signal may correspond to an audio signal component spanning a range of relatively high frequencies (eg, containing frequencies greater than a predetermined crossover frequency). The low band signal and the high band signal may complement each other with respect to the frequency range covered by the low band signal and the high band signal. The audio decoder may be configured to perform high frequency reconstruction (HFR) such as spectral band replication (SBR), high band signal with metadata and waveform data. In this regard, the metadata may comprise HFR or SBR metadata indicating the spectrum envelope of the high band signal.

The audio decoder may comprise a waveform processing path configured to generate a plurality of subband waveforms from the waveform data. The plurality of subband waveforms may correspond to a time domain waveform representation in the subband domain (eg, quadrature mirror filter (QMF) domain). The time domain waveform may correspond to the above lower band signal, and the plurality of subband waveforms may correspond to the plurality of the lower band signal. In addition, the audio decoder may comprise a metadata processing path configured to generate decoded metadata from the metadata.

In addition, the audio decoder may include a metadata application and synthesis unit configured to generate a reconstructed audio frame from the plurality of subband waveforms and from the decoded metadata. In particular, the metadata application and synthesis unit may be configured to implement an HFR and/or SBR scheme to generate a plurality of (e.g., scaled) upper band subband signals from a plurality of subband waveforms (i.e., in this case, from a plurality of subband waveforms). low band signals) and decoded metadata. The reconstructed audio frame can in this case be determined based on a plurality of (eg scaled) upper band sub-band signals and on the basis of a plurality of lower band signals.

Alternatively or in addition, the audio decoder may comprise a deployer configured to expand - or capable of expanding - a plurality of subband waveforms with at least some decoded metadata, in particular with said one or more than one of the deployment options included in the decoded metadata. To this end, the spreader may be configured to apply one or more of the spread gains to a plurality of subband waveforms. The expander may be configured to determine said one or more of the sweep gains based on a plurality of subband waveforms, based on one or more predefined compression/deployment rules or functions, and/or based on said one or more of the sweep parameters.

The waveform processing path and/or the metadata processing path may include at least one delay unit configured to timing the plurality of subband waveforms and the decoded metadata. In particular, said at least one delay unit may be configured to time-align a plurality of subband waveforms and decoded metadata and/or introduce at least one delay into the waveform processing path and/or into the metadata processing path such such that the total delay of the waveform processing path corresponds to the total delay of the metadata processing path. Alternatively, or in addition, said at least one delay unit may be configured to time-align the plurality of subband waveforms and the decoded metadata such that the plurality of subband waveforms and the decoded metadata are timely provided to the metadata application and synthesis unit for processing performed by the metadata application and synthesis unit. In particular, the plurality of subband waveforms and decoded metadata may be output to the metadata application and synthesis block such that the metadata application and synthesis block need not buffer the plurality of subband waveforms and/or decoded metadata before performing processing (e.g., HFR or SBR processing). ) a plurality of subband waveforms and/or decoded metadata.

In other words, the audio decoder may be configured to delay output of the decoded metadata and/or the plurality of subband signal forms to the metadata application and synthesis unit, which may be configured to implement the HFR scheme such that the decoded metadata and/or the plurality of subband signal forms signals are issued for processing as needed. The introduced delay may be selected to reduce (eg, minimize) the overall delay of the audio codec (comprising an audio decoder and a corresponding audio encoder) while still allowing the bitstream containing the access block sequence to be connected. In this regard, the audio decoder may be configured to process time-consistent access blocks that contain waveform data and metadata to determine a particular audio frame with minimal impact on overall audio codec delay. In addition, the audio decoder may be configured to process time-consistent access blocks without the need for metadata resampling. At the same time, the audio decoder is configured to determine a specific reconstructed frame of the audio signal in a computationally efficient manner and without degrading the audio quality. Therefore, the audio decoder can be configured to solve connection problems in a computationally efficient manner while maintaining high audio quality and low overall delay.

In addition, using at least one delay unit configured to timing multiple subband waveforms and decoded metadata can provide accurate and stable timing of the multiple subband waveforms and decoded metadata in a subband domain (in which, typically, multiple subband waveforms and decoded metadata are processed).

The metadata processing path may include a metadata delay block configured to delay the decoded metadata by a greater than zero integer multiple of the length N of the reconstructed audio frame. The additional delay introduced by the metadata delay block may be referred to as the metadata delay. The length N may correspond to the number N of time domain samples contained in the reconstructed audio frame. An integer multiple may be such that the delay introduced by the metadata delay block is greater than the delay introduced by the waveform processing path (eg, without considering the additional waveform delay introduced by the waveform processing path). The metadata delay may depend on the length N of the reconstructed audio frame. This may be due to the fact that the delay caused by processing in the waveform processing path depends on the length N of the frame. In particular, an integer multiple may be one for frame lengths N greater than 960, and/or an integer multiple may be two for frame lengths N greater than 960.

As stated above, the metadata application and synthesis unit may be configured to process decoded metadata and multiple subband waveforms in the subband domain (eg, in the QMF domain). In addition, the decoded metadata may indicate metadata (eg, indicate spectral coefficients describing the spectrum envelope of the high band signal) in the subband region. In addition, the metadata delay unit may be configured to delay the decoded metadata. The use of metadata delays that are greater than zero integer multiples of the frame length N may be advantageous as it provides stable alignment of multiple subband waveforms and decoded metadata in the subband domain (eg, for processing in the metadata application and synthesis block). In particular, this ensures that the decoded metadata can be applied to the proper frame of the waveform (ie, the proper frame of the plurality of subband waveforms) without the need for metadata resampling.

The waveform processing path may comprise a waveform delay block configured to delay a plurality of subband waveforms such that the total delay of the waveform processing path corresponds to a greater than zero integer multiple of the reconstructed audio frame length N. The additional delay introduced by the waveform delay block may be referred to as waveform delay. An integer multiple of the waveform processing path may correspond to an integer multiple of the metadata processing path.

The waveform delay block and/or the metadata delay block may be implemented as buffers that are configured to store a plurality of subband waveforms and/or decoded metadata for a period of time corresponding to the waveform delay and/or for a period of time corresponding to metadata delay. The waveform delay block may be installed anywhere in the waveform processing path before the metadata application and synthesis block. In this regard, the waveform delay block may be configured to delay waveform data and/or multiple subband waveforms (and/or intermediate data or signals in the waveform processing path). In one example, a waveform delay block may be distributed along the waveform processing path, with each of the distributed delay blocks providing some fraction of the total waveform delay. Allocation of the waveform delay block may be appropriate for a cost effective implementation of the waveform delay block. Similar to the waveform delay block, the metadata delay block can be installed anywhere in the metadata processing path before the metadata application and synthesis block. In addition, the waveform delay block may be distributed along the metadata processing path.

The waveform processing path may include a decoding and dequantization block configured to decode and dequantize the waveform data to obtain a plurality of frequency coefficients indicative of the waveform. In this regard, the waveform data may comprise or indicate a plurality of frequency coefficients, which enables the waveform of the reconstructed audio frame to be generated. Further, the waveform processing path may comprise a waveform synthesis unit configured to generate a waveform from a plurality of frequency coefficients. The waveform synthesis block may be configured to perform a conversion from the frequency domain to the time domain. In particular, the waveform synthesis block may be configured to perform an inverse modified discrete cosine transform (MDCT). The waveform synthesis block or the processing of the waveform synthesis block may introduce a delay that depends on the length N of the reconstructed audio frame. In particular, the delay introduced by the waveform synthesis block may correspond to the frame length N.

After reconstructing the waveform from the waveform data, it can be processed according to the decoded metadata. In one example waveform, the signal may be used with an HFR or SBR scheme to determine the high band signal using the decoded metadata. To this end, the waveform processing path may comprise an analysis unit configured to generate a plurality of subband waveforms from the waveform. The parsing unit may be configured to perform a transformation from the time domain to the subband domain, for example by applying a quadrature mirror filter (QMF) bank. Typically, the frequency resolution of the transformation performed by the waveform synthesis block is higher (eg, at least 5 or 10 times) than the frequency resolution of the transformation performed by the analysis block. This may be referred to by the terms "frequency domain" and "subband domain", wherein the frequency domain may be associated with a higher frequency resolution than the subband domain. The parser may introduce a constant delay that is independent of the length N of the reconstructed audio frame. The constant delay introduced by the parser may depend on the length of the filters in the filterbank used by the parser. For example, the constant delay introduced by the analyzer may correspond to 320 audio samples.

The overall delay of the waveform processing path may further depend on a predetermined prediction between the metadata and the waveform data. Such a prediction may be useful in order to increase continuity between adjacent reconstructed audio frames. The predetermined prediction and/or the corresponding prediction delay may correspond to 192 or 384 audio samples. The prediction delay may be a delay in case of determining the HFR or SBR metadata indicating the spectrum envelope of the high band signal. In particular, the prediction may allow the corresponding audio encoder to determine the HFR or SBR metadata of a particular audio frame based on a predetermined number of samples from the immediately following audio frame. This may be useful in cases where a particular frame includes an acoustic transient. The prediction delay may be applied by a prediction delay block contained in the waveform processing path.

In this regard, the total delay of the waveform processing path, i.e., the waveform delay, may depend on various processing that is performed in the waveform processing path. In addition, the waveform delay may depend on the metadata delay that is introduced into the metadata processing path. The waveform delay may correspond to an arbitrary multiple of the audio signal sample. For this reason, it may be advantageous to use a waveform delay block which is configured to delay the waveform, the waveform being represented in the time domain. In other words, it may be useful to apply a waveform delay to the waveform. This can ensure accurate and stable application of a waveform delay that corresponds to an arbitrary multiple of the audio signal sample.

One example of a decoder may include a metadata delay block that is configured to apply a metadata delay to the metadata, wherein the metadata can be represented in a subband domain, and a waveform delay block that is configured to apply a waveform delay to a waveform represented in time. areas. The metadata delay block may apply a metadata delay that corresponds to an integer multiple of the frame length N, and the waveform delay block may apply a metadata delay that corresponds to an integer multiple of the audio sample. As a result, accurate and stable alignment of the plurality of subband waveforms and decoded metadata for processing in the metadata application and synthesis block can be ensured. Processing of multiple forms of subband signals and decoded metadata may occur in the subband domain. Equalization of multiple subband waveforms and decoded metadata can be achieved without resampling the decoded metadata, thereby providing a computationally efficient and quality-preserving equalization means.

As noted above, an audio decoder may be configured to implement an HFR or SBR scheme. The metadata application and synthesis block may comprise a metadata application block that is configured to perform high frequency restoration (such as SBR) with a plurality of lower band signals and with the decoded metadata. In particular, the metadata application unit may be configured to transpose one or more of the plurality of low band signals to generate a plurality of high band subband signals. In addition, the metadata applying unit may be configured to apply the decoded metadata to the plurality of high band subband signals to obtain the plurality of scaled high band subband signals. The plurality of scaled high band subband signals may indicate the high band signal of the reconstructed audio frame. To generate a reconstructed audio frame, the metadata application and synthesis unit may further comprise a synthesis unit configured to generate a reconstructed audio frame from a plurality of low band signals and from a plurality of scaled high band subband signals. The synthesis block may be configured to perform an inverse transformation with respect to the transformation performed by the analysis block, for example, by applying a set of inverse QMFs. The number of filters contained in the synthesis block filter bank may be higher than the number of filters contained in the analysis block filter bank (eg, to account for the extended frequency range due to the many scaled high band subband signals).

As stated above, the audio decoder may comprise a deployer. The expander may be configured to change (eg increase) the dynamic range of the plurality of subband waveforms. The deployment block can be placed before the metadata application and synthesis block. In particular, a plurality of expanded subband waveforms may be used to implement an HFR or SBR scheme. In other words, the plurality of lower band signals used to implement the HFR or SBR scheme may correspond to the plurality of unwrapped subband waveforms at the output of the expander.

The spread block is preferably installed after the forecast delay block. In particular, the deployment unit may be installed between the prediction delay unit and the metadata application and synthesis unit. By placing the expander after the prediction delay block, i.e., by applying the prediction delay to the waveform data before the multiple subband waveforms are deployed, it is ensured that said one or more of the spreading parameters contained in the metadata are applied to the proper waveform data. signal. In other words, deploying the waveform data that has already been delayed by the prediction delay ensures that said one or more of the metadata spreading parameters are synchronized with the waveform data.

In this regard, the decoded metadata may comprise one or more of the deployment parameters, and the audio decoder may comprise a expander configured to generate a plurality of deployed subband waveforms based on the plurality of subband waveforms using said one or more of the deployment parameters. In particular, the expander may be configured to generate a plurality of unwrapped subband waveforms with an inverse function of a predetermined compression function. Said one or more of the expansion parameters may indicate an inverse function to a predetermined compression function. The reconstructed audio frame may be determined from a plurality of expanded subband waveforms.

As mentioned above, the audio decoder may include a prediction delay unit configured to delay the plurality of subband waveforms in accordance with a predetermined prediction to obtain the plurality of delayed subband waveforms. The expander may be configured to generate a plurality of deployed subband waveforms by unwrapping a plurality of delayed subband waveforms. In other words, the deployment block may be installed after the prediction delay block. This provides synchronism between the specified one or more of the deployment parameters and the plurality of subband waveforms to which the specified one or more of the deployment parameters apply.

The metadata application and synthesis unit may be configured to generate a reconstructed audio frame with the decoded metadata (namely, with the corresponding SBR/HFR metadata) for the temporal element of the subband waveform set. The time element may correspond to the number of time slots of the plurality of subband waveforms. The duration of the time element can be variable, ie. the duration of the time element of the plurality of subband waveforms to which the decoded metadata is applied may vary from one frame to another. In other words, the framing for the decoded metadata may change. The change in the duration of the time element may be limited to predetermined limits. The predetermined limits may correspond to frame duration minus delay and frame duration plus delay, respectively. The use of decoded waveform data (or portions thereof) for temporal elements of different durations may be useful for processing transient audio signals.

The expander may be configured to generate a plurality of deployed subband waveforms using said one or more of the deployment parameters for the same temporal element of the plurality of subband waveforms. In other words, the framing of said one or more of the deployment parameters may be the same as the framing of the decoded metadata that is used by the metadata application and synthesis block (eg, framing for SBR/HFR metadata). In this way, the stability of the SBR scheme and the companding scheme can be ensured, and the quality of the coding system can be improved.

In accordance with yet another aspect, an audio encoder is described, configured to encode a frame of an audio signal into a data stream access block. The audio encoder may be configured to perform appropriate processing tasks in consideration of the processing tasks performed by the audio decoder. In particular, the audio encoder may be configured to determine the waveform data and metadata from the audio frame and input the waveform data and metadata to the access unit. The waveform data and metadata may indicate a frame reconstructed from an audio frame. In other words, the waveform data and metadata may allow the corresponding audio decoder to determine a reconstructed version of the original audio frame. The audio signal frame may comprise a low band signal and a high band signal. The waveform data may indicate the low band signal, and the metadata may indicate the spectrum envelope of the high band signal.

The audio encoder may include a waveform processing path configured to generate waveform data per frame of the audio signal, such as a low band signal (eg, using a basic audio decoder such as an Advanced Audio Encoder (AAC). In addition, an audio the encoder comprises a metadata processing path configured to generate metadata per frame of an audio signal, such as on a high band signal and on a low band signal.As an example, an audio encoder may be configured to implement High Efficiency (NOT) AAC, and a corresponding audio decoder may be configured to decode the received data stream in accordance with HE AAC.

The waveform processing path and/or the metadata processing path may comprise at least one delay unit configured to time the waveform data and metadata such that the access unit for the audio frame contains the waveform data and metadata for the same frame of the audio signal. The at least one delay block may be configured to timing the waveform data and metadata such that the total delay of the waveform processing path corresponds to the total delay of the metadata processing path. In particular, said at least one delay block may be a waveform delay block configured to introduce additional delay into the waveform processing path such that the total delay of the waveform processing path corresponds to the total delay of the metadata processing path. Alternatively, or in addition, said at least one delay unit may be configured to timing the waveform data and metadata such that the waveform data and metadata are timely output to the audio encoder access unit generation unit to generate single access block for waveform data and metadata. In particular, the waveform data and metadata can be provided such that a single access block can be generated without the need for a buffer to buffer the waveform data and/or metadata.

The audio encoder may comprise an analysis unit configured to generate a plurality of subband signals per audio signal frame, wherein the plurality of subband signals may include a plurality of lowband signals indicative of a lowband signal. The audio encoder may comprise a despreader configured to compress the plurality of low band signals with a compression function to obtain a plurality of compressed low band signals. The waveform data may indicate a plurality of low band compressed signals, and the metadata may indicate a compression function used by the despreader. The metadata indicating the spectrum envelope of the high band signal may be applicable to the same audio element as the metadata indicating the compression function. In other words, the metadata indicating the spectrum envelope of the high band signal may be synchronized with the metadata indicating the compression function.

In accordance with yet another aspect, a data stream is described containing a sequence of access blocks for a sequence of audio frames, respectively. An access block from a sequence of access blocks contains waveform data and metadata. The waveform data and metadata are associated with the same particular frame of the audio frame sequence. The waveform data and metadata may indicate the reconstructed frame of a particular frame. In one example, a particular audio frame comprises a low band signal and a high band signal, wherein the waveform data indicates the low band signal and wherein the metadata indicates the spectrum envelope of the high band signal. The metadata may allow an audio decoder to generate a high band signal from a low band signal using an HFR scheme. Alternatively, or in addition, the metadata may indicate the compression function applied to the lower band signal. Therefore, the metadata may allow the audio decoder to perform dynamic range expansion of the received low band signal (using the inverse function of the compression function).

In accordance with yet another aspect, a method for determining a reconstructed audio frame from an access block of a received data stream is described. The access block contains waveform data and metadata, wherein the waveform data and metadata are associated with the same reconstructed audio frame. In one example, the reconstructed audio frame comprises a low band signal and a high band signal, wherein the waveform data indicates the low band signal (e.g., frequency coefficients describing the low band signal), and wherein the metadata indicates the spectrum envelope of the high band signal (e.g., , about scaling factors for multiple ranges of the scaling factor of the high band signal). The method includes generating a plurality of subband waveforms from the waveform data and generating decoded metadata from the metadata. In addition, the method includes timing a plurality of subband waveforms and decoded metadata, as described herein. In addition, the method includes generating a reconstructed audio frame from a time-consistent plurality of subband waveforms and decoded metadata.

In accordance with yet another aspect, a method for encoding a frame of an audio signal into an access unit of a data stream is described. The audio frame is encoded such that the access block contains waveform data and metadata. The waveform data and metadata indicate a frame reconstructed from an audio frame. In one example, an audio frame contains a low band signal and a high band signal, and the frame is encoded such that the waveform data indicates the low band signal and such that the metadata indicates the spectrum envelope of the high band signal. The method includes generating waveform data per audio frame, such as a low band signal, and generating metadata per audio frame, such as a high band signal and a low band signal (eg, in accordance with an HFR scheme). Further, the method includes timing the waveform data and metadata such that an access unit for an audio frame contains waveform data and metadata for the same audio frame.

According to yet another aspect, a software system program is described. A software system program may be designed to be executed on a processor and to implement the steps of the method set forth herein when executed on the processor.

According to another aspect, a storage medium (non-volatile storage medium) is described. The storage medium may comprise a software system program for execution on a processor and for implementing the steps of the method set forth herein when executed on the processor.

In accordance with yet another aspect, a computer program product is described. The computer program may contain executable instructions for implementing the steps of the method set forth herein when executed on a computer.

It should be noted that the methods and systems, including their preferred embodiments set forth in this patent application, can be used independently or in combination with other methods and systems described in this document. In addition, all aspects of the methods and systems set forth in this patent application can be arbitrarily combined. In particular, the features of the claims can be combined with each other in an arbitrary manner.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is clearly explained below with reference to the accompanying drawings, in which:

in fig. 1 is a block diagram of an example audio decoder;

in fig. 2a is a block diagram of another example of an audio decoder;

in fig. 2b is a block diagram of an example audio encoder; and

in fig. 3a is a block diagram of an example of an audio decoder that is capable of deploying audio;

in fig. 3b is a block diagram of an example of an audio encoder that is configured to compress audio; and

fig. 4 illustrates an example of framing a sequence of audio signal frames.

DETAILED DESCRIPTION OF THE INVENTION

As stated above, this document relates to metadata negotiation. Next, the metadata negotiation is set forth in relation to the MPEG HE (High Efficiency) AAC (Advanced Audio Coding) scheme. However, it should be noted that the principles of metadata negotiation that are described in this document are also applicable to other encoding/decoding systems. In particular, the metadata negotiation schemes described herein are applicable to audio encoding/decoding systems that use HFR (High Frequency Recovery) and/or SBR (Spectral Range Copy) and that transmit HFR/SBR metadata from an audio encoder to an appropriate audio decoder. In addition, the metadata negotiation schemes that are described herein are applicable to audio coding/decoding systems that use subband applications (namely, QMF). One example of such an application is SBR. Other examples are A-joint, post-processing, etc. The following metadata negotiation schemes are described in relation to SBR metadata negotiation. However, it should be noted that the metadata negotiation schemes are also applicable to other types of metadata, namely other types of metadata in the subband domain.

The HE-AAC MPEG data stream contains SBR metadata (also called A-SPX metadata). The SBR metadata in a particular encoded frame of a data stream (also referred to as an AU (Access Unit) of a data stream) typically refers to past waveform (W) data. SBR metadata and waveform data contained in a data stream AU generally do not correspond to the same frame of the original audio signal. This is because after decoding the waveform data, the waveform data is fed to several processing steps (such as IMDCT (Inverse Modified Discrete Cosine Transform) and QMF (Quadrature Mirror Filter) analysis) which introduce signal delay. The moment the SBR metadata is applied to the waveform data, the SBR metadata is consistent with the processed waveform data. In this regard, SBR metadata and waveform data are inserted into the MPEG HE-AAC data stream such that the SBR metadata reaches the audio decoder when the SBR metadata is needed to process the SBR in the audio decoder. This form of metadata delivery may be referred to as "Just in Time" (JIT) metadata delivery because the SBR metadata is inserted into the data stream in such a way that the SBR metadata can be directly applied in the audio decoder's signaling or processing chain.

JIT delivery of metadata may be appropriate for the traditional chain "encode - transmit - decode" in order to reduce the overall encoding delay and to reduce the memory requirements of the audio decoder. However, the connection of the data stream along the transmission path may lead to a mismatch between the waveform data and the corresponding SBR metadata. This mismatch can lead to audio artifacts at the junction because the audio encoder uses incorrect SBR metadata to copy the spectral range.

In view of the foregoing, it is desirable to provide an audio encoding/decoding system that enables the connection of data streams while simultaneously maintaining a low overall encoding delay.

In FIG. 1 is a block diagram of an example audio decoder 100 that solves the above technical problem. In particular, audio decoder 100 in FIG. 1 enables the decoding of data streams from the AU 110 that contain waveform data 111 of a particular segment (eg, frame) of an audio signal and that contain corresponding metadata 112 of a particular audio segment. By providing audio decoders 100 that decode data streams containing AUs 110 using time-consistent waveform data 111 and associated metadata 112, a stable data stream connection is provided. In particular, it is guaranteed that the data stream can be connected in such a way that the corresponding pairs of waveform data 111 and corresponding metadata 112 are stored.

The audio decoder 100 includes a delay block 105 in the waveform data processing chain 111 . The delay block 105 may be installed after or after the MDCT synthesis block 102 and before or before the QMF synthesis block 107 in the audio decoder 100. In particular, the delay block 105 may be installed before the metadata application block 106 (for example, by the SBR block 106), which is configured to apply the decoded metadata 128 to or before the processed waveform data. A delay block 105 (also called a waveform delay block 105) is configured to apply a delay (called a waveform delay) to the processed waveform data. The waveform delay is preferably chosen such that the total processing delay of the waveform processing chain or waveform processing path (e.g., from MDCT synthesis block 102 to metadata application at metadata application block 106) adds up to exactly one frame (or an integer multiple thereof) . In this case, the parametric control data may be delayed by a frame (or an integer multiple thereof), and matching is achieved in AU 110.

In FIG. 1 shows the components of an example audio decoder 100. The waveform data 111 received from the AU 110 is decoded and dequantized in a decoder and dequantizer 101 to obtain a plurality of frequency coefficients 121 (in the frequency domain). A plurality of frequency coefficients 121 is synthesized into a lower band (time domain) signal 122 by a frequency domain to time domain transform (e.g., inverse MDCT - Modified Discrete Cosine Transform) applied in a lower band synthesis block 102 (e.g., MDCT synthesis block) . Then, the lower band signal 122 is converted into a plurality of lower band signals 123 by the analysis unit 103 . The analysis unit 103 may be configured to apply a quadrature mirror filter (QMF) bank to the low band signal 122 to obtain a plurality of low band signals 123. The metadata 112 is typically applied to the low band signal set 123 (or a transposed version thereof).

The metadata 112 from the AU 110 is decoded and dequantized in a decoder and dequantizer 108 to obtain the decoded metadata 128. In addition, the audio decoder 100 may include an additional delay block 109 (called a metadata delay block 109) that is configured to apply a delay (called a metadata delay). ) to the decoded metadata 128. The metadata delay may correspond to an integer multiple of the frame length N, eg, D ₁ =N, where D ₁ is the metadata delay. In this regard, the total delay of the metadata processing chain corresponds to D ₁ , ie, D ₁ =N.

To allow processed waveform data (i.e., delayed low band signal set 123) and processed metadata (i.e., delayed decoded metadata 128) to be sent to metadata applicator 106 at the same time, the total processing chain (or path) delay the waveform must match the total delay of the metadata processing chain (or path) (ie, D ₁ ). In the waveform processing chain, low band synthesis block 102 typically introduces a delay equal to N/2 (ie, half the frame duration). Analysis unit 103 typically introduces a constant delay (eg, 320 samples). In addition, prediction (i.e., a constant shift between metadata and waveform data) may need to be taken into account. In the case of HE-AAC MPEG, eg SBR, the prediction may correspond to 384 samples (represented by prediction block 104). The prediction block 104 (which may also be referred to as the prediction delay block 104) may be configured to delay the waveform data 111 (eg, delay the plurality of low band signals 123) by a constant SBR prediction delay. The prediction delay allows the corresponding audio encoder to determine the SBR metadata based on the subsequent audio frame.

To provide a total metadata processing chain delay that corresponds to a total waveform processing chain delay, the waveform delay D ₂ should be such that:

D ₁ \u003d 320 + 384 + D ₂ + N / 2,

those. D ₂ =N/2-320-384 (in the case of D ₁ =N).

Table 1 shows waveform delays D ₂ for a variety of different durations N. It can be seen that the maximum waveform delay D ₂ for various HE-AAC durations N is 928 samples, with a total maximum decoder delay time of 2177 samples. In other words, matching the waveform data 111 and the corresponding metadata 112 in a single AU 110 results in an additional PCM delay of maximum 928 samples. For frame size block N=1920/1536, metadata is delayed by 1 frame, and for frame sizes N=960/768/512/384, metadata is delayed by 2 frames. This means that the playback delay in the audio decoder 100 increases depending on the block size N, and the total encoding delay by 1 or 2 full frames. The maximum PCM delay in the respective audio encoder is 1664 samples (corresponding to the inherent delay time of the audio decoder 100).

Table 1 N Reverse MDCT (N/2) QMF Analysis SBR forecast Own delay time (Σ) _D2 Number of frames _D1 QMF synthesis Total Decoder Delay Time 1920 960 320 384 1664 256 one 1920 257 2177 1536 768 320 384 1472 64 one 1536 257 1793 960 480 320 192 992 928 2 1920 257 2177 768 384 320 192 896 640 2 1536 257 1793 512 256 320 192 768 256 2 1024 257 1281 384 192 320 192 704 64 2 768 257 1025

In this regard, this paper proposes to address the disadvantage of JIT metadata by applying signal-aligned metadata (SAM) 112 that is aligned with the corresponding waveform data 111 in a single AU 110. In particular, it is proposed to introduce one or more of the additional delay blocks in audio decoder 100 and/or to a corresponding audio encoder such that each encoded frame (or AU) contains metadata (e.g., A-SPX) that it uses in a subsequent processing step, e.g., in a processing step when the metadata is applied to the underlying based on waveform data.

It should be noted that - in principle - one can consider applying a metadata delay D ₁ that corresponds to a fraction of the frame duration N. In this case, the total encoding delay can possibly be reduced. However, as shown in FIG. 1, the metadata delay D ₁ is applied in the QMF region (ie, in the subband region). With this in mind, and given that the metadata 112 is typically set only once per frame, i.e. given that the metadata 112 typically contains one special parameter set per frame, introducing a metadata delay D ₁ that corresponds to a fraction of the frame duration N can lead to timing problems with respect to the waveform data 111. On the other hand, the D ₂ waveform delay is applied in the time domain (as shown in FIG. 1), where delays that correspond to a fraction of a frame can be realized with high accuracy (for example, by delaying the time domain signal by a number of samples , which corresponds to the delay D ₂ of the waveform). Therefore, it is useful to delay the metadata 112 by integer multiples of a frame (with the frame corresponding to the lowest temporal resolution for which the metadata 112 is defined) and to delay the waveform data 111 by a waveform delay D ₂ , which can take on arbitrary values. The metadata delay D ₁ that corresponds to a fraction of the frame duration N can be implemented in the subband domain with high accuracy, and the waveform delay D ₂ that corresponds to an arbitrary sample multiple can be implemented in the time domain with high accuracy. Therefore, the combination of the metadata delay D ₁ and the waveform delay D ₂ ensures accurate synchronization of the metadata 112 and the waveform data 111.

Applying a metadata delay D ₁ that corresponds to a fraction of the frame duration N can be implemented by resampling the metadata 112 in accordance with the metadata delay D ₁ . However, resampling the metadata 112 typically involves significant computational overhead. In addition, resampling the metadata 112 may corrupt the metadata 112, thereby affecting the quality of the reconstructed audio frame. In view of this, it is advisable - taking into account the computational efficiency and taking into account the sound quality - to limit the metadata delay D ₁ to integer multiples of the duration N of the frame.

In FIG. 1 also shows the post-processing of the delayed metadata 128 and the delayed set of lower band signals 123. The metadata applicator 106 is configured to generate a plurality of (e.g., scaled) high band subband signals 126 based on a plurality of low band signals 123 and based on metadata 128. To this end, the metadata applicator 106 may be configured to transpose one or more of the plurality of low band signals 123 to generate a plurality of high band subband signals. The transposition may include the process of copying said one or more of the plurality of low range signals 123. In addition, the metadata applicator 106 may be configured to apply metadata 128 (eg, scaling factors contained in metadata 128) to a plurality of highband subband signals to generate a plurality of scaled highband subband signals 126. The set of scaled high-band sub-band signals 126 are typically scaled by scaling factors such that the spectrum envelope of the set of scaled high-band sub-band signals 126 reproduces the spectrum envelope of the high-band signal of the original audio frame (which corresponds to the reconstructed audio frame 127 generated at based on a plurality of lower band signals 123 and on a plurality of scaled sub-band signals 126 of a higher band).

In addition, audio decoder 100 includes a synthesis unit 107 configured to generate a reconstructed audio frame 127 from a plurality of low band signals 123 and from a plurality of scaled high band subband signals 126 (eg, using a set of inverse QMFs).

In FIG. 2a is a block diagram of yet another example of audio decoder 100. Audio decoder 100 in FIG. 2a contains the same components as the audio decoder in FIG. 1. In addition, examples of components 210 for processing multi-channel audio are shown. It can be seen that in the example of Fig. 2a, the waveform delay block 105 is located immediately after the inverse MDCT block 102. Determining the reconstructed audio frame 127 may be performed for each channel of the multi-channel audio (eg, 5.1 or 7.1 multi-channel audio).

In FIG. 2b is a block diagram of an example audio encoder 250 corresponding to audio decoder 100 in FIG. 2a. The audio encoder 250 is configured to generate a data stream comprising an AU 110 that transmits pairs of respective waveform data 111 and metadata 112. The audio encoder 250 includes a metadata processing chain 256, 257, 258, 259, 260 to determine the metadata. The metadata processing chain may include a metadata delay block 256 for aligning the metadata with the corresponding waveform data. In the example shown, the metadata delay block 256 of the audio encoder 250 does not introduce any additional delay (because the delay introduced by the metadata processing chain is greater than that introduced by the waveform processing chain).

In addition, the audio encoder 250 includes a waveform processing circuit 251, 252, 253, 254, 255 configured to determine waveform data from the original audio signal at the input of the audio encoder 250. The waveform processing circuit includes a waveform delay block 252, configured to introduce additional delay into the waveform processing chain in order to align the waveform data with the corresponding metadata. The delay introduced by the waveform delayer 252 may be such that the total delay of the metadata processing chain (including the waveform delay introduced by the waveform delayer 252) corresponds to the total delay of the waveform processing chain. In the case of a frame duration of N=2048, the delay of the waveform delay block 252 may be 2048-320=1728 samples.

In FIG. 3a shows a fragment of the audio decoder 300 containing the block 301 deployment. Audio decoder 300 in FIG. 3a may correspond to audio decoder 100 in FIG. 1 and/or 2a and further comprises a deployer 301 that is configured to determine a plurality of deployed low band signals from the plurality of low band signals 123 using one or more of the deployment parameters 310 obtained from the decoded metadata 128 of the access unit 110. Typically, said one or more of the deployment parameters 310 are associated with the SBR metadata (eg, A-SPX) contained in the access block 110 . In other words, the specified one or more of the deployment parameters 310 generally apply to the same fragment or audio element as the SBR metadata.

As noted above, the metadata 112 of the access unit 110 is generally associated with the waveform data 111 of an audio frame, the frame containing a predetermined number N of samples. The SBR metadata is typically determined based on a plurality of lowband signals (also referred to as a plurality of subband waveforms), wherein the plurality of lowband signals may be determined using QMF analysis. QMF analysis gives a time-frequency representation of an audio frame. In particular, N audio frame samples may be represented by Q (eg, Q=64) low band signals, each containing N/Q slots or slots. For a frame with N=2048 samples and for Q=64, each low band signal contains N/Q=32 slots.

In the case of a transient in a particular frame, it may be useful to determine the SBR metadata based on the samples of the immediately following frame. This feature is called SBR prediction. In particular, the SBR metadata may be determined based on a predetermined number of slots over the next frame. As an example, up to 6 intervals of the next frame can be taken into account (ie, Q*6=384 samples).

The use of the SBR prediction is illustrated in FIG. 4, which shows a sequence of audio frames 401, 402, 403 using different framing 400, 430 for an SBR or HFR scheme. In the case of framing 400, the SBR/HFR scheme does not take advantage of the flexibility provided by the SBR prediction. However, a permanent shift, i.e. the constant SBR prediction delay 480 is used to ensure that the SBR prediction is used. In the example shown, the constant shift corresponds to 6 time slots. As a result of this constant shift 480, the metadata 112 of a particular access block 110 of a particular frame 402 is partially applicable to the time slots of the waveform data 111 contained in the access block 110 that precedes the particular access block 110 (and that is associated with the immediately preceding frame 401). This is illustrated by a shift between SBR metadata 411, 412, 413 and frames 401, 402, 403. Therefore, SBR metadata 411, 412, 413 contained in access block 110 can be applied to waveform data 111 that is shifted by delay 480 SBR forecast. The SBR metadata 411, 412, 413 is applied to the waveform data 111 to obtain reconstructed frames 421, 422, 423.

Framing 430 uses the SBR prediction. It can be seen that SBR metadata 431 is applicable to more than 32 timeslots of waveform data 111, for example, due to the occurrence of a transient in frame 401. On the other hand, SBR metadata 432 is applicable to less than 32 timeslots of waveform data 111. Metadata 433 SBR, in turn, is applicable to 32 time intervals. Therefore, the SBR prediction provides flexibility regarding the temporal resolution of the SBR metadata. It should be noted that despite the use of the SBR prediction, and despite the applicability of the SBR metadata 411, 412, 413, the reconstructed frames 421, 422, 423 are generated with a constant offset 480 relative to the frames 401, 402, 403.

The audio encoder may be configured to determine the SBR metadata and said one or more of the deployment parameters using the same audio fragment or element. Therefore, if the SBR metadata is determined using the SBR prediction, the specified one or more of the deployment parameters may be determined and may be applicable for the same SBR prediction. In particular, said one or more of the deployment parameters may be applicable for the same number of time slots as the corresponding SBR metadata 431, 432, 433.

Deployer 301 may be configured to apply one or more of the sweep gains to a plurality of low band signals 123, said one or more of the sweep factors typically being dependent on said one or more of the sweep parameters 310. In particular, said one or more deployment parameters 310 may influence one or more of the compression/expansion rules that are used to determine said one or more of the deployment gains. In other words, said one or more of the deployment parameters 310 may indicate the compression function that is used by the compression unit of the respective audio encoder. Said one or more of the deployment parameters 310 may allow an audio decoder to define an inverse function with respect to this compression function.

Said one or more of the deployment parameters 310 may include a first deployment parameter indicating whether the corresponding audio encoder has compressed the plurality of low band signals. If no compression has been applied, no expansion by the audio decoder will be applied. In this regard, the first deployment option can be used to enable or disable the companding capability.

Alternatively or in addition, said one or more of the spreading parameters 310 may include a second spreading parameter indicating whether the same one or more of the spreading gains should be applied to all channels of the multi-channel audio signal. In this regard, the second deployment parameter may switch the companding capability between per-channel application and multi-channel application.

Alternatively or in addition, said one or more of the spreading parameters 310 may include a third spreading parameter indicating whether to apply the same one or more of the spreading gains for all time slots of the frame. In this regard, the third deployment parameter can be used to control the temporal resolution of the companding opportunity.

Using said one or more of the expander parameters 310, the expander 301 can determine the plurality of deployed low band signals by applying an inverse function to the compression function applied in the corresponding audio encoder. The compression function that is applied in the corresponding audio encoder is reported to the audio decoder 300 using the specified one or more of the deployment parameters 310.

Deployment block 301 may be installed after prediction delay block 104 . This ensures that the specified one or more of the deployment parameters 310 are applied to the correct element of the set of low band signals 123 . In particular, this ensures that said one or more of the deployment parameters 310 are applied to the same member of the low band signal set 123 as the SBR parameters (at SBR application block 106). In this regard, the deployment is guaranteed to operate with the same temporal framing 400, 430 as the SBR scheme. In view of the SBR prediction, the framing 400, 430 may include a variable number of slots, and as a result, the deployment may operate with a variable number of slots (as described with respect to FIG. 4). By placing the expander 301 after the prediction delay 104, it is ensured that the proper framing 400, 430 is applied to the specified one or more of the deployment parameters. As a result, a high quality audio signal can be provided even after the connection point.

In FIG. 3b shows a fragment of an audio encoder 350 containing a despreader 351. Audio encoder 350 may include components of audio encoder 250 in FIG. 2b. The compressor 351 may be configured to compress (eg, reduce dynamic range) a plurality of lower band signals with a compression function. In addition, the despreader 351 may be configured to determine one or more of the expander parameters 310 that indicate the compression function used by the despreader 351 to cause the corresponding expander 301 of the audio decoder 300 to apply an inverse function to the compression function.

The compression of the plurality of low band signals may be performed after the SBR prediction 258 . In addition, the audio encoder 350 may include an SBR framer 353 that is configured to provide an SBR metadata definition for the same audio element as specified one or more of the deployment parameters 310. In other words, the SBR framing unit 353 may operate the SBR scheme with the same framing 400, 430 as the companding scheme. Since the SBR scheme can operate on extended frames (eg, in the case of transients), the companding scheme may also operate on extended frames (containing additional time slots).

This document describes an audio encoder and a corresponding audio decoder that encodes an audio signal into a sequence of time-aligned AUs containing waveform data and metadata associated with a sequence of audio segments, respectively. The use of time-aligned AUs ensures the connection of data streams with reduced artifacts at the connection points. In addition, the audio encoder and the audio decoder are designed such that the data streams to be connected are processed in a computationally efficient manner and such that the overall encoding delay remains small.

The methods and systems described herein may be implemented in software, firmware, and/or hardware. Some components may, for example, be implemented as software running on a digital signal processor or microprocessor. Other components may, for example, be implemented in hardware and/or as ASICs. The signals encountered in the described methods and systems may be stored in media such as random access memory or optical storage media. They may be transmitted over networks such as radio networks, satellite networks, wireless networks, or wired networks such as the Internet. Typical devices using the methods and systems described herein are portable electronic devices or other consumer equipment that are used to store and/or reproduce audio signals.

Claims (16)

1. An audio decoder configured to determine a reconstructed audio frame from an access block of a received data stream; wherein the access block contains waveform data and metadata; wherein the waveform data and metadata are associated with the same reconstructed audio frame; wherein the audio decoder contains a waveform processing path configured to generate a plurality of subband waveforms from the waveform data; - a metadata processing path configured to generate decoded metadata from the metadata; and - a metadata application and synthesis unit configured to generate a reconstructed audio signal frame from the plurality of subband waveforms and from the decoded metadata; wherein the waveform processing path includes at least one waveform delay block configured to apply a waveform delay to a waveform that is represented in the time domain, and/or the metadata processing path includes at least one metadata delay block, wherein the waveform delay block and/or the metadata delay block are configured to time-match the plurality of subband waveforms and decoded metadata, and wherein said at least one analysis block is configured to include a constant delay that is independent of length N of the reconstructed frame of the audio signal. 2. The audio decoder according to claim 1, wherein the constant delay introduced by the analysis unit corresponds to 320 audio samples. 3. The audio decoder of claim 1, wherein the total delay of the metadata processing path depends on one of: an encoded bitstream or a predetermined prediction between metadata and waveform data. 4. The audio decoder of claim 1, wherein the waveform delay block and/or the metadata delay block are configured to timing the plurality of subband waveforms and the decoded metadata such that the total delay of the waveform processing path corresponds to the total delay of the metadata processing path. 5. The audio decoder of claim 1, wherein the waveform delay block and/or the metadata delay block are configured to time-align the plurality of subband waveforms and decoded metadata such that the plurality of subband waveforms and decoded metadata are output for processing by the block. application and synthesis of metadata. 6. Method for determining the restored frame of the audio signal by the access block of the received data stream; wherein the access block contains waveform data and metadata; wherein the waveform data and metadata are associated with the same reconstructed audio frame; wherein the method includes - generating a plurality of subband waveforms from the waveform data; - generating decoded metadata from metadata; - timing of multiple forms of subband signals and decoded metadata; and - generating a reconstructed audio frame from a time-consistent set of sub-band waveforms and decoded metadata; wherein generating the plurality of subband waveforms from the waveform data comprises applying a waveform delay to a waveform that is represented in the time domain and including a constant delay that is independent of the length N of the reconstructed audio frame. 7. A storage medium configured to perform the method of claim 6 when executed by the processor.

Images