RU2825309C2 - Multiple-delay audio encoding format - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: present document describes a method of encoding an audio signal. Method includes: generating a plurality of subband audio signals based on the audio signal; determining the envelope of the spectrum of the audio signal; determining, for the audio signal of each subband, autocorrelation information for the subband audio signal based on the autocorrelation function of the subband audio signal; and generating an encoded representation of the audio signal, wherein the encoded representation comprises a representation of the envelope of the spectrum of the audio signal and a representation of autocorrelation information for the plurality of subband audio signals.

EFFECT: high efficiency of encoding while reducing the amount of data for encoding audio content.

32 cl, 8 dwg

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from the following priority applications: U.S. Provisional Application No. 62/889,118 (Reference: D19076USP1), filed August 20, 2019, and European Application No. 19192552.8 (Reference: D19076EP), filed August 20, 2019, which are incorporated herein by reference.

AREA OF TECHNOLOGY

The present invention generally relates to a method for encoding an audio signal into an encoded representation and a method for decoding the audio signal from the encoded representation.

Although certain embodiments will be described herein with particular reference to the present invention, it should be understood that the present invention is not limited to such scope of use and may be applicable in broader contexts.

BACKGROUND OF THE INVENTION

Any discussion of the current state of the art throughout this specification should in no way be construed as an admission that such state of the art is widely known or forms part of the general knowledge in the art.

In high-quality audio coding systems, most of the information typically describes the detailed properties of the waveforms. A smaller portion of the information is used to describe more statistically specified features, such as energies in frequency bands, or control data intended to shape the quantization noise according to known properties of simultaneous auditory masking (e.g., additional information in an MDCT-based waveform encoder that conveys the quantization step size and range information needed to correctly dequantize the data representing the waveform in the decoder). These high-quality audio coding systems, however, require comparatively large amounts of data to encode the audio content, i.e., are characterized by comparatively low coding efficiency.

There is a need for audio coding methods and apparatus that can encode audio data with improved coding efficiency.

ESSENCE OF THE INVENTION

The present invention provides a method for encoding an audio signal, a method for decoding an audio signal, an encoder, a decoder, a computer program and a machine-readable storage medium.

According to a first aspect of the present invention, a method for encoding an audio signal is provided. The encoding may be performed for each of a plurality of consecutive parts (e.g., groups of samples, segments, frames) of the audio signal. In some implementations, the parts may overlap one another. An encoded representation may be generated for each such part. The method may include generating a plurality of sub-band audio signals based on the audio signal. Generating a plurality of sub-band audio signals based on the audio signal may include spectral decomposition of the audio signal, which may be performed using a bandpass filter bank (BPF). The frequency resolution of the filter bank may be related to the frequency resolution of the human auditory system. For example, the BPF filters may be complex-valued BPF filters. Alternatively, generating a plurality of sub-band audio signals based on the audio signal may include spectral and/or temporal smoothing of the audio signal, optionally windowing the smoothed audio signal using a window function and spectral decomposition of the resulting signal into a plurality of sub-band audio signals. The method may further include determining a spectral envelope of the audio signal. The method may further include determining, for the audio signal of each sub-band, autocorrelation information for the sub-band audio signal based on the autocorrelation function (ACF) of the sub-band audio signal. The method may also further include generating an encoded representation of the audio signal, wherein the encoded representation comprises a representation of the spectral envelope of the audio signal and a representation of the autocorrelation information for the plurality of sub-band audio signals. For example, the encoded representation may relate to a portion of a bitstream. In some implementations, the encoded representation may also contain waveform information related to the waveform of the audio signal and/or one or more waveforms of the subband audio signals. The method may further include outputting the encoded representation.

The proposed method, implemented as described above, provides an encoded representation of an audio signal that is characterized by extremely high coding efficiency (i.e., requires very low bit rates for encoding the audio), but at the same time contains the appropriate information to achieve very high tonal quality after reconstruction. This is done by providing, in addition to the spectral envelope, also autocorrelation information for a plurality of sub-bands of the audio signal. In particular, it has been shown that two values per sub-band are sufficient to achieve high tonal quality: one delay value and one autocorrelation value.

In some embodiments, the autocorrelation information for the audio signal of a given sub-band may comprise a lag value for the audio signal of the corresponding sub-band and/or an autocorrelation value for the audio signal of the corresponding sub-band. Preferably, the autocorrelation information may comprise both a lag value for the audio signal of the corresponding sub-band and an autocorrelation value for the audio signal of the corresponding sub-band. Here, the lag value may correspond to a delay value (e.g., abscissa) at which the autocorrelation function reaches a local maximum, and the autocorrelation value may correspond to said local maximum (e.g., ordinate).

In some embodiments, the spectral envelope may be determined with a first update rate, and the autocorrelation information for a plurality of sub-band audio signals may be determined with a second update rate. In this case, the first and second update rates may differ from each other. Update rates may also be referred to as sampling rates. In one such embodiment, the first update rate may be higher than the second update rate. In addition, different update rates may be applied to different sub-bands, i.e., the update rates for the autocorrelation information for the audio signals of different sub-bands may differ from each other.

By reducing the update rate of the autocorrelation information compared to the update rate for the spectral envelope, it is possible to further increase the coding efficiency of the proposed method without affecting the tonal quality of the reconstructed audio signal.

In some embodiments, generating a plurality of sub-band audio signals may include applying spectral and/or temporal smoothing to the audio signal. Generating a plurality of sub-band audio signals may further include windowing the smoothed audio signal using a window function. Generating a plurality of sub-band audio signals may also further include spectral decomposition of the windowed smoothed audio signal into a plurality of sub-band audio signals. In this case, spectral and/or temporal smoothing of the audio signal may include, for example, generating a perceptually weighted residual LPC signal of the audio signal.

In some embodiments, generating a plurality of sub-band audio signals may include spectral decomposition of the audio signal. Then, determining an autocorrelation function for the audio signal of the given sub-band may include determining a sub-band envelope of the sub-band audio signal. Determining the autocorrelation function may further include smoothing the envelope of the sub-band audio signal based on the sub-band envelope. The sub-band envelope may be determined by taking amplitude values of the windowed sub-band audio signal. Determining the autocorrelation function may further include windowing the sub-band audio signal with the smoothed envelope using a window function. Determining the autocorrelation function may also further include determining (e.g. calculating) the autocorrelation function of the windowed sub-band audio signal with the smoothed envelope. The autocorrelation function can be defined for a real-valued sub-band signal (windowed, with a smoothed envelope).

Another aspect of the present invention relates to a method for decoding an audio signal from an encoded representation of the audio signal. The encoded representation may include a representation of a spectral envelope of the audio signal and a representation of autocorrelation information for each of a plurality of audio signals of subbands (or generated from) the audio signal. The autocorrelation information for an audio signal of a given subband may be based on an autocorrelation function of the audio signal of the subband. The method may include receiving an encoded representation of the audio signal. The method may further include extracting the spectral envelope and (several fragments of) the autocorrelation information from the encoded representation of the audio signal. The method may also further include determining a reconstructed audio signal based on the spectral envelope and the autocorrelation information. The reconstructed audio signal may be defined so that the autocorrelation function of each of the plurality of audio signals of the sub-bands (or generated from) the reconstructed audio signal satisfies a condition obtained from the autocorrelation information for the audio signal of the corresponding sub-band (or generated from) the audio signal. For example, the reconstructed audio signal may be defined so that for the audio signal of each sub-band of the reconstructed audio signal, the value of the autocorrelation function of the audio signal of the sub-band (or generated from) the reconstructed audio signal at a delay value (for example, a delay value) indicated by the autocorrelation information for the audio signal of the corresponding sub-band (or generated from) the audio signal, substantially coincides with the autocorrelation value indicated by the autocorrelation information for the audio signal of the corresponding sub-band of the audio signal. This may involve that the decoder may determine the autocorrelation function of the subband audio signals in the same manner as the encoder does. This may include any, some, or all of smoothing, windowing, and normalization. In some implementations, the reconstructed audio signal may be determined such that the autocorrelation information for each of a plurality of subband signals of (or generated from) the reconstructed subband audio signal substantially matches the autocorrelation information for the audio signal of the corresponding subband of (or generated from) the audio signal. For example, the reconstructed audio signal may be defined such that, for example, for the audio signal of each subband of (or generated from) the reconstructed audio signal, the autocorrelation value and the lag value (e.g., the delay value) of the autocorrelation function of the subband signal of the reconstructed audio signal substantially coincide with the autocorrelation value and the lag value indicated by the autocorrelation information for the audio signal of the corresponding subband of (or generated from) the audio signal. This may imply that the decoder can determine the autocorrelation information (i.e., the lag value and the autocorrelation value) for the signal of each subband of the reconstructed audio signal in the same way as the encoder does. Here, the term "substantially coincides" may mean, for example, "matches up to a predetermined limit". In those implementations in which the encoded representation comprises waveform information, the reconstructed audio signal may be defined additionally on the basis of the waveform information. The sub-band audio signals can be obtained, for example, by spectral decomposition of the corresponding audio signal (i.e. the original audio signal on the encoder side or the reconstructed audio signal on the decoder side), or they can be obtained by smoothing, windowing and subsequent spectral decomposition of the corresponding audio signal.

Thus, it can be said that the decoder operates according to the synthesis-by-analysis approach in the sense that it seeks to find a reconstructed audio signal. , which satisfies at least one condition obtained from the encoded representation coded audio signal or for which the coded representation would essentially coincide with the encoded representation original audio signal , Where is the encoding mapping used by the encoder. In other words, we can say that the decoder finds such a decoding mapping , What . It has been found that this "synthesis by analysis" approach produces results that are perceptually very close to the original audio signal if the encoded representation that the decoder is attempting to reproduce contains spectral envelopes and autocorrelation information as described in the present invention.

In some embodiments, the reconstructed audio signal may be determined in an iterative procedure that starts with an initial candidate for the reconstructed audio signal and generates a corresponding intermediate reconstructed audio signal at each iteration. At each iteration, an update mapping may be applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration. The update mapping may be adapted such that the autocorrelation functions of the sub-band audio signals of (or generated from) the intermediate reconstruction of the audio signal become closer to satisfying a condition obtained from the autocorrelation information for the corresponding sub-band audio signals of (or generated from) the audio signal, and/or such that the difference between the measured signal powers of the sub-band audio signals of (or generated from) the reconstructed audio signal and the signal powers for the audio signal of the corresponding sub-band of (or generated from) the audio signal, which are indicated by the spectral envelope, decrease from one iteration to the next. If both the autocorrelation information and the spectral envelope are considered, a corresponding difference metric can be defined for the degree to which these conditions are satisfied and the difference between the signal powers for the sub-band audio signals. In some implementations, the update mapping may be adapted such that the difference between the encoded representation of the intermediate reconstructed audio signal and the encoded representation of the audio signal is successively reduced from one iteration to the next. For this purpose, a corresponding difference metric for the encoded representations (including spectral envelopes and/or autocorrelation information) may be defined and applied. The autocorrelation function of the audio signals of the subbands of (or generated from) the intermediate reconstructed audio signal may be determined in the same way as the encoder does for the audio signals of the subbands of (or generated from) the audio signal. Similarly, the encoded representation of the intermediate reconstructed audio signal may be the encoded representation that would be obtained if the intermediate reconstructed audio signal were subjected to the same encoding technique as that which resulted in the encoded representation of the audio signal.

This iterative approach allows for a simple but effective implementation of the above-mentioned synthesis-by-analysis approach.

In some embodiments, determining the reconstructed audio signal based on the spectral envelope and the autocorrelation information may include applying a generative model based on machine learning, which receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of audio signals of the subbands of the audio signal and generates and outputs the reconstructed audio signal. In those implementations in which the encoded representation comprises waveform information, the generative model based on machine learning may additionally receive waveform information as input. This suggests that the generative model based on machine learning may also be trained using waveform information.

This machine learning-based method enables the above-mentioned synthesis-by-analysis approach to be implemented extremely efficiently and can achieve reconstructed audio signals that are perceptually very close to the original audio signals.

Another aspect of the present invention relates to an encoder for encoding an audio signal. The encoder may comprise a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any of the encoding methods described in the present invention.

Another aspect of the present invention relates to a decoder for decoding an audio signal from an encoded representation of the audio signal. The decoder may comprise a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any of the decoding methods described in the present invention.

Another aspect relates to a computer program comprising instructions for causing a computer executing the instructions to perform steps of a method according to any of the methods described in the present invention.

Another aspect of the present invention relates to a machine-readable storage medium on which a computer program according to the previous aspect is stored.

BRIEF DESCRIPTION OF GRAPHIC MATERIALS

Exemplary embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Fig. 1 is a block diagram illustrating an example of an encoder according to embodiments of the present invention,

Fig. 2 is a block diagram illustrating an example of a coding method according to embodiments of the present invention,

Fig. 3 represents schematic representations of examples of signal forms that may be present in the structure of the coding method according to Fig. 2 ,

Fig. 4 is a block diagram illustrating an example of the synthesis-by-analysis approach for determining a decoding function,

Fig. 5 is a block diagram illustrating an example of a decoding method according to embodiments of the present invention,

Fig. 6 is a block diagram illustrating an example of a step in the decoding method according to Fig. 5 ,

Fig. 7 is a block diagram illustrating another example of an encoder according to embodiments of the present invention, and

Fig. 8 is a block diagram illustrating an example of a decoder according to embodiments of the present invention.

DESCRIPTION OF EXAMPLES OF IMPLEMENTATION OPTIONS

Introduction

High-quality audio coding systems typically require relatively large amounts of data to encode the audio content, i.e., are characterized by relatively low coding efficiency. Although the development of tools such as noise filling and high-frequency restoration have shown that the data describing the waveform can be partially replaced by a smaller set of control data, no high-quality audio codec relies primarily on perceptually significant features. However, increasing computing power and modern advances in machine learning have increased the robustness of audio decoding primarily from arbitrary-format coders. The present invention provides an example of such a format coder.

In a broader sense, the present invention proposes a coding format based on sub-band envelopes based on auditory resolution and additional information. The additional information includes one autocorrelation value and one lag value for each sub-band (and for each update step). The envelopes can be calculated at a first update rate, and the additional information can be sampled at a second update rate. For example, decoding of the coding format can be performed using a "synthesis by analysis" approach, which can be implemented using iterative methods or machine learning-based methods.

Coding

The encoding format (encoded representation) proposed in the present invention can be called a "multiple-delay format" since it provides one delay for each subband (and update stage). Fig. 1 is a block diagram illustrating an example of an encoder 100 for generating an encoding format according to embodiments of the present invention.

The encoder 100 receives a target sound 10, which corresponds to an audio signal to be encoded. The audio signal 10 may comprise a plurality of consecutive or partially overlapping parts (e.g., groups of samples, segments, frames, etc.), which are processed by the encoder. The audio signal 10 is spectrally decomposed into a plurality of audio signals 20 sub-bands in corresponding frequency sub-bands using a filter bank 15. The filter bank 15 may be a bank of bandpass filters (BPF), which, for example, may be complex-valued BPF filters. For audio signals, it is natural to use a BPF filter bank with a frequency resolution that is associated with the human auditory system.

The spectral envelope 30 of the audio signal 10 is extracted in the envelope extraction unit 25. For each sub-band, the power is measured with predetermined time steps as a base model of the auditory envelope or the excitation pattern of the cochlea as a result of the input audio signal, in order to thereby determine the spectral envelope 30 of the audio signal 10. That is, the spectral envelope 30 can be determined based on a plurality of audio signals 20 of the sub-bands, for example by measuring (for example, estimating, calculating) the power of the corresponding signal for each of the plurality of audio signals 20 of the sub-bands. However, the spectral envelope 30 can be determined using any suitable alternative tool, such as, for example, a description of linear predictive coding (LPC). In particular, in some implementations, the spectral envelope can be determined from the audio signal before spectral decomposition using the filter bank 15.

Optionally, the extracted spectral envelope 30 may be down-sampled in a down-sampling unit 35, and the down-sampled spectral envelope 40 (or spectral envelope 30) is output as part of an encoding format, or an encoded representation (the corresponding part), of the audio signal 10.

Reconstructed signals that are reconstructed only from the spectral envelopes may still be characterized by insufficient tonal quality. To solve this problem, the present invention proposes to include one value (i.e., ordinate and abscissa) of the autocorrelation function of the signal (possibly with a smoothed envelope) for each sub-band, which leads to a significant improvement in the sound quality. For this purpose, the audio signals of the 20 sub-bands are optionally smoothed (subject to envelope smoothing) in the divider 45 and input to the autocorrelation unit 55. The autocorrelation unit 55 determines the autocorrelation function (ACF) of its input signal and outputs corresponding fragments of the autocorrelation information 50 for each of the audio signals of the 20 sub-bands (i.e., for each of the sub-bands) based on the ACF of the corresponding audio signals of the 20 sub-bands. The autocorrelation information 50 for a given subrange contains 50 representations (i.e. consists of representations) of the value lags and values autocorrelations. That is, for each sub-range, one lag value is output (e.g. transmitted) as autocorrelation information 50, which is part of the encoded representation and the corresponding (possibly normalized) value autocorrelation (ACF value). Here the value lag corresponds to the delay value at which the ACF reaches a local maximum, and the value autocorrelation corresponds to the specified local maximum. In other words, the autocorrelation information for a given subrange may contain the delay value (i.e., the abscissa) and the autocorrelation value (i.e., the ordinate) of the local maximum of the ACF.

Thus, the encoded representation of the audio signal contains the envelope of the audio signal spectrum and autocorrelation information for each of the sub-ranges. The autocorrelation information for a given sub-range includes representations of the value lags and values autocorrelations. The encoded representation corresponds to the output of the encoder. In some implementations, the encoded representation may further comprise waveform information related to the waveform of the audio signal and/or one or more waveforms of the subband audio signals.

Using the above procedure, the coding function is determined. (or encoding mapping) that maps the input audio signal into its encoded representation.

As noted above, the spectral envelope and the autocorrelation information for the sub-band audio signals may be determined and output at different update rates (sampling rates). For example, the spectral envelope may be determined at a first update rate, and the autocorrelation information for a plurality of sub-band audio signals may be determined at a second update rate different from the first update rate. The spectral envelope representation and the autocorrelation information representations (for all sub-bands) may be written into the bitstream at the corresponding update rates (sampling rates). In this case, the encoded representation may refer to the portion of the bitstream output by the encoder. In this regard, it should be noted that for each time instant, the current spectral envelope and the current set of autocorrelation information fragments (one for each sub-band) are determined by the bitstream and may be taken as the encoded representation. Alternatively, the spectral envelope representation and the autocorrelation information representation (for all subbands) may be updated in the corresponding encoder output units at the corresponding update rates. In this case, each output unit (e.g., encoded frame) of the encoder corresponds to an instance of the encoded representation. The spectral envelope and autocorrelation information representations may be the same across a series of consecutive output units, depending on the corresponding update rates.

Preferably, the first update rate is higher than the second update rate. In one example, the first update rate R ₁ may be equal to R ₁ = 1/(2.5 ms), and the second update rate R ₂ may be equal to R ₂ = 1/(20 ms), so that an updated representation of the spectral envelope is output every 2.5 ms, while the updated representations of the autocorrelation information are output every 20 ms. With respect to parts (e.g., frames) of the audio signal, the spectral envelope may be determined for every n-th part (e.g., for every part), while the autocorrelation information may be determined for every m-th part, where m> n.

The encoded representation(s) may be output as a sequence of frames with a certain frame length. Among other factors, the frame length may depend on the first and/or second refresh rates. If we consider that a frame has a length equal to the first period L ₁ (e.g., 2.5 ms), which corresponds to the first refresh rate R ₁ (e.g., 1/(2.5 ms)), according to L ₁ = 1/R ₁ , then this frame will include one representation of the spectral envelope and a representation of one set of fragments of autocorrelation information (one fragment per sub-band audio signal). For the first and second refresh rates equal to 1/(2.5 ms) and 1/(20 ms), respectively, the autocorrelation information will be the same for eight consecutive frames of the encoded representations. In general, the autocorrelation information would be the same for R ₁ /R ₂ consecutive frames of coded representations, given that R ₁ and R ₂ are suitably chosen to give an integer ratio. On the other hand, assuming that a frame has a length equal to the second period L ₂ (e.g. 20 ms), which corresponds to a second update rate R ₂ (e.g. 1/(20 ms)), according to L ₂ = 1 / R ₂ , this frame will contain a representation of one set of fragments of autocorrelation information, and R ₁ Ь /R ₂ (e.g. eight) representations of the spectral envelopes.

In some implementations, different update rates may even be applied to different sub-bands, i.e. autocorrelation information for different sub-band audio signals may be generated and output at different update rates.

Fig. 2 is a block diagram illustrating an example of a coding method 200 according to embodiments of the present invention. The method, which can be implemented by the encoder 100 described above, receives an audio signal as input.

In step S210, a plurality of sub-band audio signals are generated based on the audio signal. This step may include spectral decomposition of the audio signal, in which case this step may be performed in accordance with the action of the above-described filter bank 15. Alternatively, it may include spectral and/or temporal smoothing of the audio signal, optionally windowing the smoothed audio signal using a window function, and spectral decomposition of the resulting signal into a plurality of sub-band audio signals.

In step S220, the envelope of the spectrum of the audio signal is determined (for example, calculated). This step can be performed in accordance with the action of the envelope extraction unit 25 described above.

In step S230, for the audio signal of each sub-band, autocorrelation information is determined for the audio signal of the sub-band based on the ACF of the audio signal of the sub-band. This step can be performed in accordance with the action of the above-described autocorrelation block 55.

At step S240, an encoded representation of the audio signal is generated. The encoded representation comprises a representation of the spectral envelope of the audio signal and a representation of autocorrelation information for each of the plurality of sub-band audio signals.

Below, examples will be described with details of the implementation of the stages of method 200.

For example, as noted above, generating a plurality of sub-band audio signals may involve spectral decomposition (or be equivalent to spectral decomposition) of the audio signal, such as by means of a filter bank. In this case, determining the autocorrelation function for the sub-band audio signal may involve determining a sub-band envelope for the sub-band audio signal. The sub-band envelope may be determined by taking the amplitude values of the sub-band audio signal. The ACF itself may be calculated for a real-valued sub-band signal (windowed, with a smoothed envelope).

Assuming that the subband filter characteristics are complex-valued, with the Fourier transforms essentially maintained with respect to positive frequencies, the subband signals become complex-valued. The subband envelope can then be determined by taking the amplitude of the complex-valued subband signal. This subband envelope contains the same number of samples as the subband signal and may still be somewhat oscillatory. Optionally, the subband envelope can be downsampled, such as by computing a triangular-window weighted sum of the squares of the envelope in segments of a certain length (e.g., 5 ms long, rising at 2.5 ms, falling at 2.5 ms) for each shift of half a certain length (e.g., 2.5 ms) along the signal, and then taking the square root of this sequence to obtain the downsampled subband envelope. It can be said to correspond to the definition of "root mean square envelope". The triangular window can be normalized such that a constant envelope with a value of one yields a sequence of ones. Other ways of defining the subband envelope are also feasible, such as, for example, in the case of a real-valued subband signal, a half-wave rectification followed by low-pass filtering. In any case, the subband envelopes can be said to carry information about the energy in the subband signals (at the chosen refresh rate).

The sub-band audio signal can then be envelope smoothed based on the sub-band envelope. For example, to obtain a fine-grained (carrier) signal from which to calculate the ACF data, a new envelope signal at full sampling rate can be created by linearly interpolating the down-sampled values and dividing the original (complex-valued) sub-band signals by their linearly interpolated envelope.

The sub-band audio signal with the smoothed envelope may then be windowed using a suitable window function. Finally, the ACF of the windowed sub-band audio signal with the smoothed envelope is determined (e.g., calculated). In some implementations, determining the ACF for the sub-band audio signal may further include normalizing the ACF of the windowed sub-band audio signal with the smoothed envelope using an autocorrelation function of the window function.

In Fig. 3, curve 310 in the upper panel indicates the real value of the windowed subband signal with smoothed envelope, which is used to calculate the ACF. Solid curve 320 in the lower panel indicates the real values of the complex ACF.

The main idea now is to find the largest local maximum of the subband signal ACF among those local maxima that lie above the absolute value ACF of the impulse response of the (complex-valued) subband filter (i.e. the corresponding BPF from the filter bank). At this point, for a subband signal ACF that is complex-valued, one can consider real values of the ACF. Finding the largest local maximum above the absolute value ACF of the impulse response may be necessary to avoid capturing delays that are related to the subband center frequency rather than to the properties of the input signal. As a final refinement, this maximum value can be divided by the maximum ACF of the window function used to window the subband ACF (assuming that the subband signal ACF itself has been normalized, e.g. such that the autocorrelation value for zero delay is normalized to unity). This results in better use of the interval between 0 and 1, where is the maximum tonality.

Accordingly, determining the autocorrelation information for the audio signal of a given sub-band based on the ACF of the sub-band audio signal may further include comparing the ACF of the sub-band audio signal with the ACF of the absolute value of the impulse response of the corresponding band-pass filter associated with the sub-band audio signal. The ACF of the absolute value of the impulse response of the corresponding band-pass filter associated with the sub-band audio signal is indicated by the solid curve 330 in the lower panel of Fig. 3 . The autocorrelation information is then determined based on the highest local maximum of the ACF of the sub-band signal above the ACF of the absolute value of the impulse response of the corresponding band-pass filter associated with the sub-band audio signal. In the lower panel of Fig. 3 the local maxima of the ACF are indicated by crosses, and the selected highest local maximum of the ACF of the sub-band signal above the ACF of the absolute value of the impulse response of the corresponding band-pass filter is indicated by a circle. Optionally, the selected local maximum ACF may be normalized by the ACF value for the window function ACF (assuming that the ACF itself has been normalized, such that the autocorrelation value for zero lag is normalized to unity). The normalized selected highest local maximum ACF is indicated by the asterisk in the lower panel of Fig. 3 , and the dashed curve 340 indicates the window function ACF.

The autocorrelation information determined at this stage may contain the autocorrelation value and the delay value (i.e. the ordinate and abscissa) of the selected (normalized) highest local maximum ACF of the sub-band audio signal.

A similar coding format can be defined in the LPC-based vocoder structure. Also in this case, the autocorrelation information is extracted from a subband signal that is affected by at least some degree of spectral and/or temporal smoothing. Unlike the above example, this is done by creating a (perceptually weighted) LPC residual signal, windowing it, and decomposing it into subbands to produce a plurality of subband audio signals. This is followed by calculating the ACF and extracting the lag value and the autocorrelation value for each subband audio signal.

For example, generating a plurality of sub-band audio signals may include applying spectral and/or temporal smoothing to an audio signal (e.g., by generating a perceptually weighted LPC residual signal based on the audio signal using an LPC filter). This may be followed by windowing the smoothed audio signal using a window function and spectral decomposition of the windowed smoothed audio signal into a plurality of sub-band audio signals. As noted above, the result of the temporal and/or spectral smoothing may correspond to a perceptually weighted LPC residual signal, which is then windowed and spectrally decomposed into sub-bands. The perceptually weighted LPC residual signal may be, for example, a "pink" LPC residual signal.

Decoding

The present invention relates to audio decoding which is based on the "synthesis by analysis" approach. At the most abstract level, it is assumed that a coding mapping is given from signals into the perceptually conditioned region, so that the original sound signal appears as At best, a good predictor of the subjective difference measured by a population of listeners is a simple least-squares-like distortion criterion in the perceptual domain.

One remaining problem is the construction of a decoder. , performing the mapping from (encoded and decoded version) into the sound signal . For this purpose, the concept of "synthesis by analysis" can be used, which involves "finding the waveform that comes closest to generating the desired pattern." The goal is that And should sound similar, so the decoder must solve the inverse problem As for the composition of the mappings, should approximate the left inverse function , this means that . This inverse problem is often ill-posed in the sense that it has many solutions. The possibility of realizing significant bitrate savings lies in the observation that a large number of different waveforms will produce the same audio impression.

Fig. 4 shows a block diagram illustrating an example of the synthesis-by-analysis approach to determining a decoding function (or decoding mapping) , given a coding function (or coding mapping) . Original audio signal 410, is affected by the encoding mapping , 415, which gives the encoded representation , 420, where . Encoded representation can be defined in the field of perception. The goal is to find the decoding function (decoding mapping) , 425, which displays the encoded representation into the restored audio signal , 430, the property of which is that the application of the coding mapping , 435, to the restored sound signal would give a coded representation , 440, essentially identical to the encoded representation . Here the term "essentially matches" can mean, for example, "matches up to a predetermined limit." In other words, given the encoding mapping the goal is to find such a decoding mapping , What .

Fig. 5 is a block diagram illustrating an example of a decoding method 500 according to the "synthesis by analysis" approach according to embodiments of the present invention. The method 500 is a method for decoding an audio signal from an encoded representation of the (original) audio signal. It is assumed that the encoded representation includes a representation of the spectral envelope of the original audio signal and a representation of autocorrelation information for each of a plurality of subband audio signals of the original audio signal. The autocorrelation information for the audio signal of a given subband is based on the ACF of the subband audio signal.

In step S510, an encoded representation of the audio signal is received.

At step S520, a spectral envelope and autocorrelation information are extracted from the encoded representation of the audio signal.

In step S530, a reconstructed audio signal is determined based on the spectral envelope and the autocorrelation information. Here, the reconstructed audio signal is determined such that the autocorrelation function of each of the plurality of sub-band signals of the reconstructed audio signal (substantially) satisfies a condition obtained from the autocorrelation information for the audio signals of the corresponding sub-bands of the audio signal. This condition may consist, for example, in that for the audio signal of each sub-band of the reconstructed audio signal, the ACF value of the audio signal of the sub-band of the reconstructed audio signal at a delay value (for example, a lag value) indicated by the autocorrelation information for the audio signal of the corresponding sub-band of the audio signal substantially coincides with the autocorrelation value indicated by the autocorrelation information for the audio signal of the corresponding sub-band of the audio signal. This may imply that the decoder can determine the ACF of the subband audio signals in the same manner as the encoder does. This may include any, some, or all of smoothing, windowing, and normalization. In one implementation, the reconstructed audio signal may be determined such that, for the audio signal of each subband of the reconstructed audio signal, an autocorrelation value and a lag value (e.g., a delay value) of the ACF of the subband signal of the reconstructed audio signal substantially coincide with the autocorrelation value and the lag value indicated by the autocorrelation information for the audio signal of the corresponding subband of the original audio signal. This may imply that the decoder can determine the autocorrelation information for the signal of each subband of the reconstructed audio signal in the same manner as the encoder does. In implementations in which the encoded representation also includes waveform information, the reconstructed audio signal may be determined additionally based on the waveform information. The audio signals of the sub-bands of the reconstructed audio signal can be generated in the same way as the encoder does. For example, this may involve spectral decomposition or a sequence of smoothing, windowing and spectral decomposition.

Preferably, the determination of the reconstructed audio signal in step S530 also takes into account the spectral envelope of the original audio signal. Then, the reconstructed audio signal can be further determined such that, for the audio signal of each sub-band of the reconstructed audio signal of the sub-band, the measured (e.g. estimated or calculated) signal power for the audio signal of the sub-band of the reconstructed audio signal substantially coincides with the signal power for the audio signal of the corresponding sub-band of the original audio signal, which is indicated by the spectral envelope.

As can be seen from the above, it can be said that the proposed method 500 is based on the “synthesis through analysis” approach in the sense that it seeks to find a reconstructed audio signal , which (essentially) satisfies at least one condition obtained from the encoded representation original audio signal , Where is the encoding mapping used by the encoder. In some implementations, it can even be said that the proposed method operates according to the "synthesis by analysis" approach in the sense that it seeks to find a reconstructed audio signal , for which the encoded representation would essentially coincide with the encoded representation original audio signal . In other words, we can say that the decoding method finds such a decoding mapping , What . Two non-limiting examples of implementations of method 500 will be described below.

Example 1 of implementation: parametric synthesis or iterations for each signal

The inverse problem can be solved using iterative methods given the update mapping , which modifies So is located closer to , how For example, the starting point of the iteration (i.e., the initial candidate for the reconstructed audio signal) may either be a random noise signal (e.g., white noise), or may be determined based on an encoded representation of the audio signal (e.g., a manually performed zero-order approximation). In the latter case, the initial candidate for the reconstructed audio signal may refer to an educated guess that is made based on the spectral envelope and/or autocorrelation information for a plurality of subband audio signals. In those implementations in which the encoded representation includes waveform information, the educated guess may be made additionally based on the waveform information.

In more detail, the reconstructed audio signal in this implementation example is determined in an iterative procedure that starts with an initial candidate for the reconstructed audio signal and generates a corresponding intermediate reconstructed audio signal at each iteration. At each iteration, an update mapping is applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration. The update mapping is chosen such that the difference between the encoded representation of the intermediate reconstructed audio signal and the encoded representation of the original audio signal is successively reduced from one iteration to the next. For this purpose, a corresponding difference metric for the encoded representations (e.g., spectral envelope, autocorrelation information) can be defined and applied to evaluate the difference. The encoded representation of the intermediate reconstructed audio signal may be the encoded representation that would be obtained if the intermediate reconstructed audio signal were subjected to the same encoding scheme as that which resulted in the encoded representation of the audio signal.

In the case where the procedure searches for a reconstructed audio signal that satisfies at least one condition obtained from (several fragments of) autocorrelation information, the update mapping can be selected such that the autocorrelation functions of the audio signals of the intermediate reconstruction sub-bands of the audio signal become closer to satisfying the corresponding conditions obtained from the autocorrelation information for the corresponding audio signals of the sub-bands of the audio signal, and/or such that the difference between the measured signal powers of the audio signals of the sub-bands of the reconstructed audio signal and the signal powers for the audio signal of the corresponding sub-band of the audio signal, which are indicated by the spectral envelope, decrease from one iteration to the next. If both the autocorrelation information and the spectral envelope are taken into account, it is possible to determine a corresponding difference metric for the degree to which these conditions are satisfied and the difference between the signal powers for the audio signals of the sub-bands.

Example 2 of implementation: generative models based on machine learning

Another possibility provided by modern machine learning methods is training a machine learning-based generative model (or generative model for short) for an audio signal. , prepared on the basis of data . That is, given a large set of examples , Where , a parametric conditional distribution is trained from to . Then the decoding algorithm can consist of sampling from the distribution .

This possibility was found to be particularly advantageous for the case in which represents a speech vocoder and is defined by a sequential generative model using samples from a recurrent neural network (RNN). However, other generative models such as variational autoencoders or generative adversarial models are also relevant to this task. Thus, without implied restriction, a generative machine learning model can be one of a recurrent neural network, a variational autoencoder, or a generative adversarial model (e.g., a Generative Adversarial Network (GAN)).

In this example of implementation, determining the reconstructed audio signal based on the spectral envelope and autocorrelation information includes applying a generative model based on machine learning, which receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of audio signals of the subbands of the audio signal and generates and outputs the reconstructed audio signal. In those implementations in which the encoded representation also contains information about the waveform, the generative model based on machine learning can additionally receive as input the information about the waveform.

As described above, a generative machine learning model may contain a parametric conditional distribution , which links encoded representations sound signals and corresponding sound signals with the corresponding probabilities . Then the definition of the reconstructed audio signal may involve sampling from a parametric conditional distribution for coded representation of an audio signal.

In the training stage before decoding, a generative machine learning model can be trained on a dataset of multiple audio signals and corresponding encoded representations of the audio signals. If the encoded representation also contains waveform information, the generative machine learning model can also be trained using the waveform information.

Fig. 6 is a block diagram illustrating an exemplary implementation 600 of step S530 of the decoding method 500 according to Fig. 5. In particular, the implementation 600 relates to an implementation of step S530 for each subband.

At step 610, based on the spectral envelope and the autocorrelation information, a plurality of reconstructed sub-band audio signals are determined. Here, the plurality of reconstructed sub-band audio signals are determined so that for each reconstructed sub-band audio signal, the autocorrelation function of the reconstructed sub-band audio signal satisfies a condition obtained from the autocorrelation information for the audio signal of the corresponding sub-band of the audio signal. In some implementations, the plurality of reconstructed sub-band audio signals are determined so that for each reconstructed sub-band audio signal, the autocorrelation information for the reconstructed sub-band audio signal substantially coincides with the autocorrelation information for the audio signal of the corresponding sub-band.

Preferably, determining the plurality of reconstructed sub-band audio signals in step S610 also takes into account the spectral envelope of the original audio signal. Then, the plurality of reconstructed sub-band audio signals are further determined so that for the reconstructed audio signal of each sub-band, the measured (e.g. estimated, calculated) signal power of the reconstructed sub-band audio signal substantially coincides with the signal power for the audio signal of the corresponding sub-band, which is indicated by the spectral envelope.

In step S620, a reconstructed audio signal is determined based on a plurality of reconstructed sub-band audio signals using spectral synthesis.

The above-described implementation examples 1 and 2 can also be applied in the implementation of step S530 for each sub-band. For implementation example 1, each reconstructed sub-band audio signal can be determined in an iterative procedure that starts with an initial candidate for the reconstructed sub-band audio signal and generates a corresponding intermediate reconstructed sub-band audio signal at each iteration. At each iteration, an update mapping may be applied to the intermediate reconstructed sub-band audio signal to obtain an intermediate reconstructed sub-band audio signal for the next iteration, such that the difference between the autocorrelation information for the intermediate reconstructed sub-band audio signal and the autocorrelation information for the audio signal of the corresponding sub-band is successively reduced from one iteration to the next, or such that the reconstructed sub-band audio signals satisfy to a greater extent the corresponding conditions obtained from the autocorrelation information for the corresponding audio signals of the corresponding sub-bands of the audio signal.

Again, the spectral envelope may also be taken into account at this stage. That is, the update mapping may be such that the (aggregate) difference between the corresponding signal powers of the sub-band audio signals and between the corresponding elements of the autocorrelation information is successively reduced. This may involve defining an appropriate difference metric for estimating the (aggregate) difference. Otherwise, the same explanations as given above for Implementation Example 1 apply to this case.

When applying Example 2 in the implementation of step S530, for each sub-band, determining a plurality of reconstructed sub-band audio signals based on the spectral envelope and the autocorrelation information may include applying a generative model based on machine learning that takes as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of sub-band audio signals of the audio signal and generates and outputs the plurality of reconstructed sub-band audio signals. Otherwise, the same explanations as those given above for Implementation Example 2 apply to this case.

The present invention further relates to encoders for encoding an audio signal, which are configured and adapted to perform the encoding methods described in the present disclosure. An example of such an encoder 700 is schematically illustrated in Fig. 7 in the form of a block diagram. The encoder 700 comprises a processor 710 and a memory 720 coupled to the processor 710. The processor 710 is adapted to perform the steps of the method according to any of the encoding methods described in the present invention. For this purpose, the memory 720 may contain appropriate instructions for execution by the processor 710. The encoder 700 may further comprise an interface 730 for receiving an input audio signal 740 to be encoded and/or outputting an encoded representation 750 of the audio signal.

The present invention further relates to decoders for decoding an audio signal from an encoded representation of the audio signal, which are configured and adapted to perform the decoding methods described in the present disclosure. An example of such a decoder 800 is schematically illustrated in Fig. 8 in the form of a block diagram. The decoder 800 comprises a processor 810 and a memory 820 coupled to the processor 810. The processor 810 is adapted to perform the steps of the method according to any of the decoding methods described in the present disclosure. For this purpose, the memory 820 may contain appropriate instructions for execution by the processor 810. The decoder 800 may further comprise an interface 830 for receiving an input encoded representation 840 of the audio signal to be decoded and/or outputting a decoded (i.e. reconstructed) audio signal 850.

The present invention further relates to computer programs containing instructions for causing a computer executing the instructions to execute the encoding or decoding methods described in the present invention.

Finally, the present invention also relates to a machine-readable storage medium on which the above-described computer programs are stored.

Interpretations

Unless otherwise expressly stated, as is evident from the following discussions, it should be understood that throughout this description, discussions that use terms such as “processing,” “calculating,” “computing,” “determining,” “analyzing,” or the like, refer to the action and/or processes of a computer or computing system or similar electronic computing devices that manipulate and/or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

Similarly, the term "processor" may refer to any device or part of a device that processes electronic data, such as from registers and/or a memory device, for the purpose of converting that electronic data into other electronic data, such as those stored in registers and/or a memory device. A "computer" or "computing machine" or "computing platform" may include one or more processors.

The methodologies described herein are, in one exemplary embodiment, adapted for execution by one or more processors receiving computer-readable (also referred to as machine-readable) code comprising a set of instructions that, when executed by the one or more processors, perform at least one of the methods described herein. Any processor adapted to (sequentially or otherwise) execute a set of instructions that define actions to be taken is included. Thus, one example is a typical processing system that includes one or more processors. Each processor may include one or more of a CPU (central processing unit), a graphics processor, and a programmable DSP unit. The processing system may further include a memory subsystem comprising a main RAM (random access memory) and/or a static RAM and/or a ROM (read only memory). A bus subsystem may be included to provide communication between the components. The processing system may further be a distributed processing system with the processors interconnected via a network. If the processing system requires a display, such a display may include, for example, a liquid crystal display (LCD) or a cathode ray tube (CRT) display. If manual data entry is required, the processing system also comprises an input device, such as one or more of an alphanumeric input unit, such as a keyboard, a pointing device, such as a mouse, etc. The processing system may also include a data storage system, such as a disk drive unit. The processing system in some configurations may include an audio output device and a network interface device. Thus, the storage subsystem comprises a machine-readable storage medium carrying machine-readable code (e.g., software) that comprises a set of instructions for causing one or more methods described in this document to be executed by one or more processors. It should be noted that if a method includes several elements, such as several steps, then, unless otherwise noted, no ordering of these elements is implied. The software may be located on a hard disk or may also be located, in whole or at least in part, in RAM and/or in a processor during its execution by a computer system. Thus, the storage device and the processor also constitute a machine-readable data carrier carrying a machine-readable code. In addition, the machine-readable data carrier may form a computer program product or may be contained therein.

In alternative exemplary embodiments, one or more processors operate as a stand-alone device or may be connected, such as via a network, to another processor(s) in a networked deployment, where one or more processors may operate as a server or a client machine in a client-server network environment or as a peer machine in a peer-to-peer or distributed network environment. The one or more processors may form a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a cellular telephone, a web interface device, a network router, a switch, or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the machine.

It should be noted that the term "machine" should be taken to include any set of machines that, individually or collectively, execute a set (or sets) of instructions for the purpose of performing any one or more of the methodologies described in this document.

Thus, one exemplary embodiment of each of the methods described herein has the form of a computer-readable storage medium carrying a set of instructions, such as a computer program, that is adapted to be executed on one or more processors, such as one or more processors that form part of a web server arrangement. Therefore, as will be understood by those skilled in the art, exemplary embodiments of the present invention can be implemented as a method, a device, such as a special-purpose device, a device, such as a data processing system, or a computer-readable storage medium, such as a computer program product. A computer-readable storage medium carries computer-readable code comprising a set of instructions that, when executed on one or more processors, cause the processor or processors to implement the method. Accordingly, aspects of the present invention can take the form of a method, an all-hardware exemplary embodiment, an all-software exemplary embodiment, or an exemplary embodiment combining aspects of software and hardware. Furthermore, the present invention can take the form of a storage medium (e.g., a computer program product on a computer-readable storage medium) carrying computer-readable program code implemented on the medium.

The software may further be transmitted or received over a network using a network interface device. Although the storage medium in the exemplary embodiment is a single storage medium, the term "storage medium" should be understood to include a single storage medium or a plurality of storage media (e.g., a centralized or distributed database and/or associated cache memory devices and servers) that store one or more sets of instructions. The term "storage medium" should also be understood to include any storage medium adapted to store, encode, or carry a set of instructions intended for execution by one or more processors and to cause the one or more processors to execute any one or more methodologies of the present invention. The storage medium may take many forms, including, but not limited to, non-volatile storage media, volatile storage media, and data transmission media. Non-volatile storage media include, for example, optical, magnetic disks, and magneto-optical disks. Volatile storage media include dynamic storage, such as main memory. Data transmission media include coaxial cables, copper wire and fiber optic cables, including wires that contain a bus subsystem. Data transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data transmission. For example, the term "data carrier" should accordingly be understood as including, but not limited to, solid-state memory devices, a computer product implemented on optical and magnetic media; a medium carrying a propagating signal detectable by at least one processor or one or more processors and representing a set of instructions that, when executed, implement a method; and a network data transmission medium carrying a propagating signal detectable by at least one processor of one or more processors and representing a set of instructions.

It should be understood that the discussed method steps are performed in one exemplary embodiment by a suitable processor (or processors) of a processing system (e.g., a computer system) executing instructions (machine-readable code) stored in a data store. It should also be understood that the present invention is not limited to any particular implementation or software solution and that the present invention can be implemented using any suitable solutions for implementing the functionality described herein. The present invention is not limited to any particular programming language or operating system.

Reference in this specification to "one exemplary embodiment," "some exemplary embodiments," or "an exemplary embodiment" means that a particular feature, structure, or characteristic described in connection with the exemplary embodiment is included in at least one exemplary embodiment of the present invention. Therefore, appearances of the phrases "in one exemplary embodiment," "in some exemplary embodiments," or "in an exemplary embodiment" in various places in this specification are not necessarily all referring to the same exemplary embodiment. Furthermore, particular features, structures, or characteristics may be combined in one or more exemplary embodiments in any suitable manner, as would be apparent from this specification to one skilled in the art.

In the context of this document, unless otherwise indicated, the use of the ordinal numerals "first," "second," "third," etc., to describe an ordinary object merely indicates that reference is being made to different instances of similar objects, and is not intended to indicate that the objects so described are intended to be in a given sequence, whether in time, space, rank, or otherwise.

In the claims below and in the present description, any of the terms "comprising", "consisting of" or "which comprises" is an open term, which means including at least the elements/features that follow it, but not excluding the rest. Therefore, the term "comprising" when used in the claims should not be interpreted as limiting with respect to the means or elements or steps listed after it. For example, the scope of the expression "a device comprising A and B" should not be limited to devices consisting only of elements A and B. Any of the terms "comprising" or "which comprises" used in this document is also an open term, which also means including at least the elements/features that follow this term, but not excluding the rest. Thus, "comprising" is synonymous and means "comprising".

It should be understood that in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together in a single exemplary embodiment, figure, or description thereof for the purpose of selecting the best way of describing and to provide an understanding of one or more various aspects of the invention. Such manner of disclosure, however, should not be interpreted as reflecting an intention that the claims require more features than those expressly recited in each claim. Instead, as the following claims reflect, aspects of the invention reside in less than all of the features of a single exemplary embodiment described above. Therefore, the claims following the Description section are hereby expressly incorporated into this Description section, with each claim independently representing a separate exemplary embodiment of the present invention.

Furthermore, although some exemplary embodiments described herein include some and not other features included in other exemplary embodiments, combinations of features from various exemplary embodiments are intended to be within the scope of the present invention and to form other exemplary embodiments, as will be understood by those skilled in the art. For example, in the following claims, any of the claimed exemplary embodiments may be used in any combination.

The description provided herein sets forth numerous specific details. However, it should be understood that exemplary embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order not to obscure the understanding of this description.

Thus, while what are considered to be the best embodiments of the present invention have been described herein, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the present invention, and all such changes and modifications are intended to be claimed as being within the scope of the present invention. For example, any formulas set forth above are merely examples of procedures that may be used. Functionality may be added to or deleted from block diagrams, and operations may be interchanged between functional blocks. Steps may be added to or deleted from methods described within the scope of the present invention.

Various aspects and implementations of the present invention can be understood from the following numbered exemplary embodiments (EEAs).

PPVO1. A method for encoding an audio signal, including:

generating a plurality of sub-range audio signals based on an audio signal;

determination of the envelope of the spectrum of the audio signal;

determining, for the audio signal of each sub-band, autocorrelation information for the audio signal of the sub-band based on the autocorrelation function of the audio signal of the sub-band; and

generating an encoded representation of the audio signal, wherein the encoded representation comprises a representation of the spectral envelope of the audio signal and a representation of autocorrelation information for a plurality of audio signals of sub-bands.

PPVO 2. The method according to PPVO 1, in which the spectral envelope is determined on the basis of a plurality of sub-range audio signals.

PPVO 3. The method according to PPVO 1 or 2, in which the autocorrelation information for the audio signal of a given sub-band contains a delay value for the audio signal of the corresponding sub-band and/or an autocorrelation value for the audio signal of the corresponding sub-band.

PPVO 4. The method according to the previous PPVO, in which the delay value corresponds to the delay value at which the autocorrelation function reaches a local maximum, and the autocorrelation value corresponds to the specified local maximum.

PPVO 5. A method according to any of the previous PPVOs, wherein the spectral envelope is determined at a first update rate, and the autocorrelation information for a plurality of sub-band audio signals is determined at a second update rate; and

where the first and second refresh rates differ from each other.

PPVO 6. The method according to the previous PPVO, in which the first update frequency is higher than the second update frequency.

PPVO 7. The method according to any of the previous PPVO, in which generating a plurality of sub-band audio signals includes:

applying spectral and/or temporal smoothing to an audio signal;

window processing of the smoothed audio signal; and

spectral decomposition of a windowed smoothed audio signal into a set of sub-range audio signals.

PPVO 8. Method according to any of PPVO 1–6,

wherein generating a plurality of sub-band audio signals comprises spectral decomposition of the audio signal; and

in this case, the definition of the autocorrelation function for the audio signal of a given sub-range includes:

determination of the spectrum envelope of a sub-range audio signal;

smoothing the envelope of a sub-range audio signal based on the envelope of the sub-range;

window processing of a sub-band audio signal with a smoothed envelope using a window function; and

determination of the autocorrelation function of a windowed audio signal sub-range with a smoothed envelope.

PPVO 9. The method according to PPVO 7 or 8, in which the determination of the autocorrelation function for the audio signal of a given sub-range additionally includes:

normalization of the autocorrelation function of a windowed audio signal of a sub-range with a smoothed envelope using the autocorrelation function of the window function.

PPVO 10. A method according to any of the previous PPVOs, in which determining autocorrelation information for an audio signal of a given sub-band based on the autocorrelation function of the audio signal of the sub-band includes:

comparing the autocorrelation function of the sub-band audio signal with the autocorrelation function of the absolute value of the impulse response of the corresponding band-pass filter associated with the sub-band audio signal; and

determining autocorrelation information based on the highest local maximum of the autocorrelation function of the sub-band signal above the autocorrelation function of the absolute value of the impulse response of the corresponding bandpass filter associated with the sub-band audio signal.

PPVO 11. A method according to any of the previous PPVOs, wherein determining the spectral envelope includes measuring the signal power for each of a plurality of sub-band audio signals.

PPVO 12. A method for decoding an audio signal from an encoded representation of the audio signal, wherein the encoded representation comprises a representation of the spectral envelope of the audio signal and a representation of autocorrelation information for each of a plurality of sub-band audio signals generated from the audio signal, wherein the autocorrelation information for the audio signal of a given sub-band is based on the autocorrelation function of the sub-band audio signal, wherein the method includes:

receiving a coded representation of an audio signal;

extracting the spectral envelope and autocorrelation information from the encoded representation of the audio signal; and

determination of the reconstructed audio signal based on the spectral envelope and autocorrelation information,

wherein the reconstructed audio signal is determined so that the autocorrelation function for each of the plurality of sub-range signals generated from the reconstructed audio signal satisfies a condition obtained from the autocorrelation information for the audio signals of the corresponding sub-ranges generated from the audio signal.

PPVO 13. The method according to the previous PPVO, in which the reconstructed audio signal is further determined such that for the audio signal of each sub-band of the reconstructed audio signal, the measured signal power of the audio signal of the sub-band of the reconstructed audio signal substantially coincides with the signal power for the audio signal of the corresponding sub-band of the audio signal, which is indicated by the spectral envelope.

PPVO 14. Method according to PPVO 12 or 13,

wherein the reconstructed audio signal is determined in an iterative procedure that starts with an initial candidate for the reconstructed audio signal and generates a corresponding intermediate reconstructed audio signal at each iteration; and

wherein at each iteration, an update mapping is applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration, such that the difference between the encoded representation of the intermediate reconstructed audio signal and the encoded representation of the audio signal is successively reduced from one iteration to the next.

PPVO 15. The method according to PPVO 14, in which the original candidate for the reconstructed audio signal is determined on the basis of the encoded representation of the audio signal.

PPVO 16. The method according to PPVO 14, in which the initial candidate for the reconstructed audio signal is white noise.

PPVO 17. The method according to PPVO 12 or 13, in which determining the reconstructed audio signal based on the spectral envelope and autocorrelation information includes applying a generative model based on machine learning, which receives as input data the spectral envelope of the audio signal and the autocorrelation information for each of a plurality of audio signals of sub-bands of the audio signal and generates and outputs the reconstructed audio signal.

PPVO 18. The method according to the previous PPVO, wherein the generative model based on machine learning comprises a parametric conditional distribution that associates the encoded representations of the audio signals and the corresponding audio signals with corresponding probabilities; and

wherein determining the reconstructed audio signal includes sampling from a parametric conditional distribution for an encoded representation of the audio signal.

PPVO 19. The method according to PPVO 17 or 18, at the training stage additionally including training a generative model based on machine learning on a data set of a plurality of sound signals and corresponding encoded representations of the sound signals.

PPVO 20. The method according to any one of PPVO 17-19, wherein the generative model based on machine learning is one of a recurrent neural network, a variational autoencoder, and a generative adversarial model.

PPVO 21. The method according to PPVO 12, in which the determination of the reconstructed audio signal based on the spectral envelope and autocorrelation information includes:

determining a plurality of reconstructed sub-band audio signals based on the spectral envelope and autocorrelation information; and

determining a reconstructed audio signal based on a plurality of reconstructed sub-band audio signals using spectral synthesis,

wherein the plurality of reconstructed sub-range audio signals are determined so that for each reconstructed sub-range audio signal, the autocorrelation function of the reconstructed sub-range audio signal satisfies the condition obtained from the autocorrelation information for the audio signal of the corresponding sub-range.

PPVO 22. The method according to the previous PPVO, in which the plurality of reconstructed sub-band audio signals are further determined such that for each reconstructed sub-band audio signal, the measured signal power of the reconstructed sub-band audio signal substantially coincides with the signal power for the audio signal of the corresponding sub-band, which is indicated by the spectral envelope.

PPVO 23. Method according to PPVO 21 or 22,

wherein each reconstructed sub-band audio signal is determined in an iterative procedure that starts with an initial candidate for the reconstructed sub-band audio signal and generates a corresponding intermediate reconstructed sub-band audio signal at each iteration; and

wherein at each iteration, an update mapping is applied to the intermediate reconstructed sub-band audio signal to obtain an intermediate reconstructed sub-band audio signal for the next iteration, such that the difference between the autocorrelation information for the intermediate reconstructed sub-band audio signal and the autocorrelation information for the audio signal of the corresponding sub-band is successively reduced from one iteration to the next.

PPVO 24. The method according to PPVO 21 or 22, in which determining a plurality of reconstructed sub-band audio signals based on the spectral envelope and autocorrelation information includes applying a generative model based on machine learning, which receives as input data the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of sub-band audio signals of the audio signal and generates and outputs a plurality of reconstructed sub-band audio signals.

PPVO 25. An encoder for encoding an audio signal, comprising a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any of PPVO 1–11.

PPVO 26. A decoder for decoding an audio signal from an encoded representation of the audio signal, comprising a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any of PPVO 12–24.

PPVO 27. A computer program containing instructions for causing the computer to execute, when executing these instructions, a method according to any of PPVO 1–24.

PPVO 28. Machine-readable data carrier on which a computer program is stored in accordance with the previous PPVO.

Claims (66)

1. A method for encoding an audio signal, including: generating a plurality of sub-range audio signals based on an audio signal; determination of the envelope of the spectrum of the audio signal; determining, for the audio signal of each sub-band, autocorrelation information for the audio signal of the sub-band based on an autocorrelation function of the audio signal of the sub-band, wherein the autocorrelation information comprises an autocorrelation value for the audio signal of the sub-band; and encoding into an encoded representation of an audio signal the envelope of the spectrum of the audio signal and autocorrelation information for a plurality of audio signals of sub-ranges, wherein the autocorrelation information for the audio signal of the given sub-range additionally contains the delay value for the audio signal of the corresponding sub-range. 2. The method according to claim 1, characterized in that it additionally includes outputting a bit stream defining the encoded representation. 3. The method according to paragraph 1 or 2, characterized in that the spectral envelope is determined on the basis of a plurality of sub-range audio signals. 4. The method according to any one of paragraphs 1–3, characterized in that the delay value corresponds to the delay value at which the autocorrelation function reaches a local maximum, and wherein the autocorrelation value corresponds to the specified local maximum. 5. The method according to any of the preceding paragraphs, characterized in that the spectral envelope is determined at a first update frequency, and autocorrelation information for a plurality of sub-band audio signals is determined at a second update rate; and where the first and second refresh rates differ from each other. 6. The method according to item 5, characterized in that the first refresh rate is higher than the second refresh rate. 7. A method according to any of the preceding paragraphs, characterized in that generating a plurality of sub-range audio signals includes: applying spectral and/or temporal smoothing to an audio signal; window processing of the smoothed audio signal; and spectral decomposition of a windowed smoothed audio signal into a set of sub-range audio signals. 8. The method according to any of paragraphs 1–6, characterized in that generating a plurality of sub-range audio signals includes spectral decomposition of the audio signal; and in this case, the definition of the autocorrelation function for the audio signal of a given sub-range includes: determination of the spectrum envelope of a sub-range audio signal; smoothing the envelope of a sub-range audio signal based on the envelope of the sub-range; window processing of a sub-band audio signal with a smoothed envelope using a window function; and determination of the autocorrelation function of a windowed audio signal sub-range with a smoothed envelope. 9. The method according to paragraph 7 or 8, characterized in that determining the autocorrelation function for the audio signal of a given sub-range additionally includes: normalization of the autocorrelation function of a windowed audio signal of a sub-range with a smoothed envelope using the autocorrelation function of the window function. 10. A method according to any of the preceding paragraphs, characterized in that determining the autocorrelation information for the audio signal of a given sub-range based on the autocorrelation function of the audio signal of the sub-range includes: comparing the autocorrelation function of the sub-band audio signal with the autocorrelation function of the absolute value of the impulse response of the corresponding band-pass filter associated with the sub-band audio signal; and determining autocorrelation information based on the highest local maximum of the autocorrelation function of the sub-band signal above the autocorrelation function of the absolute value of the impulse response of the corresponding bandpass filter associated with the sub-band audio signal. 11. A method according to any of the preceding paragraphs, characterized in that determining the spectral envelope includes measuring the signal power for each of a plurality of sub-range audio signals. 12. A method for decoding an audio signal from an encoded representation of the audio signal, wherein the encoded representation comprises a spectral envelope of the audio signal and autocorrelation information for each of a plurality of sub-band audio signals generated from the audio signal, wherein the autocorrelation information for the audio signal of a given sub-band is based on autocorrelation function of the audio signal of the sub-range, wherein the method includes: receiving a coded representation of an audio signal; extracting the spectral envelope and autocorrelation information from the encoded representation of the audio signal; and determination of the reconstructed audio signal based on the spectral envelope and autocorrelation information, wherein the autocorrelation information for the audio signal of the given sub-range contains the autocorrelation value for the audio signal of the sub-range and the delay value for the audio signal of the corresponding sub-range. 13. The method according to claim 12, characterized in that the reconstructed audio signal is determined such that the autocorrelation function for each of the plurality of sub-range signals generated from the reconstructed audio signal satisfies a condition obtained from the autocorrelation information for the audio signals of the corresponding sub-ranges generated from the audio signal. 14. The method according to claim 12 or 13, characterized in that the reconstructed audio signal is determined such that the autocorrelation information for each of the plurality of signals of the sub-ranges of the reconstructed audio signal coincides, up to a predetermined limit, with the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal. 15. The method according to paragraph 12 or 13, characterized in that the reconstructed audio signal is determined so that for the audio signal of each sub-range of the reconstructed audio signal, the value of the autocorrelation function of the audio signal of the sub-range of the reconstructed audio signal at the delay value specified by the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal coincides, up to a predetermined limit, with the autocorrelation value specified by the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal. 16. The method according to any one of paragraphs 12–15, characterized in that the reconstructed audio signal is additionally determined such that for the audio signal of each sub-range of the reconstructed audio signal, the measured signal power of the audio signal of the sub-range of the reconstructed audio signal coincides, up to a predetermined limit, with the signal power for the audio signal of the corresponding sub-range of the audio signal, which is indicated by the spectral envelope. 17. The method according to any of paragraphs 12–16, characterized in that the reconstructed audio signal is determined in an iterative procedure that starts with an initial candidate for the reconstructed audio signal and generates a corresponding intermediate reconstructed audio signal at each iteration; and wherein at each iteration, an update mapping is applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration in such a way that the difference between the encoded representation of the intermediate reconstructed audio signal and the encoded representation of the audio signal is successively reduced from one iteration to the next. 18. The method according to claim 17, characterized in that the original candidate for the reconstructed audio signal is determined on the basis of the encoded representation of the audio signal. 19. The method according to claim 17, characterized in that the initial candidate for the reconstructed audio signal is white noise. 20. The method according to any one of paragraphs 12-16, characterized in that determining the reconstructed audio signal based on the spectral envelope and autocorrelation information includes using a generative model based on machine learning, which receives as input data the spectral envelope of the audio signal and autocorrelation information for each of a plurality of audio signals of sub-ranges of the audio signal and generates and outputs the reconstructed audio signal. 21. The method according to claim 20, characterized in that the generative model based on machine learning comprises a parametric conditional distribution that relates the encoded representations of the sound signals and the corresponding sound signals with corresponding probabilities; and wherein determining the reconstructed audio signal includes sampling from a parametric conditional distribution for an encoded representation of the audio signal. 22. The method according to claim 20 or 21, characterized in that at the training stage it additionally includes training a generative model based on machine learning on a data set of a plurality of sound signals and corresponding encoded representations of the sound signals. 23. The method according to any one of paragraphs 20–22, characterized in that the generative model based on machine learning is one of a recurrent neural network, a variational autoencoder, and a generative adversarial model. 24. The method according to paragraph 13, characterized in that determining the reconstructed audio signal based on the spectral envelope and autocorrelation information includes: determining a plurality of reconstructed sub-band audio signals based on the spectral envelope and autocorrelation information; and determining a reconstructed audio signal based on a plurality of reconstructed sub-band audio signals using spectral synthesis, wherein the plurality of reconstructed sub-range audio signals is determined so that for each reconstructed sub-range audio signal, the autocorrelation function of the reconstructed sub-range audio signal satisfies the condition obtained from the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal. 25. The method according to claim 24, characterized in that the plurality of reconstructed sub-range audio signals are determined so that the autocorrelation information for each reconstructed sub-range audio signal coincides, up to a predetermined limit, with the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal. 26. The method according to claim 24, characterized in that the plurality of reconstructed sub-range audio signals are determined so that for each reconstructed sub-range audio signal, the value of the autocorrelation function of the reconstructed sub-range audio signal at the delay value specified by the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal coincides, up to a predetermined limit, with the autocorrelation value specified by the autocorrelation information for the audio signal of the corresponding sub-range of the audio signal. 27. The method according to any one of paragraphs 24–26, characterized in that the plurality of reconstructed sub-range audio signals additionally determine such that for each reconstructed sub-band audio signal, the measured signal power of the reconstructed sub-band audio signal matches, up to a predetermined limit, the signal power for the audio signal of the corresponding sub-band that is indicated by the spectral envelope. 28. The method according to any of paragraphs 24–27, characterized in that each reconstructed sub-band audio signal is determined in an iterative procedure that starts with an initial candidate for the reconstructed sub-band audio signal and generates a corresponding intermediate reconstructed sub-band audio signal at each iteration; and wherein at each iteration, an update mapping is applied to the intermediate reconstructed sub-band audio signal to obtain an intermediate reconstructed sub-band audio signal for the next iteration in such a way that the difference between the autocorrelation information for the intermediate reconstructed sub-band audio signal and the autocorrelation information for the audio signal of the corresponding sub-band is successively reduced from one iteration to the next. 29. The method according to any one of paragraphs 24-27, characterized in that determining a plurality of reconstructed sub-band audio signals based on the spectral envelope and autocorrelation information includes using a generative model based on machine learning, which receives as input data the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of sub-band audio signals of the audio signal and generates and outputs a plurality of reconstructed sub-band audio signals. 30. An encoder for encoding an audio signal, comprising a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any one of claims 1–11. 31. A decoder for decoding an audio signal from an encoded representation of the audio signal, comprising a processor and a memory device connected to the processor, wherein the processor is adapted to perform the steps of the method according to any one of claims 12 to 29. 32. A machine-readable data carrier on which a computer program is stored containing instructions for causing the computer to execute, when executing the instructions, the method according to any of paragraphs 1–29.