RU2803449C2 - Audio decoder, device for determining set of values setting filter characteristics, methods for providing decoded audio representation, methods for determining set of values setting filter characteristics, and computer software - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: processing audio data. The technical result is achieved by an audio decoder to provide a decoded audio presentation based on the encoded audio presentation, which contains a filter to provide an improved audio presentation of the decoded audio presentation. The filter is configured to obtain a plurality of scaling values that are associated with different frequency bins or frequency bands based on the spectral values of the decoded audio representation that are associated with different frequency bins or frequency bands, and the filter is configured to scale the spectral values of the decoded representation audio signals or a pre-processed version of them using scaling values to obtain an enhanced audio presentation. A device is also described for determining a set of values specifying filter characteristics for providing enhanced audio presentation based on the decoded audio presentation.

EFFECT: improved quality of audio data reproduction while maintaining the bit rate.

44 cl, 14 dwg

Description

1. Field of technology to which the invention relates

Embodiments of the present invention relate to an audio decoder.

Additional embodiments of the present invention relate to an apparatus for determining a set of values defining filter characteristics.

Additional embodiments of the invention relate to a method for providing a decoded audio presentation.

Additional embodiments of the invention relate to a method for determining a set of values defining the characteristics of a filter.

Additional embodiments of the invention relate to corresponding computer programs.

Embodiments of the invention relate to a real-value mask based post-filter for improving the quality of encoded speech.

Embodiments of the present invention generally relate to a post-filter for enhancing a decoded audio audio decoder by determining a set of values defining the characteristics of the filter based on the decoded audio representation.

2. State of the art

Below, an introduction to some traditional solutions is provided.

Given this situation, there is a need for a concept that provides a greater compromise between bit rate, audio quality and complexity when decoding audio content.

3. Essence of the invention

An embodiment of the present invention provides an audio decoder (eg, a speech decoder or a general audio decoder, or an audio decoder switching between a speech decoding mode, such as a linear prediction decoding mode, and a general audio decoding mode, such as a spectral domain representation-based coding mode with using scaling factors to scale the decoded spectral values) to provide a decoded audio representation based on the encoded audio representation.

The audio decoder contains a filter (or "post-filter") to provide an enhanced audio presentation (e.g. ) decoded audio representation (e.g. ), wherein the input audio representation that is used by the filter may, for example, be provided by a decoder core to the audio decoder.

The filter (or post-filter) is configured to obtain a plurality of scaling values (e.g., mask values, e.g., M(k, n)), which, for example, may be real-valued and which, for example, may be non-negative, and which, for example, may be limited to a predetermined range, and which are associated with different frequency bins or frequency ranges (eg, having a frequency bin index or frequency band index k), based on the spectral values of the decoded audio representation that are associated with the different frequency bins or frequency ranges (for example, having an index of frequency resolution element or index k of the frequency range).

The filter (or post-filter) is configured to scale the spectral values of the decoded representation of the audio signals (e.g. ) or a pre-processed version thereof using scaling values (e.g. M(k, n)) to produce an improved audio representation (e.g. ).

This embodiment is based on the idea that audio quality can be effectively improved by using scaling of spectral values of the decoded audio representation, wherein the scaling values are extracted based on the spectral values of the decoded audio representation. It has been discovered that filtering, which is performed by scaling spectral values, can effectively adapt to signal characteristics based on the spectral values of the decoded audio representation and can improve the quality of the decoded audio representation. For example, based on the spectral values of the decoded audio representation, the filter setting (which may be specified by scaling values) may be adjusted to reduce the effect of quantization noise. For example, adjusting the scaling values based on the spectral values of the decoded audio representation may use a machine learning-based framework or neural network that can provide the scaling values in a computationally efficient manner.

In particular, it has been found that extracting scaling values from the spectral values of the decoded audio representation is still advantageous and possible with good results, even if the quantization noise is generally correlated with the signal. Accordingly, the concept can be applied with very good results in this situation.

As a conclusion, the above-described audio encoder provides the ability to improve the achievable audio quality using a filter whose response is adjusted based on the spectral values of the decoded audio representation, wherein the filtering operation, for example, can be performed in an efficient manner by scaling the spectral values using the scaling values. In this way, the hearing sensation can be improved without necessarily relying on additional auxiliary information to control the filter adjustment. Conversely, filter adjustment may be based only on the decoded spectral values of the current processed frame, regardless of the encoding scheme used to generate encoded and decoded representations of the audio signal and possibly decoded spectral values of one or more previously decoded frames and/or one or more subsequent decoded frames.

In a preferred embodiment of an audio decoder, the filter is configured to use a configurable processing structure (eg, a "machine learning" based structure, like a neural network) configured based on a machine learning algorithm to provide scaling values.

By using a configurable processing framework, such as a machine learning-based framework or a neural network, filter characteristics can be easily adjusted based on coefficients specifying the functionality of the configurable processing framework. Accordingly, it is typically possible to adjust the filter characteristics over a wide range depending on the spectral values of the decoded audio representation. Therefore, it is possible to obtain improved sound quality in a variety of different circumstances.

In a preferred embodiment of an audio decoder, the filter is configured to determine scaling values based only on spectral values of the decoded audio representation in a plurality of frequency bins or frequency bands (eg, without using additional overhead in extracting scaling values from the spectral values).

Using this concept, it is possible to improve sound quality regardless of the presence of auxiliary information.

Computational and structural complexity can be kept quite low because a coherent and universal representation of the decoded audio signal (the spectral values of the decoded audio representation) is used, which is agnostic with respect to the encoding technologies used to obtain the encoded and decoded representation. In this case, complex and specific operations for specific auxiliary information values are not allowed. In addition, it is generally possible to extract scaling values based on the spectral values of the decoded audio representation using a generic processing framework (such as a neural network) that uses a limited number of different computational functionalities (such as scaled summations and estimation of activation functions).

In a preferred embodiment of an audio decoder, the filter is configured to receive values absolute value (which, for example, may describe the absolute value or amplitude, or rate) of the enhanced audio presentation according to the following:

,

wherein M(k, n) is a scaling value, wherein k is a frequency index (eg, denoting different frequency bins or frequency bands), wherein n is a temporal index (eg, denoting different overlapping or non-overlapping frames), and wherein is the absolute magnitude value of the spectral value of the decoded audio representation. Meaning The absolute value can be an absolute value, an absolute value, or any norm of a spectral value obtained by applying a time-frequency transform such as STFT, FFT, or MDCT to the decoded audio signal.

Alternatively, the filter may be configured to receive values improved audio presentation according to the following:

,

wherein M(k, n) is a scaling value, wherein k is a frequency index (eg, denoting different frequency bins or frequency bands), wherein n is a temporal index (eg, denoting different overlapping or non-overlapping frames), and wherein is the spectral value of the decoded audio representation.

It has been found that such simple extraction of the absolute value of the enhanced audio representation or (typically complex valued) values of the enhanced audio representation can be performed with good efficiency and still results in a significant improvement in audio quality.

In a preferred embodiment of an audio decoder, the filter is configured to obtain scaling values such that the scaling values cause scaling (or, in some cases, gain) to one or more spectral values of the decoded representation of the audio signals, or to one or more preprocessed spectral values that are based on the spectral values of the decoded representation of the audio signals.

By performing such scaling, which may preferably, but not necessarily, cause attenuation or attenuation of at least one spectral value (and which typically may also cause attenuation of at least one spectral value), the spectrum of the decoded audio representation can be generated effective way. For example, by allowing both enhancement and attenuation through scaling, artifacts that may be caused by limited precision in number representation may also be reduced in some cases. In addition, adjusting the scaling values does not necessarily include an additional degree of freedom by preventing the scaling values from being limited to values less than one. Accordingly, good improvement in sound quality can be achieved.

In a preferred embodiment of an audio decoder, the filter comprises a neural network or machine learning-based structure configured to provide scaling values based on a plurality of spectral values describing the decoded audio representation (e.g., describing the absolute values of the transformed representation of the decoded audio representation), wherein the spectral values are associated with different frequency resolution elements or frequency ranges.

The use of a neural network or machine learning based structure in such a filter is found to result in comparatively high performance. It has also been found that a neural network or machine learning framework can easily process spectral values of a decoded audio representation of an input quantity, in cases in which the number of spectral values input to the neural network or machine learning framework is comparatively high. It has been found that neural networks or machine learning based structures can handle such a high number of input signals or input quantities well, and they can also offer a large number of different scaling values as output quantities. In other words, neural networks or machine learning-based frameworks are found to be optimally suited to extract a relatively large number of scaling values from a relatively large number of spectral values, without the need for excessive computational resources. In this way, the scaling values can be adjusted to the spectral values of the decoded audio representation in a very precise manner without excessive computational load, and the spectral details of the decoded audio representation can be considered when adjusting the filtering characteristic. It has also been discovered that the coefficients of the neural network or machine learning framework providing the scaling values can be determined with reasonable effort, and that the neural network or machine learning framework provides sufficient degrees of freedom to achieve accurate determination of the scaling values.

In a preferred embodiment of an audio decoder, the input signals of the neural network or machine learning framework represent the logarithmic magnitude, amplitude, or norm of spectral values of the decoded audio representation, wherein the spectral values are associated with different frequency bins or frequency ranges.

It has been found that it is preferable to provide logarithmic absolute magnitudes of spectral values, amplitudes of spectral values, or norms of spectral values as inputs to a neural network or machine learning framework. It has been found that the sign or phase of the spectral values is of secondary importance for filter control, i.e. to define scaling values. In particular, it has been found that logarithmization of the absolute values of the spectral values of the decoded audio representation is particularly advantageous since the dynamic range may be reduced. It has been found that a neural network or machine learning framework can typically handle logarithmized absolute values of spectral values better than the spectral values themselves, since spectral values typically have an extended dynamic range. By using logarithmized values, it is also possible to use a simplified representation of numbers in an (artificial) neural network or machine learning framework, since it is often not necessary to use a floating point representation. Conversely, it is possible to design a neural network or machine learning framework using a fixed-point representation of numbers, significantly reducing implementation effort.

In a preferred embodiment of the audio decoder, the outputs of the neural network or machine learning framework represent scaling values (eg, mask values).

By providing scaling values as outputs (or output values) of a neural network or machine learning framework, implementation effort can be kept fairly low. For example, a neural network or machine learning framework that offers a relatively large number of scaling values is easy to implement. For example, a homogeneous structure can be used, which reduces implementation effort.

In a preferred embodiment of an audio decoder, a neural network or machine learning framework is trained to constrain, reduce, or minimize the deviation (e.g., mean square error; e.g., MSE _MA ) between a plurality of scaling targets (e.g., IRM(k, n)) and a set of scaling values (for example, M(k, n)) obtained using a neural network or using a machine learning-based framework.

By training a neural network or machine learning-based structure in this manner, it can be achieved that the improved audio representation, which is obtained by scaling the spectral values of the decoded representation of the audio signals (or a pre-processed version thereof) using the scaling values, provides a good sensation to the ear. For example, target scaling values can be easily determined, for example, based on knowledge of encoder-side lossy processing. In this way, it can be determined with little effort which scaling values best approximate the spectral values of the decoded audio representation into an ideal enhanced audio representation (which, for example, may be equal to the input audio representation of the audio encoder). In other words, by training a neural network or machine learning-based framework to constrain, reduce, or minimize the deviation between a plurality of target scaling values and a plurality of scaling values obtained using the neural network or using a machine learning-based framework, for example, for a variety of different audio content or types of audio content, it can be achieved that the neural network or machine learning-based structure provides appropriate scaling values even for different audio content or different types of audio content. In addition, by using the extraction between target scaling values and scaling values obtained using a neural network or using a machine learning-based structure as the optimization value, the complexity of the training process can be kept small, and numerical problems can be avoided.

In a preferred embodiment of an audio decoder, a neural network or machine learning framework is trained to constrain, reduce, or minimize the deviation (e.g., MSE _SA ) between a target absolute magnitude spectrum, a target amplitude spectrum, a target absolute spectrum, or a target normal spectrum (e.g., , e.g., the original spectrum of the training audio signal) and the (improved) absolute magnitude spectrum, amplitude spectrum, absolute spectrum, or normal spectrum obtained using scaling (e.g., frequency dependent scaling) processed (e.g., decoded, e.g., quantized, encoded, and decoded ) spectrum (which, for example, is based on the target absolute value spectrum and/or on the training audio signal), which uses scaling values that are provided by a neural network, or through a machine learning-based framework (wherein the input signals of the neural network, for example, are based on the decoded spectrum).

By using such a learning-based approach, typically good quality enhanced audio presentation can be achieved. In particular, it has been found that neural networks or machine learning-based frameworks also provide appropriate scaling factors if the decoded audio representation represents different audio content compared to the audio content used for training. In addition, it has been found that an enhanced audio presentation is perceived as being of good quality if the absolute magnitude spectrum or amplitude spectrum, or the absolute spectrum, or normal spectrum is in reasonably good agreement with the desired (target) magnitude spectrum or (target) amplitude spectrum, or ( target) absolute spectrum, or (target) normal spectrum.

In a preferred embodiment, the audio, neural network or machine learning framework is trained such that it scales to one or more spectral decomposition spectral values of the decoded representation of the audio signals or to one or more preprocessed spectral values that are based on the spectral decomposition spectral values of the decoded representation audio signals, is in the range between 0 and a predefined maximum value.

It has been found that limiting the scaling (or scaling values) is useful to prevent excessive amplification of the spectral values. It has been discovered that very high gain (or scaling) of one or more spectral values can lead to audible artifacts. In addition, it has been found that excessively large scaling values can be achieved during training, for example, if the spectral values of the decoded audio representation are very small or even zero. Thus, the quality of the enhanced audio presentation can be improved by using such a constraint-based approach.

In a preferred embodiment of the audio decoder, the maximum value is greater than 1 (and, for example, could be 2, 5 or 10).

This restriction on scaling (or scaling values) has been found to produce very good results. For example, by providing amplification capability (eg, by providing a scaling capability or a scaling value greater than one), artifacts that are caused by spectral holes can also be partially compensated. At the same time, excess noise can be limited through attenuation (eg, using scaling or scaling values less than one). Therefore, very flexible signal enhancement can be obtained by scaling.

In a preferred embodiment of an audio decoder, a neural network or machine learning framework is trained such that the scaling (or scaling values) for one or more spectral values of the spectral decomposition of the decoded representation of the audio signals or for one or more preprocessed spectral values that are based on the spectral spectral decomposition values of the decoded representation of audio signals are limited to 2, or limited to 5, or limited to 10, or limited to a predetermined value greater than 1.

By using this approach, artifacts can be kept fairly small while enhancement is resolved (which, for example, can help prevent "spectral holes"). In this way, a good hearing sensation can be obtained.

In a preferred embodiment of the audio decoder, the neural network or machine learning framework is trained such that the scaling values are limited to 2, or limited to 5, or limited to 10, or limited to a predetermined value greater than 1.

By limiting the scaling values to such a range, very good quality enhanced audio presentation can be achieved.

In a preferred embodiment of the audio decoder, the number of input features of the neural network or machine learning framework (eg, 516 or 903) is greater by at least a factor of 2 than the number of output values (eg, 129) of the neural network or machine learning framework.

It has been found that using a relatively large number of input features to a neural network or machine learning framework that is greater than the number of output values (or outputs) of the neural network or machine learning framework results in specific robust scaling values. In particular, by selecting a relatively high number of neural network input features, information from previous frames and/or subsequent frames can be considered, and it has been found that consideration of such additional input features typically improves the quality of the scaling values and thereby the quality of the enhanced audio presentation.

In a preferred embodiment of an audio decoder, the filter is configured to normalize input features (eg, represented by input signals) of a neural network or machine learning framework (eg, absolute magnitudes of spectral values obtained using a short-time Fourier transform) to a predetermined average value ( e.g., to a mean of zero) and/or to a predetermined variance (e.g., to unit variance) or standard deviation.

Normalization of the input features of a neural network or machine learning framework is found to make the provision of scaling values independent of loudness or loudness level, or the intensity of the decoded audio representation. Accordingly, a neural network or machine learning framework can "focus" on the structural characteristics of the spectrum of the decoded audio representation and is not (or not significantly affected by) changes in volume. In addition, by performing such normalization, it is possible to prevent the nodes of the neural network from becoming overly saturated. In addition, the dynamic range is reduced, which is useful to keep the representation of numbers used in a neural network or machine learning framework efficient.

In a preferred embodiment of an audio decoder, the neural network includes an input layer, one or more hidden layers, and an output layer.

This neural network structure turned out to be preferable for the present application.

In a preferred embodiment of an audio decoder, one or more hidden layers use linear rectification units as activation functions.

It has been found that the use of linear rectification blocks as activation functions provides the ability to provide scaling vectors based on the spectral values of the decoded audio representation with good reliability.

In a preferred embodiment of an audio decoder, the output layer uses (unconstrained) linear rectifiers or constrained linear rectifiers, or sigmoid functions (eg, scaled sigmoid functions) as activation functions.

By using linear rectification blocks or restricted linear rectification blocks, or sigmoid functions as activation functions in the output layer, scaling values can be obtained in a reliable manner. In particular, the use of limited linear rectification blocks or sigmoid functions provides the ability to limit the scaling values to the required range, as explained above. In this way, scaling values can be obtained in an efficient and reliable manner.

In a preferred embodiment of an audio decoder, the filter is configured to obtain short-time Fourier transform coefficients (e.g. ), which represent spectral values of the decoded audio representation that are associated with different frequency bins or frequency bands.

It is found that the short-time Fourier transform coefficients constitute, in particular, a meaningful representation of the decoded audio representation. For example, it should be recognized that short-time Fourier transform coefficients are better applied by a neural network or machine learning-based framework than MDCT coefficients in some cases (even though the MDCT coefficients can be used by an audio decoder to reconstruct the decoded spectral representation).

In a preferred embodiment of an audio decoder, the filter is configured to extract log absolute value, amplitude, absolute or normal values (eg, based on short time Fourier transform coefficients) and determine scaling values based on log absolute value, amplitude, absolute or normal values.

Extracting scaling values based on non-negative values, such as log absolute values, amplitude values, absolute values, or normal values, has been found to be effective since considering phase must significantly increase the computational load without resulting in significant improvement in scaling values. Thus, removing the sign and typically also the phase of spectral values (eg obtained through the short-time Fourier transform) promotes a good compromise between complexity and sound quality.

In a preferred embodiment of an audio decoder, the filter is configured to determine a plurality of scaling values associated with a current frame (eg, a current frame of a decoded audio representation or a current frame of a short time Fourier transform) based on spectral values of the decoded audio representation that are associated with various frequency bins or frequency ranges of the current frame, and based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges of one or more frames preceding the current frame (eg, previous context frames).

However, it has been found that considering the spectral values of one or more frames preceding the current frame helps improve the scaling vectors. This is due to the fact that many types of audio content contain temporal correlation between subsequent frames. Thus, a neural network or machine learning framework, for example, can consider the temporal evolution of spectral values when determining scaling values. For example, a neural network or machine learning framework may adjust the scaling values to prevent (or counteract) the scaled spectral values (eg, in an enhanced audio representation) from changing excessively over time.

In a preferred embodiment of an audio decoder, the filter is configured to determine a plurality of scaling values associated with a current frame (eg, a current frame of a decoded audio representation or a current frame of a short time Fourier transform) based on spectral values of the decoded audio representation that are associated with various frequency bins or frequency ranges of one or more frames after the current frame (eg, future context frames).

By considering the spectral values of the decoded audio representation of one or more frames after the current frames, correlations between subsequent frames can also be exploited, and the quality of the scaling values can typically be improved.

An embodiment of the present invention provides an apparatus for determining a set of values (e.g., neural network coefficients or coefficients of another machine learning-based structure) defining characteristics of a filter (e.g., a neural network-based filter or a filter based on another machine learning-based structure) for providing an enhanced audio presentation (e.g. ) based on the decoded audio representation (which, for example, may be provided by audio decoding).

The device is configured to obtain spectral values (for example, absolute magnitudes or phases, or MDCT coefficients, for example, represented by absolute magnitude values, for example, ) decoded audio representations that are associated with different frequency bins or frequency ranges.

The apparatus is configured to determine a set of values defining filter characteristics such that scaling values provided by the filter based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges approximate the target scaling values (which may be calculated based on the comparison the required enhanced audio presentation and the decoded audio presentation).

Alternatively, the apparatus is configured to determine a set of values defining filter characteristics such that the spectrum obtained by the filter is based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges, and using scaling values obtained based on the decoded audio representation. audio representation, approximates the target spectrum (which may correspond to the desired enhanced audio representation and which may be equal to the input signal of an audio encoder in a processing chain comprising an audio encoder and an audio decoder including a filter).

Using such a device, a set of values specifying the characteristics of the filter that is used in the above audio decoder can be obtained with little effort. In particular, a set of values, which may be coefficients of a neural network or coefficients of another machine learning-based structure, specifying the characteristics of the filter, may be determined such that the filter uses scaling values that result in good audio quality and result in an improved enhanced audio presentation over compared to the decoded audio representation. For example, determining a set of values defining filter characteristics may be performed based on a plurality of training audio content or reference audio content, where target scaling values or target spectrum may be derived from the reference audio content. However, it has been found that the set of values defining the filter characteristics is typically also suitable for audio content that is different from the reference audio content, provided that the reference audio content is at least to some extent representative of the audio content that is to be decoded by the audio decoder. mentioned above. In addition, it has been found that using scaling values provided by the filter or using the spectrum obtained by the filter as the optimization value results in a reliable set of values defining the characteristics of the filter.

In a preferred embodiment of the device, the device is configured to train a machine learning-based structure (e.g., a neural network) that is part of the filter and that provides scaling values for scaling the absolute magnitude values of the decoded audio signal or the spectral values of the decoded audio signal to reduce or minimize deviation (e.g., root mean square error; e.g., MSE _MA ) between a set of target scaling values (e.g., IRM(k, n)) and a set of scaling values (e.g., M(k, n)) obtained using a neural network based on spectral decoded audio representation values that are associated with different frequency bins or frequency ranges.

By training a machine learning-based structure using scaling targets that, for example, can be derived from the original audio content that is encoded and decoded in a processing chain containing an audio decoder (which extracts the decoded audio representation), the machine learning-based structure can be designed (or configured) with the ability to at least partially compensate for signal degradation in the processing chain. For example, scaling targets may be determined such that the scaling targets scale the decoded audio representation such that the decoded audio representation approximates the (original) audio representation input to the processing chain (eg, input to an audio encoder). Thus, the scaling values provided by the machine learning-based framework can have a high degree of reliability and can be configured to improve the recovery of audio content that is subjected to the processing chain.

In a preferred embodiment, the device is configured to train a machine learning-based structure (e.g., a neural network) so as to reduce or minimize the deviation (e.g., MSE _SA ) between a target (absolute value) spectrum (e.g., , e.g., the original spectrum of the training audio signal) and the (enhanced) spectrum (or absolute magnitude spectrum) obtained using scaling (e.g., frequency-dependent scaling) of the processed (e.g., decoded, e.g., quantized, encoded, and decoded) spectrum (which, for example, based on a target absolute magnitude spectrum and/or on a training audio signal) that uses scaling values that are provided by a machine learning-based framework (eg, a neural network). For example, machine learning or neural network based structure inputs are based on the decoded spectrum.

It is found that such machine learning-based structure training also results in scaling values that provide the ability to compensate for signal degradation in the signal processing chain (which may include audio encoding and audio decoding). For example, the target spectrum may be the spectrum of reference audio content or training audio content that is input into a processing chain comprising an audio encoder (lossy) and an audio decoder providing a decoded audio representation. Thus, the machine learning framework can be trained such that the scaling values scale the decoded audio representation to approximate the reference audio content input to the audio encoder. Therefore, the machine learning-based framework can learn to provide scaling values that help overcome performance degradation in the (lossy) processing chain.

In a preferred embodiment, the device is configured to train a machine learning-based structure (eg, a neural network) such that a scaling (or scaling value) for the spectral values of the decoded representation of the audio signals or for one or more pre-processed spectral values that is based on spectral values of the decoded representation of audio signals, is in the range between 0 and 2, or is in the range between 0 and 5, or is in the range between 0 and 10, or is in the range between 0 and the maximum value (which, for example, may be greater than 1) .

By limiting the scaling to a predefined range (eg, between zero and a predefined value, which typically may be greater than one), artifacts that may be caused by, for example, excessively large scaling values can be avoided. Additionally, it should be noted that limiting the scaling values (which may be provided as the outputs of the neural network or machine learning framework) allows the output stages (eg, output nodes) of the neural network or machine learning framework to be implemented relatively easily.

In a preferred embodiment of the device, the device is configured to train a machine learning-based structure (e.g., a neural network) such that an absolute magnitude scaling (or scaling value) for spectral values of the decoded representation of the audio signals or for one or more pre-processed spectral values, which are based on the spectral values of the decoded representation of the audio signals are limited such that they are in the range between 0 and a predetermined maximum value.

By limiting the absolute value scaling (or scaling values) so that they are in the range between zero and a predetermined maximum, switching degradation caused by excessively strong absolute value scaling is prevented.

In a preferred embodiment of the audio decoder, the maximum value is greater than 1 (and, for example, could be 2, 5 or 10).

By allowing the maximum absolute value scaling value to be greater than one, attenuation and gain can be achieved by scaling using the scaling values. This concept has been shown to be very flexible and to provide a very good listening experience.

An embodiment of the invention provides a method for providing a decoded audio representation based on an encoded audio representation.

The method comprises providing an enhanced audio presentation (for example, ) decoded audio representation (e.g. ), wherein an input audio representation that is used by a filter providing an enhanced audio representation, for example, may be provided by a decoder core for an audio decoder.

The method comprises obtaining a plurality of scaling values (e.g., mask values, e.g., M(k, n)), which, for example, may be real-valued and which, for example, may be non-negative, and which, for example, may be limited to a predetermined range, and which are associated with different frequency bins or frequency bands (eg, having a frequency bin index or frequency band index k), based on the spectral values of the decoded audio representation that are associated with the different frequency bins or frequency bands (eg, having frequency resolution element index or frequency range index k).

The method comprises scaling spectral values of a decoded representation of audio signals (for example, ) or a pre-processed version thereof using scaling values (e.g. M(k, n)) to produce an enhanced audio representation (e.g. ).

This method is based on approaches identical to those for the above-described device. In addition, it should be noted that the method may be extended by any of the features, functionality and details described herein also with respect to the apparatus. It should further be noted that the method may be enhanced by any of these features, functionality and details, either individually or in combination.

An embodiment provides a method for determining a set of values (e.g., neural network coefficients or coefficients of another machine learning-based structure) defining characteristics of a filter (e.g., a neural network-based filter or a filter based on another machine learning-based structure) to provide an enhanced audio presentation (For example, ) based on the decoded audio representation (which, for example, may be provided by audio decoding).

The method comprises obtaining spectral values (for example, absolute magnitudes or phases, or MDCT coefficients represented by absolute magnitude values, for example, ) decoded audio representations that are associated with different frequency bins or frequency ranges.

The method comprises defining a set of values defining filter characteristics such that the scaling values provided by the filter based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges approximate the target scaling values (which can be calculated based on a comparison of the desired improved audio presentation and decoded audio presentation).

Alternatively, the method comprises defining a set of values defining characteristics of the filter, such that a spectrum obtained by the filter based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency bands, and using scaling values obtained based on the decoded audio representation, approximates the target spectrum (which may correspond to the desired enhanced audio representation and which may be equal to the input signal of the audio encoder in a processing chain comprising an audio encoder and an audio decoder including a filter).

This method is based on approaches identical to those for the device described above. However, it should be noted that the method can be extended by any of the features, functionality and details described herein also with respect to the apparatus. In addition, the method can be complemented by features, functionality and details, either individually or in combination.

An embodiment of the invention provides a computer program for implementing the method described herein when the computer program runs on a computer.

4. Brief description of drawings

Embodiments according to the present invention will now be described with reference to the accompanying drawings, in which:

Fig. 1 shows a circuit block diagram of an audio decoder according to an embodiment of the present invention;

Fig. 2 shows a schematic block diagram of an apparatus for determining a set of values defining filter characteristics, according to an embodiment of the present invention;

Fig. 3 shows a circuit block diagram of an audio decoder according to an embodiment of the present invention;

Fig. 4 shows a schematic block diagram of an apparatus for determining a set of values defining filter characteristics, according to an embodiment of the present invention;

Fig. 5 shows a schematic block diagram of an apparatus for determining a set of filter characteristic values according to an embodiment of the present invention;

Table 1 shows a representation of the percentage of mask values that are in the interval (0, 1) for different signal-to-noise ratio (SNR);

Table 2 shows a representation of the percentage of mask values in various threshold regions measured at the lowest three AMR-WB bit rates;

Fig. 6 shows a schematic representation of a fully connected neural network (FCNN) that converts log absolute value into real-valued masks;

Fig. 7 shows a graphical representation of the average PESQ and POLQA scores evaluating the oracle experiment with different mask limits at 6.65 Kbps;

Fig. 8 shows a graphical representation of the average quantitative PESQ and POLQA indicators evaluating the performance of the proposed methods and the EVS post-processor;

Fig. 9 shows a flowchart of a method according to an embodiment of the present invention; And

Fig. 10 shows a flowchart of a method according to an embodiment of the present invention.

5. Detailed Description of Embodiments

1. Audio decoder according to FIG. 1

Fig. 1 shows a circuit block diagram of an audio decoder 100 according to an embodiment of the present invention. Audio decoder 100 is configured to receive the encoded audio representation 110 and provide therefrom an enhanced audio representation 112, which may be an improved form of the decoded audio representation.

Audio decoder 100 optionally includes a decoder core 120 that can receive an encoded audio presentation 110 and provide, based thereon, a decoded audio presentation 122. The audio decoder further includes a filter 130 that is configured to provide an enhanced audio presentation 112 based on the decoded audio presentation 122. A filter 130 that can regarded as a post-filter, is configured to obtain a plurality of scaling values 136 that are associated with different frequency bins or frequency bands, based on the spectral values 132 of the decoded audio representation that are also associated with different frequency bins or frequency bands. For example, filter 130 may include a scaling value determination or scaling value determination module 134 that receives spectral values 132 of the decoded audio representation and that provides scaling values 136 . The filter 130 is further configured to scale the spectral values of the decoded representation of the audio signals or a pre-processed version thereof using the scaling values 136 to obtain an improved audio representation 112.

It should be noted that the spectral values of the decoded audio representation that are used to obtain the scaling values may be identical to the spectral values that are actually scaled (eg, by scaling or scaling module 138), or may be different from the spectral values that are actually scaled. For example, a first subset of spectral values of the decoded audio representation may be used to determine scaling values, and a second subset of spectral values of a spectrum or amplitude spectrum or absolute spectrum or normal spectrum may actually be scaled. The first subset and the second subset may be equal, or may overlap partially, or may even be completely different (with no common spectral values at all).

With respect to the functionality of the audio decoder 100, it can be said that the audio decoder 100 provides a decoded audio representation 122 based on the encoded audio representation. Because encoding (i.e., providing a coded audio representation) is typically lossy, the decoded audio representation 122 provided by, for example, the decoder core may contain some degradation in performance compared to the original audio content (which may be provided to the audio encoder providing the coded audio representation 110). It should be noted that the decoded audio representation 122 provided by, for example, the decoder core may take any form, and, for example, may be provided by the decoder core in the form of a time domain representation or in the form of a spectral domain representation. The spectral domain representation, for example, may contain (discrete) Fourier transform coefficients or (discrete) MDCT coefficients and the like.

Filter 130, for example, may receive (or receive) spectral values representing the decoded audio representation. However, the spectral values used by the filter 130, for example, may be of a different type compared to the spectral values provided by the decoder core. For example, filter 130 may use Fourier coefficients as spectral values, while decoder core 120 initially only provides MDCT coefficients. In addition, filter 130 may optionally extract spectral values from the time domain representation of decoded audio representation 120, for example, through a Fourier transform or an MDCT transform, or the like. (for example, short-time Fourier transform (STFT)).

The scaling value determination 134 extracts the scaling values 136 from a plurality of spectral values of the decoded audio representation (eg, extracted from the decoded audio representation). For example, the scaling value determination 134 may comprise a neural network or machine learning framework that takes the spectral values 132 and extracts the scaling values 136 . In addition, the spectral values of the enhanced audio representation 112 may be obtained by scaling the spectral values of the decoded audio representation (which may be equal to or different from the spectral values used by determining the scaling values 134) according to the scaling values 136. For example, scaling values 136 may specify scaling of spectral values in different frequency bins or frequency bands. Additionally, it should be noted that scaling 136 may operate on complex-valued spectral values or real-valued spectral values (eg, amplitude values or absolute magnitude values, or normal values).

Accordingly, by using an appropriate determination of the scaling values 136 based on the spectral values 132 of the decoded audio representation, the scaling 138 can counteract the degradation in audio quality caused by the lossy coding used to provide the encoded audio representation 110.

For example, scaling 138 may reduce quantization noise, such as by selectively attenuating spectral bins or spectral bands containing high quantization noise. Alternatively or in addition, scaling 138 may also result in a smoothing of the spectrum in time and/or frequency, which may also help reduce quantization noise and/or improve perceptual feel.

However, it should be noted that the audio decoder 100 of FIG. 1 may optionally be supplemented by any of the features, functionality and details disclosed herein, either alone or in combination.

2. The device according to Fig. 2

Fig. 2 shows a schematic block diagram of an apparatus 200 for determining a set of values (e.g., neural network coefficients or coefficients of another machine learning-based structure) defining characteristics of a filter (e.g., a neural network-based filter or a filter based on another machine learning-based structure) .

The device 200 according to FIG. 2 is configured to receive the decoded audio representation 210 and provide, from it, a set of filter values 212, wherein the set of filter values 212, for example, may comprise coefficients of a neural network or coefficients of another machine learning-based structure. Optionally, device 200 may receive scaling target values 214 and/or target spectrum information 216 . However, device 200 may optionally directly generate target scaling values and/or target spectrum information 216.

It should be noted that target scaling values, for example, may describe scaling values that significantly bring (or push) the decoded audio presentation 210 closer to an ideal (undistorted) state. For example, target scaling values may be determined based on knowledge of the reference audio representation from which the decoded audio representation 210 is extracted through encoding and decoding. For example, from knowledge of the spectral values of the reference audio representation and from knowledge of the spectral values of the decoded audio representation, what scaling causes the enhanced audio representation (which is obtained based on the spectral values of the decoded audio representation using scaling) to approximate the reference audio representation can be extracted.

In addition, target spectrum information 216, for example, may be based on knowledge of a reference audio representation from which a decoded audio representation is extracted through encoding and decoding. For example, the target spectrum information may take the form of spectral values of the reference audio representation.

As can be seen in FIG. 2, device 200 may optionally include a spectral value definition, in which spectral values of the decoded audio representation 210 are extracted from the decoded audio representation 210. The spectral value definition is denoted by 220, and the spectral values of the decoded audio representation are denoted by 222. However, it should be noted that determining spectral values 220 should be considered optional since the decoded audio representation 210 can be directly provided in the form of spectral values.

The device 200 also includes a definition 230 of a set of values defining a filter. Definition 230 may receive or receive spectral values 222 of the decoded audio representation and, based on them, provide a set 212 of values defining a filter. Definition 230 may optionally use target scaling values 214 and/or target spectrum information 216 .

With respect to the functionality of the device 200, it should be noted that the device 200 is configured to obtain spectral values 222 of the decoded audio representation that are associated with various frequency bins or frequency bands. In addition, definition 230 may be configured to determine a set 212 of values defining filter characteristics such that scaling values provided by the filter based on spectral values 222 of the decoded audio representation that are associated with different frequency bins or frequency ranges approximate the target values scaling (eg, scaling targets 214). As mentioned above, target scaling values may be calculated based on a comparison of the desired enhanced audio representation and the decoded audio representation, wherein the desired enhanced audio representation may correspond to the reference audio representation mentioned above. In other words, definition 230 may determine and/or optimize a set of values (e.g., a set of neural network coefficients or a set of coefficients of another machine learning-based structure) specifying the characteristics of a filter (e.g., a neural network-based filter or a filter based on another machine learning-based structure). machine learning) such that the filter provides scaling values based on the spectral values of the decoded audio representation that approximate the target scaling values 214. Determining the set of values 214 defining the filter can be done using a one-pass forward calculation, but typically can be done using iterative optimization. However, any known training procedures for neural networks or for computer-learning-based structures can be used.

Alternatively, defining 230 a set of values 212 defining the filter may be configured to determine a set 212 of values defining filter characteristics such that the spectrum obtained by the filter based on the spectral values of the decoded audio representation (which are associated with different frequency bins or frequency bands ), and using scaling values derived from the decoded audio representation, approximates the target spectrum (which, for example, may be described by target spectrum information 216). In other words, definition 230 may select a set 212 of values defining a filter such that the filtered version of the spectral values of the decoded audio representation 210 approximates the spectral values described by target spectrum information 216. As an output, device 200 may determine a set 212 of values defining the filter such such that the filter at least partially approximates the spectral values of the decoded audio representation as "ideal" or "reference" or "target" spectral values. To this end, the device typically uses decoded audio representations representing various audio content. By defining a set of filter defining values 212 based on different audio content (or different types of audio content), the set of filter defining values 212 can be selected such that the filter performs well enough for audio content that is different from the reference audio content used to train the set 212 values defining the filter.

In this way, it can be achieved that the set of filter defining values 212 is optimally suited for improving the decoded audio representation obtained in the audio decoder, for example, in the audio decoder 100 according to FIG. 1. In other words, the set of filter defining values 212 may be used, for example, in the audio decoder 100 to specify the operation of determining the scaling values 134 (and thereby specifying the operation of the filter 130).

However, it should be noted that the device 200 of FIG. 2 may optionally be supplemented by any of the features, functionality and details described herein, either alone or in combination.

3. Audio decoder 300 according to FIG. 3

Fig. 3 shows a schematic block diagram of an audio decoder 300, according to another embodiment of the present invention. Audio decoder 300 is configured to receive an encoded audio representation 310, which may correspond to an encoded audio representation 110, and therefrom provide an enhanced audio representation 312, which may correspond to an enhanced audio representation 112. Audio decoder 300 includes a decoder core 320, which may correspond to a decoder core 120. Decoder core 320 provides a decoded audio representation 322 (which may correspond to a decoded audio representation 122) based on the encoded audio representation 310. The decoded audio representation may be in a time domain representation, but may also be in a spectral domain representation.

Optionally, audio decoder 300 may include a transform 324 that can take the decoded audio representation 322 and provide a spectral domain representation 326 based on the decoded audio representation 322. This transform 324, for example, may be useful if the decoded audio representation does not take the form of spectral values associated with different frequency resolution elements or frequency ranges. For example, transform 324 may transform decoded audio representation 322 into a plurality of spectral values if decoded audio representation 322 is in a time domain representation. However, transform 324 may also transform from the first type of spectral domain representation to the second type of spectral domain representation in the event that decoder core 320 does not provide spectral values usable through the post-processing steps. Spectral domain representation 326, for example, may include spectral values 132, as shown in audio decoder 100 of FIG. 1.

In addition, audio decoder 300 includes a scaling value definition 334, which, for example, includes an absolute value definition 360, a logarithmic calculation 370, and a neural network or machine learning framework 380. Defining scaling values 334 provides scaling values 336 based on spectral values 326 that may correspond to spectral values 132.

Audio decoder 300 also includes a scaling 338 that may correspond to scaling 138. When scaling, the spectral values of the decoded audio representation or a preprocessed version thereof are scaled depending on the scaling values 336 provided by the neural network/machine learning framework 380. Accordingly, scaling 338 provides an improved audio presentation.

Determining 334 scaling values and scaling 338 can be considered a filter or "post-filter".

Some additional details are described below.

The definition of 334 scaling values contains the definition of 360 absolute values. The definition 360 of absolute values may take a spectral domain representation 326 of the decoded audio representation, e.g. . The absolute value determination 360 can then provide the absolute values 362 of the spectral domain representation 326 of the decoded audio representation. Absolute values 362, for example, can be indicated by

Determining the scaling values also includes a logarithmic calculation 370 that takes the absolute values 362 of the spectral domain representation of the decoded audio representation (e.g., a plurality of absolute values of the spectral values) and provides, based thereon, logarithmized absolute values 372 of the spectral domain representation of the decoded audio representation. For example, logarithmized absolute values of 372 can be denoted using log ₁₀

It should be noted that the absolute value determination 360, for example, can determine the absolute values or absolute magnitude values, or normal values for a plurality of spectral values of the spectral domain representation 326, such that, for example, the signs or phases of the spectral values are removed. Logarithmic calculations, for example, calculate the ordinary logarithm (base 10) or natural logarithm, or any other logarithm that may be appropriate. In addition, it should be noted that the logarithmic calculation may not necessarily be replaced by any other calculation that reduces the dynamic range of the spectral values 362. In addition, it is known that the logarithmic calculation 370 may include limiting negative and/or positive values such that the logarithmic absolute values 372 may be limited to a reasonable range of values.

The scaling value determination 334 also includes a neural network or machine learning framework 380 that takes the logarithmized absolute values 372 and that provides scaling values 332 based on them. A neural network or machine learning-based framework 380, for example, may be parameterized by a set 382 of values defining the characteristics of the filter. The value set, for example, may contain machine learning-based structure coefficients or neural network coefficients. For example, the set 382 of values may contain the weights of the neural network branches and optionally also activation function parameters. A set 382 of values, for example, may be determined by device 200, and a set 382 of values, for example, may correspond to a set 212 of values.

In addition, the neural network or machine learning framework 380 may also optionally contain logarithmized absolute spectral domain representation values of the decoded audio representation for one or more frames preceding the current frame and/or for one or more frames after the current frame. In other words, the neural network or machine learning framework 380 may not only use the logarithmized absolute values of the spectral values associated with the current processed frame (for which the scaling values are applied), but may also consider the logarithmized absolute values of the spectral values of one or more previous frames and /or one or more subsequent frames. Thus, the scaling values associated with a given (currently processed) frame may be based on the spectral values of the given (currently processed) frame, as well as the spectral values of one or more previous frames and/or one or more subsequent frames.

For example, the logarithmized absolute values of the spectral domain representation of the decoded audio representation (denoted by 372) may be applied to the inputs (eg, input neurons) of the neural network or machine learning framework 380. The scaling values 336 may be provided through the outputs of a neural network or machine learning framework 380 (eg, through output neurons). Additionally, a neural network or machine learning framework may perform processing according to a set 382 of values defining the characteristics of the filter.

Scaling 338 may take scaling values 336, which may also be referred to as “masking values” and which, for example, may be denoted by M(k, n), as well as spectral values or pre-processed spectral representation values in the spectral domain of the decoded audio representation. For example, the spectral values that are input to the scaling 338 and that are scaled according to the scaling values 336 may be based on the spectral domain representation 326 or may be based on absolute values 362, where, optionally, preprocessing may be applied before how 338 scaling is performed. The pre-processing, for example, may comprise filtering, for example in the form of a fixed scaling or a scaling determined by the auxiliary information of the encoded audio information. However, the preprocessing may also be fixed, independent of the auxiliary information of the encoded audio representation. In addition, it should be noted that the spectral values that are input into the scaling 338 and that are scaled using the scaling values 336 need not be identical to the spectral values that are used to extract the scaling values 336 .

Accordingly, scaling 338, for example, may multiply spectral values that are input to scaling 338 by scaling values, with different scaling values associated with different frequency bins or frequency ranges. Accordingly, an enhanced audio representation 312 is obtained, wherein the enhanced audio representation, for example, may comprise a scaled spectral domain representation (e.g. ) or scaled absolute values of such a spectral domain representation (e.g. ). Thus, scaling 338, for example, can be performed using a simple multiplication between the spectral values associated with the decoded audio representation 322 and the associated scaling values provided by a neural network or machine learning framework 380.

As an output, device 300 provides an enhanced audio representation 312 based on the encoded audio representation 310, with scaling 338 applied to spectral values that are based on the decoded audio representation 322 provided by the decoder core 320. The scaling values 336 that are used in scaling 338 are provided by a neural network or machine learning framework, wherein the input signals of the neural network or machine learning framework 380 are preferably obtained by logarithmizing the absolute values of the spectral values that are based on the decoded audio representation 322 However, by appropriately selecting a set 382 of values defining the filter characteristics, a neural network or machine learning framework can provide scaling values such that scaling 338 improves the auditory experience of an improved audio representation relative to the decoded audio representation.

In addition, it should be noted that audio decoder 300 may not necessarily be complemented by any of the features, functionality and details described herein.

4. The device according to Fig. 4

Fig. 4 shows a schematic block diagram of an apparatus 400 for determining a set of values (eg, neural network coefficients or coefficients of other machine learning-based structure) defining filter characteristics. The device 400 is configured to receive the training audio presentation 410 and, based on it, provide a set 412 of values defining the characteristics of the filter. It should be noted that the training audio presentation 410, for example, may contain various audio content that is used to determine a set 412 of values.

The device 400 includes an audio encoder 420 that is configured to encode the training audio representation 410 to thereby obtain an encoded training audio representation 422. The device 400 also includes a decoder core 430 that receives the encoded training audio representation 422 and provides, based thereon, a decoded audio representation 432. It should be noted that the decoder core 420, for example, may be identical to the decoder core 320 and the decoder core 120. The decoded audio presentation 432 may also correspond to the decoded audio presentation 210.

The device 400 also optionally includes a transform 442 that converts the decoded audio representation 432, which is based on the training audio representation 410, into a spectral domain representation 446. Transform 442, for example, may correspond to transform 324, and spectral domain representation 446, for example, may correspond to spectral domain representation 326. Apparatus 400 also includes an absolute value definition 460 that takes the spectral domain representation 446 and provides, based thereon, the absolute values 462 of the spectral domain representation. The definition of 460 absolute values, for example, may correspond to the definition of 360 absolute values. The device 400 also includes a logarithmic calculation 470 that takes the absolute values 462 of the spectral domain representation and provides, based thereon, the logarithmized absolute values 472 of the spectral domain representation of the decoded audio representation. A logarithmic calculation of 470 may correspond to a logarithmic calculation of 370.

In addition, device 400 also includes a neural network or machine learning framework 480 that corresponds to the neural network or machine learning framework 380. However, the structure coefficients based on machine learning or neural network 480, which are denoted by 482, are provided through neural network/machine learning training 490. It should be noted here that the machine learning neural network/framework 480 provides the scaling values that the machine learning neural network/framework extracts based on the logarithmized absolute values 372 to the neural network/machine learning training 490 .

Apparatus 400 also includes scaling target calculation 492, which is also referred to as “ratio mask calculation.” For example, the scaling target calculation 492 takes the training audio representation 410 and the absolute values 462 of the spectral domain representation of the decoded audio representation 432. Accordingly, the scaling target calculation 492 provides scaling target information 494 that describes the desired scaling values to be provided by the neural network. 480 structures based on machine learning. Accordingly, the neural network/machine learning training 490 compares the scaling values 484 provided by the neural network/machine learning framework 480 with the scaling targets 494 provided by the scaling 492 target value calculations and adjusts the values 482 (i.e. structure coefficients based on machine learning or neural network) to reduce (or minimize) the deviation between the scaling values 484 and the target scaling values 494.

Below, a general overview of the functionality of the device 400 is provided. By encoding and decoding the training audio presentation (which, for example, may contain various audio content) in the audio encoder 420 and in the audio decoder 430, a decoded audio presentation 432 is obtained that typically contains some degradation in performance compared to the training audio presentation. due to losses in lossy coding. The calculation 492 of target scaling values determines what scaling (eg, which scaling values) should be applied to the spectral values of the decoded audio representation 432 such that the scaled spectral values of the decoded audio representation 432 well approximate the spectral values of the training audio representation. It is believed that artifacts introduced by lossy encoding can be at least partially compensated for by applying scaling to spectral values of the decoded audio representation 432. Therefore, the neural network or machine learning framework 480 is trained by neural network training/machine learning training such such that the scaling values 482 provided by the neural network/machine learning framework 480 based on the decoded audio representation 432 approximate the target scaling values 494. The optional transform 442, absolute value determination 460, and logarithmic calculation 470 constitute only (optional) preprocessing steps to extract input values 472 (which are the logarithmized absolute values of the spectral values of the decoded audio representation) for the neural network or machine learning framework 480.

The neural network training 490/machine learning training may use an appropriate learning mechanism (e.g., an optimization procedure) to adjust the machine learning or neural network structure coefficients 482 such that the difference (e.g., a weighted difference) between the scaling values 484 and the target values 494 scaling is minimized or falls below the threshold or is at least reduced.

Accordingly, the coefficients 482 of the machine learning or neural network structure (or, more generally, a set of values defining the characteristics of the filter) are provided by the device 400. These values can be used in the filter 130 (to regulate the determination of scaling values 134) or in the device 300 ( to regulate the neural network/structure 380 based on machine learning).

However, it should be noted that the device 400 may not necessarily be complemented by any of the features, functionality and details described herein.

5. The device according to Fig. 5

Fig. 5 shows a schematic block diagram of an apparatus 500 for determining a set 512 of values defining a filter, wherein the values 512, for example, may be machine learning or neural network based structure coefficients.

It should be noted that the device 500 is similar to the device 400, so identical features, functionality and details are not given again. Instead, you should refer to the above explanations.

Device 500 receives training audio representation 510, which, for example, may correspond to training audio presentation 410. Device 500 includes an audio encoder 520, which corresponds to audio encoder 420, and which provides an encoded training audio representation 522, which corresponds to encoded training audio presentation 422. Device 500 also includes a decoder core 530, which corresponds to decoder core 430 and provides a decoded audio representation 532.

Device 500 optionally includes a transform 542 that corresponds to transform 442 and that provides a spectral domain representation (eg, in the form of spectral values) of the decoded audio representation 552. The spectral domain representation is denoted by 546 and corresponds to the spectral domain representation 446. In addition, device 500 includes an absolute value definition 560 that corresponds to an absolute value definition 460. The device 500 also includes a logarithmic calculation 570 that corresponds to the logarithmic calculation 470. In addition, the device 500 includes a neural network or machine learning-based structure 580 that corresponds to the machine learning-based structure 480. However, device 500 also includes a scaling 590 that is configured to receive spectral values 546 of the decoded audio representation or absolute spectral values 562 of the decoded audio representation. The scaling also receives scaling values 584 provided by the neural network 580. Accordingly, scaling 590 scales the spectral values of the decoded audio representation or the absolute values of the spectral values of the audio representation to thereby obtain an improved audio representation 592. The improved audio representation 592, for example, may include scaled spectral values ( For example, or scaled absolute values of spectral values (e.g. ). In principle, the enhanced audio presentation 592 may correspond to the enhanced audio presentation 112 provided by the device 100 and the enhanced audio presentation 312 provided by the device 300. To such an extent, the functionality of the device 500 may correspond to the functionality of the device 100 and/or the functionality of the device 300, except that that the coefficients of the neural network or machine learning-based structure 580, which are denoted by 594, are adjusted through neural network/machine learning training 596. For example, neural network/machine learning training 596 may receive training audio representation 510 as well as enhanced audio representation 592 and may adjust coefficients 594 such that enhanced audio representation 592 approximates the training audio representation.

It should be noted here that if the enhanced audio representation 592 approximates the training audio representation 510 with good accuracy, the signal degradation caused by the lossy encoding is at least partially compensated by the scaling 590. In other words, the neural network training 596, for example, can determine (weighted) difference between the training audio presentation 510 and the enhanced audio presentation 592 and adjusting the structure coefficients 594 based on machine learning or neural network 580 to reduce or minimize this difference. Adjustment of coefficients 594, for example, may be performed in an iterative procedure.

Accordingly, it is possible to ensure that the coefficients 594 of the neural network or machine learning framework 580 are adapted such that, in normal operation, the machine learning framework or neural network 380, using certain coefficients 594, can provide scaling values 336 that result in to an improved audio presentation of 312 good quality.

In other words, the coefficients 482, 594 of the neural network or machine learning-based structure 480 or the neural network or machine learning-based structure 580 can be used in the neural network 380 of the device 300, and the device 300 can be expected to provide a high quality enhanced audio presentation 312 in this situation. . Of course, this functionality is based on the assumption that the machine learning neural network/framework 380 is similar or even identical to the machine learning neural network/framework 480 or the machine learning neural network/framework 580.

Additionally, it should be noted that factors 482, 412 or factors 594, 512 may also be used in determining 134 scaling values of audio decoder 100.

In addition, it should be noted that the device 500 may not necessarily be complemented by any of the features, functionality and details described herein, either alone or in combination.

6. Details and embodiments

Below, some of the approaches underlying the present invention are explained and several solutions are described. In particular, a certain number of details are disclosed that may not necessarily be included in any of the embodiments disclosed herein.

6.1. Formulation of the problem

6.1.1. Ideal Relationship Mask (IRM)

From a very simplified mathematical point of view, one can describe coded speech , for example, decoded speech provided by a decoder core (e.g., decoder core 120 or decoder core 320 or decoder core 430 or decoder core 530), as follows:

(1)

- where x(n) is the input to the encoder (eg, audio encoder 410, 510), and δ(n) is the quantization noise. The quantization noise δ(n) is correlated with the input speech because ACELP uses perceptual models during the quantization process. This correlation property of quantization noise makes the postfiltering problem unique to the speech enhancement problem, which allows for noise to be decorrelated. To reduce quantization noise, a real value mask per frequency and time bin is estimated, and this mask is multiplied by the absolute value mask of the encoded speech for that frequency and time bin.

(2)

- where M(k, n) is a real-valued mask, is the absolute value of the encoded speech, is the absolute value of the improved speech, k is the frequency index, and n is the time index. If the mask is ideal (eg, if the scaling values M(k, n) are ideal), clear speech can be recovered from the encoded speech.

(3)

- Where is the absolute value of pure speech.

By comparing Equation 2 and 3, an ideal ratio mask (IRM) is obtained (e.g., the ideal value of the scaling values M(k, n)), which is given as follows:

(4)

- Where is a very small constant factor to prevent division by zero. Since the absolute value values are in the range , IRM values are also in the range .

In other words, for example, improved audio presentation can be extracted based on decoded audio using scaling, where the scaling factors can be described by M(k, n). Additionally, for example, scaling factors M(k, n) may be extracted from the decoded audio representation because typically there is a correlation between noise (which is at least partially compensated for by scaling using the scaling factors M(k, n)) and the decoded audio representation For example, scaling as defined in equation (2) may be accomplished by scaling 138, wherein determining 134 scaling values may, for example, provide scaling values M(k, n) that approximate ideal scaling vectors IRM(k, n), as described, for example, by equation (4).

Thus, it is desirable for the definition of scaling values 134 to determine scaling values that approximate IRM(k, n).

This, for example, can be achieved by appropriately designing the scaling value definition 134 or scaling value definition 334, wherein, for example, the machine learning or neural network structure coefficients used to implement block 380 can be determined as follows: according to the text.

6.1.2. MMSE optimization

For example, two different types of minimum mean square error (MMSE) optimization can be used to train a neural network (eg, neural network 380): mask approximation (MA) (eg, as shown in FIG. 4) and signal approximation (MA). SA) [10] (for example, as shown in Fig. 5). The MA optimization approach attempts to minimize the mean square error (MSE) between a target mask (eg, target scaling values) and an estimated mask (eg, scaling values 484 provided by a neural network).

(5)

- where IRM(k, n) is the target mask, M(k, n) is the estimated mask.

The SA optimization approach attempts to minimize the mean square error (MSE) between the target absolute magnitude spectrum (eg, the absolute magnitude spectrum of the training audio presentation 510) and the improved absolute magnitude spectrum (eg, the absolute magnitude spectrum of the enhanced audio presentation 592).

(6)

- where the improved absolute magnitude spectrum is given by Equation 2.

In other words, the neural network used in determining the scaling values 134 or in determining the scaling values 334 may be trained, for example, as shown in FIG. 4 and 5. As can be seen from Figs. 4, neural network training 490/machine learning training optimizes the neural network coefficients or machine learning-based structure coefficients 482 according to the criterion specified in equation (5).

As shown in FIG. 5, neural network training 596/machine learning training optimizes the neural network coefficients/machine learning structure coefficients 594 according to the criterion shown in equation (6).

6.1.3. Analysis of mask values

In most of the proposed mask-based approaches for speech enhancement and dererberation, the mask values are limited to one [9]–[10]. This is because, traditionally, if mask values are not limited to one, estimation errors may cause noise or music tones to be amplified [15]. Hence, these approaches use the sigmoid as output activations to constrain the mask values to 1.

Table 1 shows the percentage of mask values that are in the interval (0.1) for various signal-to-noise ratios (SNR). These mask values are calculated by adding white noise at different SNRs to clean speech. From Table 1, it can be logically concluded that most of the mask values are in the interval [0,1], and because of this, limiting the mask values to 1 does not have a negative effect on neural network-based speech enhancement systems.

The distribution of mask values at the lower three bit rates (6.65 Kbps, 8.85 Kbps, and 12.65 Kbps) of AMR-WB is then calculated. Table 2 shows the calculated distribution. One significant difference for Table 1 is the percentage of mask values that are in the range [0,1]. Although 39% of the values are in this range at 6.65 Kbps, at 12.65 Kbps this value increases to 44%. Almost 30-36% of the mask values are in the range (1,2]. Almost 95% of the mask values are in the range [0,5]. Therefore, for the post-filtering problem, you cannot simply limit the mask value to 1. This prevents the use of sigmoid activations (or simple, unscaled sigmoid activations) in the output layer.

In other words, it has been found that it is preferable to use mask values (also referred to as scaling values) that are greater than one in embodiments of the invention. In addition, it has been found that it is preferable to limit the mask values or scaling values to a predetermined value, which should be greater than one and which, for example, may be in the region between 1 and 10 or in the region between 1.5 and 10. By limiting the mask value or scaling values, excessive scaling, which could lead to artifacts, may be avoided. For example, an appropriate range of scale division values can be achieved by using a scaled sigmoidal activation in the output layer of the neural network, or by using a (eg, rectified) constrained linear activation function as the output layer of the neural network.

6.2. Experimental layout

Some details regarding the experimental setup are described below. However, it should be noted that the functionality of the features and details described herein may not necessarily be adopted in any of the embodiments disclosed herein.

The proposed post-filter computes a short-time Fourier transform (STFT) of 16 ms frames with 50% overlap (8 ms) at a sampling rate of 16 kHz (eg, at block 324). Time frames are weighted encoded with a Han weighted encoding function, before a fast Fourier transform (FFT) of length 256 is computed, resulting in 129 frequency bins (eg, spatial domain representation 326). Logarithmic absolute magnitude values are calculated from the FFT to compress the very wide dynamic range of absolute magnitude values (eg, logarithmic absolute magnitudes 372). Since speech is time dependent, context frames around the processed time frame (eg, indicated by 373) are used. The proposed model is tested under two conditions: a) only previous context frames are used, and b) previous and future context frames are used. This is implemented because future context frames contribute to the delay of the proposed post-filter, and the benefit of using future context frames needs to be tested. A contextual encoding function with a weight of 3 is chosen for experiments with advancing latency of just one frame (16 ms) when only previous context frames are considered. When previous and future context frames are considered, the latency of the proposed post-filter is 4 frames (64 ms).

The dimension of input features (for example, values 373 and 373) for the proposed neural network when testing only with the previous 3 context frames and the current processed frame is 516 (4*129). When testing with previous and future context frames, the dimension of the input features is 903 (7*129). The input features (for example, values 372 and 373) are normalized to zero mean and unit variance. However, the target, either the real-valued mask (eg, values 494) or the absolute magnitude spectrum of uncoded speech (eg, absolute magnitude of values 410), is not normalized.

Fig. 6 shows an FCNN 600 that is trained to learn a transformation function f _θ between a log absolute value and a real-valued mask.

(7)

The FCNN is a simple neural network that has an input layer 610, one or more hidden layers 612a-612d, and an output layer 614. The FCNN is implemented in Python using the work of Keras [16], and uses Tensorflow [17] as the backend. In the experiments, 4 hidden layers with 2048 units were used. All 4 hidden layers use rectified linear units (ReLUs) as activation functions [18]. The output of the hidden layers is normalized using batch normalization [19]. To prevent overfitting, the signal dropout [20] is set to 0.2. To train the FCNN, the Adam optimizer [21] is used with a learning rate of 0.01, and the batch size used is 32.

The dimension of the output layer 614 is 129. Because FCNN evaluates to a real-valued (or real-valued) mask, and these masks can take any value between , testing is performed both with and without limiting the mask values. When the mask values are unconstrained, ReLU activation is used in the output layer. When the mask values are constrained, either a constrained ReLU activation or a sigmoid function is used, and the output of the sigmoid activation is scaled by a certain scaling factor N.

To train the FCNN, two loss functions (MSE _MA and MSE _SA ) are used, defined in Section 6.1.2. Cut-off norms are used to ensure the convergence of the model when constrained ReLU or unconstrained ReLU is used as the activation of the output layer.

The gradients in the output layer when constrained or unconstrained ReLU is used are as follows:

(8)

- where tar is either an absolute value spectrum (for example, the absolute value of audio representation 510) or an IRM (for example, values 494), out is either an improved absolute value (for example, values 542) or an estimated mask (for example, values 484), which takes any value between 0 and the threshold value, and h is the output of the hidden module, which is given as the input to the output module. When constrained ReLU is used, Equation 8 is zero outside the constrained value.

The gradients in the output layer when the scaled sigmoid is used are as follows:

(9)

- where tar is either the absolute magnitude spectrum or the IRM (e.g. 494 values), out is either the improved absolute magnitude or the estimated mask M _est , which takes any value between 0 and 1, and h is the output of the hidden module, which is given by input to the output module.

For training validation and testing, the NTT database was used [22]. Cross-database testing was also performed on the TIMIT database [23] to confirm the independence of the model from the training database. The NTT and TIMIT databases are pure speech databases. The TIMIT database consists of monospeech files at a sampling rate of 16 kHz. The NTT database consists of stereo speech files sampled at 48 kHz. To obtain mono speech files at 16 kHz, passive downmixing and resampling was performed on the NTT database. The NTT database consists of 3960 files, of which 3612 files are used for training, 198 files are used for validation, and 150 files are used for testing. The NT database consists of male and female speakers and also consists of languages such as American English and British English, German, Chinese, French and Japanese.

Enhanced time domain speech is obtained using the inverse short time Fourier transform (iSTFT). iSTFT uses the encoded speech phase without any processing at all.

As an output, the fully connected neural network 600 as shown in FIG. 6 is used in embodiments of the invention to implement scaling value determination 134 or neural network 380. In addition, neural network 600 may be trained by device 200 or by device 400 or by device 500.

As can be seen, neural network 600 receives logarithmized absolute magnitude values (e.g., logarithmized absolute values of spectral values 132, 372, 472, 572) in input layer 610. For example, logarithmized absolute values of spectral values of the current processed frame and one or more previous frames, and one or more subsequent frames may be received at input layer 610. The input layer, for example, may receive logarithmized absolute values of the spectral values. The values received by the input layer can then be forwarded, in a scaled manner, to the artificial neurons of the first hidden layers 612a. The scaling of the input values of the input layer 612, for example, can be specified by a set of values defining the characteristics of the filter. Next, artificial neurons of the first hidden layer 612, which may be implemented using nonlinear functions, provide the output values of the first hidden layer 612a. The output values of the first hidden layer 612a are then provided, in a scaled manner, to the inputs of the artificial neurons of the subsequent (second) hidden layer 612b. On the other hand, scaling is specified by a set of values that define the characteristics of the filter. Additional hidden layers containing similar functionality may be included. Finally, the outputs of the last hidden layer (eg, the fourth hidden layer 612d) are provided, in a scaled manner, to the inputs of the output layer 614 artificial neurons. The functionality of the output layer 614 artificial neurons, for example, can be specified by an output layer activation function. Accordingly, the output values of the neural network can be determined using an estimate of the activation function of the output layer.

Additionally, it should be noted that a neural network can be "fully connected", meaning, for example, that all inputs to the neural network can contribute to the inputs of all artificial neurons in the first hidden layer, and that the outputs of all artificial neurons in a given layer hidden layer can contribute to the input signals of all artificial neurons of the subsequent hidden layer. However, the actual shares may be determined by a set of values defining the characteristics of the filter, which is typically determined by training 490, 596 a neural network.

Additionally, it should be noted that neural network training 490, 596, for example, may use gradients as provided in equations (8) and (9) in determining the neural network coefficients.

It should be noted that any of the features, functionality and details described in this section may not necessarily be included in any of the embodiments disclosed herein, either alone or in combination.

6.3. Experiments and results

To evaluate the limit of the mask values, an experiment was carried out using the oracle principle. This evaluates the IRM and limits the IRM with different thresholds, as shown in FIG. 7. Used objective measures such as Perceptual Speech Quality Assessment (PESQ) [24][25][26] and Perceptual Objective Listening Quality Assessment (POLQA) [27] for evaluation. From fig. 7, it can be concluded that setting the threshold value to 1 does not work as well as setting the threshold value to 2, 4 or 10. There are very minor differences between threshold values 2, 4 and 10. Therefore, a mask value limit of 2 is chosen in additional experiments .

In addition, FIG. 8 shows the average quantitative PESQ and POLQA indicators evaluating the performance of the proposed methods and the EVS postprocessor. It can be seen that application of the concepts described herein results in improved speech quality, both for the case in which the signals are approximated (for example, as shown in FIG. 5) and for the case in which the masked approximation is used (for example, as shown in FIG. 5). shown in Fig. 4) is used to train an artificial neural network.

7. Conclusions

It is found that the quality of encoded speech suffers significantly at lower bit rates due to high quantization noise. Post filters are typically used at low bit rates to mitigate the effect of quantization noise. In this disclosure, a real-value mask based post filter is proposed to improve the quality of decoded speech at lower bit rates. To estimate this real-valued mask, for example, a fully connected neural network is used that operates on normalized logarithmic absolute values. This proposal is tested on a codec based on the Adaptive Multi-Rate Wideband (AMR-WB) codec in the lower 3 modes (6.65 Kbps, 8.85 Kbps and 12.65 Kbps). The experiment shows improvement in PESQ, POLQA and subjective listening tests.

In other words, embodiments according to the invention relate to a concept that utilizes a fully connected network in the context of speech encoding and/or speech decoding. Embodiments of the invention relate to improving coded speech. Embodiments of the invention relate to post-filtration. Embodiments of the invention relate to a concept that solves problems associated with quantization noise (or more precisely, quantization noise reduction).

In embodiments according to the invention, a CNN (convolutional neural network) is used as a transform function in the cepstral domain. Work [14] proposes a statistical contextual postfilter in the logarithmic absolute value domain.

In this work, the coded speech enhancement problem is formulated as a regression problem. A fully connected neural network (FCNN) is trained to learn a transformation function f _θ between the input (log absolute value) and the output (real valued mask). The estimated real-valued mask is then multiplied by the input absolute value to improve the encoded speech. Performance is evaluated for the AMR-WB codec at bit rates of 6.65 Kbps, 8.85 Kbps, and 12.65 Kbps. In embodiments, the post filter may be used in EVS [4][3] as a reference post filter. For further information, please refer to sections 6.1 and 6.2. As can be seen, verbal test results based on listening are provided. For example, preferred PESQ and POLQA scores can be achieved using embodiments of the invention.

Some additional important points are described below.

According to a first aspect, a mask-based post-filter to improve the quality of encoded speech is used in embodiments according to the invention.

The mask is real-valued (or the scaling values are real-valued). It is estimated for each frequency bin by a machine learning algorithm (or by a neural network) from the input features:

- Where is a rated mask, is the absolute value of the encoded speech, and is the post-processed speech at frequency bin k and time index n.

The input features currently used are the logarithmic magnitude spectrum, but can also be any derivative of the absolute magnitude spectrum.

According to a second aspect, there may optionally be a limitation on the mask values or scaling values.

The values of the estimated mask are, for example, in the range [0, ]. To prevent such a large range, a threshold value may optionally be set. In traditional speech enhancement algorithms, the mask is limited to 1. In contrast, here it is limited to a threshold value that is greater than 1. This threshold value is determined by analyzing the distribution of masks. Useful thresholds, for example, may be between 2 and 10.

a. Because the estimated mask values, for example, are limited by a threshold, and because the estimated threshold is greater than 1, the output layer can be either restricted linear units (ReLUs) or a scaled sigmoid.

b. When a machine learning algorithm is optimized using an MMSE (Minimum Mean Square Estimation Optimization)-based method for approximating masks, the target mask (eg, scaling targets) may optionally be modified either by setting the mask values (eg, scaling targets) above a threshold value in target mask equal to 1, or can be set equal to the threshold value.

According to the third aspect, the machine learning algorithm can be used as a fully connected neural network. Long short-term memory (LSTM) can also be used as an alternative.

a. A fully connected neural network consists of, for example, 4 hidden layers. Each hidden layer, for example, consists of 2048 or 2500 activations of linear rectification units (ReLUs).

b. The input dimension of a fully connected neural network depends on the context frames and the size of the FFT. System latency also depends on context frames and frame size.

c. The size of the context frames, for example, can be any value between 3 and 5. For experiments, for example, 256 (16 ms 16 kHz) was used as the frame size and FFT size. The size of the context frames is set to 3, since there is very little benefit when going beyond 3. The test is also performed with both future+previous context frames and with only previous context frames.

According to the fourth aspect, the fully connected network is trained with the following MMSE (minimum mean square estimation optimization): mask approximation and signal approximation.

a. When approximating masks, the mean squared error between the target mask (eg, the target scaling values) and the estimated mask (eg, the scaling values of the scaling values determined using a neural network) is minimized. The target mask is modified, for example, as specified in (2.b) (for example, in aspect 2, subsection b).

b. When approximating signals, the root mean square error between the improved absolute value (eg, improved absolute value spectrum 592) and the target absolute value (eg, absolute value spectrum of the audio presentation 510) is minimized. The improved absolute value is obtained by multiplying the estimated mask from the DNN (eg, a neural network) by the estimated encoded absolute value mask. The target absolute value is an uncoded speech absolute value.

As a conclusion, the embodiments described herein may not necessarily be supplemented by any of the important points or aspects described herein. However, it should be noted that the important points and aspects described herein may be used alone or in combination, and may be incorporated into any of the embodiments described herein, either alone or in combination.

8. Method according to FIG. 9

Fig. 9 shows a schematic flow diagram of a method 900 for providing an enhanced audio presentation based on an encoded audio presentation, according to an embodiment of the present invention.

The method comprises providing 910 a decoded audio representation ( ).

The method further comprises obtaining 920 a plurality of scaling values (M(k, n)) that are associated with different frequency bins or frequency bands, based on the spectral values of the decoded audio representation that are associated with the different frequency bins or frequency bands. and the method comprises scaling 930 spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof, using (M(k, n)) scaling values to obtain an enhanced audio representation ( ).

The method 900 may optionally be enhanced by any of the features, functionality and details described herein, either alone or in combination.

9. Method according to FIG. 10

Fig. 10 shows a schematic flow diagram of a method 1000 for determining a set of values defining filter characteristics to provide enhanced audio presentation ( ) based on the decoded audio representation, according to an embodiment of the present invention.

The method involves obtaining 1010 spectral values ( ) decoded audio representations that are associated with different frequency bins or frequency ranges.

The method also includes defining 1020 a set of values defining filter characteristics such that the scaling values provided by the filter based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency bands approximate the target scaling values.

Alternatively, the method comprises defining 1030 a set of values defining filter characteristics such that the spectrum obtained by the filter is based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges, and using scaling values obtained based on the decoded audio representation. audio representation, approximates the target spectrum.

10. Implementation Alternatives

Although certain aspects are described in the context of an apparatus, it will be appreciated that these aspects also represent a description of the corresponding method, wherein the block or apparatus corresponds to a method step or a feature of a method step. Likewise, aspects described in the context of a method step also provide a description of the corresponding block or element, or feature of the corresponding device. Some or all of the steps of the method may be performed by (or using) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most critical steps of the method may be performed by this device.

The inventive encoded audio signal may be stored on a digital storage medium or may be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. An implementation may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, having stored electronically readable control signals that interact (or are interoperable) with programmable computer system in such a way that the appropriate method is carried out. Therefore, the digital storage medium may be machine readable.

Some embodiments of the invention comprise a storage medium having electronically readable control signals that are capable of interfacing with a programmable computer system in a manner that implements one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program product with program code, wherein the program code is configured to implement one of the methods when the computer program product runs on a computer. The program code, for example, may be stored on a computer-readable medium.

Other embodiments comprise a computer program for implementing one of the methods described herein stored on a computer-readable medium.

In other words, an embodiment of the inventive method is therefore a computer program having program code for carrying out one of the methods described herein when the computer program runs on a computer.

Therefore, a further embodiment of the inventive methods is a storage medium (digital storage medium or computer readable medium) containing a recorded computer program for implementing one of the methods described herein. The storage medium, digital storage medium, or recorded data medium is typically tangible and/or non-volatile.

Therefore, a further embodiment of the inventive method is a data stream or signal sequence representing a computer program for implementing one of the methods described herein. The data stream or signal sequence, for example, may be configured to be transmitted over a data connection, such as the Internet.

An additional embodiment comprises processing means, such as a computer or programmable logic device, configured to implement one of the methods described herein.

A further embodiment comprises a computer having a computer program installed for performing one of the methods described herein.

A further embodiment of the invention comprises a device or system configured to transmit (eg, electronically or optically) a computer program for implementing one of the methods described herein to a receiving device. The receiving device, for example, may be a computer, mobile device, storage device, or the like. The device or system, for example, may include a file server for transmitting a computer program to a receiving device.

In some embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may interface with a microprocessor to implement one of the methods described herein. In general, the methods are preferably implemented by any hardware device.

The apparatus described herein may be implemented using a hardware device, either using a computer, or using a combination of a hardware device and a computer.

The device described herein, or any components of the device described herein, may be implemented at least in part in hardware and/or software.

The methods described herein may be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

The methods described herein, or any components of the apparatus described herein, may be performed at least in part by hardware and/or software.

The above-described embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and changes to the arrangements and details described herein will be apparent to those skilled in the art. Accordingly, they are intended to be limited only by the scope of the claims below and not by the specific details provided by the description and explanation of the embodiments herein.

11. Bibliography

1. 3GPP "Speech codec speech processing functions; Adaptive Multi-Rate - Wideband (AMR-WB) speech codec; Transcoding functions", 3rd Generation Partnership Project (3GPP), TS 26.190, 12 2009. [online]. At: http://www.3gpp.org/ftp/Specs/html-info/26190.htm

2. M. Dietz, M. Multrus, V. Eksler, V. Malenovsky, E. Norvell, H. Pobloth, L. Miao, Z. Wang, L. Laaksonen, A. Vasilache, Y. Kamamoto, K. Kikuiri, S. Ragot, J. Faure, H. Ehara, V. Rajendran, V. Atti, H. Sung, E. Oh, H. Yuan and C. Zhu "Overview of the EVS codec architecture", IEEE, 2015, pp. 5698-5702.

3. 3GPP "TS 26.445, EVS Codec Detailed Algorithmic Description; 3GPP Technical Specification (Release 12)", 3rd Generation Partnership Project (3GPP), TS 26.445, 12 2014. [online]. At: http://www.3gpp.org/ftp/Specs/html-info/26445.htm

4. T. Vaillancourt, R. Salami and M. Jelnek "New post-processing techniques for low bit rate celp codecs", in ICASSP, 2015.

5. J.-H. Chen and A. Gersho "Adaptive postfiltering for quality enhancement of coded speech", edition 3, number 1, pp. 59-71, 1995.

6. T. Bäckström, Speech Coding with Code-Excited Linear Prediction. Springer, 2017. [online]. At: http://www.springer.com/gp/book/9783319502021

7. K. Han, Y. Wang, D. Wang, W. S. Woods, I. Merks and T. Zhang "Learning spectral mapping for speech dereverberation and denoising."

8. Y. Zhao, D. Wang, I. Merks and T. Zhang "DNN-based enhancement of noisy and reverberant speech", in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016.

9. Y. Wang, A. Narayanan and D. Wang "On training targets for supervised speech separation", IEEE/ACM Transactions on Audio, Speech and Language Processing, vol. 22, pp. 1849-1858, 2014.

10. F. Weninger, J. R. Hershey, J. L. Roux and B. Schuller "Discriminatively trained recurrent neural networks for single-channel speech separation", in IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2014.

[eleven. D. S. Williamson and D. Wang "Time-frequency masking in the complex domain for speech dereverberation and denoising".

12. Z. Zhao, S. Elshamy, H. Liu and T. Fingscheidt "A CNN postprocessor to enhance coded speech", in 16th International Workshop on Acoustic Signal Enhancement (IWAENC), 2018.

13. Z. Zhao, H. Liu and T. Fingscheidt "Convolutional neural networks to enhance coded speech", IEEE/ACM Transactions on Au-dio, Speech and Language Processing, vol. 27, no. 4, pp. 663-678, April 2019.

14. S. Das and T. Bäckström "Postfiltering using log-magnitude spectrum for speech and audio coding", in Proc. Inter-speech 2018, 2018, pp. 3543-3547. [online]. At: http://dx.doi.org/10.21437/Interspeech.2018-1027

15. W. Mack, S. Chakrabarty, F.-R. Stöter, S. Braun, B. Edler and E. Habets "Single-channel dereverberation using direct MMSE optimization and bidirectional LSTM networks", in Proc. Interspeech 2018, 2018, pp. 1314-1318. [online]. At: http://dx.doi.org/10.21437/Interspeech.2018-1296

16. F. Chollet et al. “Keras”, https://keras.io, 2015.

17. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane´, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Vie´gas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu and X. Zheng "TensorFlow: Large-scale machine learning on heterogeneous systems", 2015, software available for download from tensorflow.org. [online]. At: http://tensorflow.org/

X. Glorot, A. Bordes and Y. Bengio "Deep sparse rectifier neural networks", in ^ International Conference on Artificial Intelligence and Statistics, 2011, pp. 315-323.

S. Ioffe and C. Szegedy "Batch normalization: Accelerating deep network training by reducing internal covariate shift", in International Conference on Machine Learning, vol. 37, 2015, pp. 448-456.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov "Dropout: A simple way to prevent neural networks from overfitting", J. Mach. Learn. Res., Vol. 15, No. 1, pp. 1929-1958, January 2014. [online]. At: http://dl.acm.org/citation.cfm?id=2627435.2670313

D. Kingma and J. Ba "Adam: A method for stochastic optimization", in arXiv preprint arXiv:1412.6980, 2014.

NTT-AT, "Super wideband stereo speech database", http://www.ntt-at.com/product/widebandspeech, access: 09.09.2014. [online]. At: http://www.ntt-at.com/product/widebandspeech

J. S. Garofolo, L. D. Consortium and others "TIMIT: acoustic-phonetic continuous speech corpus", Linguistic Data Consortium, 1993.

A. Rix, J. Beerends, M. Hollier and A. Hekstra "Perceptual evaluation of speech quality (PESQ) - the new method for speech quality assessment of telephone networks and codecs", in 2001 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2001.

ITU-T "P.862.1: Mapping Function for Transforming P.862 Raw Result Scores to MOS-LQO", (International Telecommunication Union), Tech. Rep. P.862.1, November 2003.

"P.862.2: Wideband Extension to Recommendation P.862 for the Assessment of Wideband Telephone Networks and Speech Codecs", (International Telecommunication Union), Tech. Rep. P.862.2, November 2005.

27. Perceptual objective listening quality assessment (POLQA), ITU-T Recommendation P.863, 2011. [online]. At: http://www.itu.int/rec/T-REC-P.863/en

28. Recommendation BS.1534, Method for the subjective assessment of intermediate quality levels of coding systems, ITU-R, 2003.

Claims (146)

1. Audio decoder (100; 300) to provide a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ). 2. Audio decoder (100; 300) according to claim 1, wherein the filter (130; 360, 370, 380, 338) is configured to use a configurable processing structure, the configuration of which is based on a machine learning algorithm, to provide scaling values (136; 336; M(k, n)). 3. Audio decoder (100; 300) according to claim 1 or 2, wherein the filter (130; 360, 370, 380, 338) is configured to determine the scaling values (136; 336; M(k, n)) based only on the spectral values (132; 326; ) decoded audio representation (122; 322; ) in a variety of frequency bins or frequency ranges. 4. Audio decoder (100; 300) according to one of paragraphs. 1-3, in which the filter (130; 360, 370, 380, 338) is configured to obtain absolute value values improved audio presentation according to the following: , where M(k, n) is the scaling value, where k is the frequency index, in this case n is a temporary index, wherein ] is the absolute value of the spectral value of the decoded audio representation; or wherein the filter is configured to receive values improved audio presentation according to the following: , where M(k, n) is the scaling value, where k is the frequency index, in this case n is a temporary index, wherein is the spectral value of the decoded audio representation. 5. Audio decoder (100; 300) according to one of paragraphs. 1-4, wherein the filter (130; 360, 370, 380, 338) is configured to obtain scaling values (136; 336; M(k, n)) such that the scaling values cause scaling or gain for one or more spectral values (132 ;326; ) decoded representation (122; 322; ) audio signals or for one or more pre-processed spectral values that are based on the spectral values (132; 326; ) decoded representation (122; 322; ) audio signals. 6. Audio decoder (100; 300) according to one of paragraphs. 1-5, wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), with spectral values associated with different frequency bins or frequency ranges. 7. Audio decoder (100; 300) according to clause 6, wherein the input signals (372) of the neural network (380; 600) or machine learning framework represent logarithmic absolute values, amplitude or norm of spectral values of the decoded audio representation, the spectral values being associated with different frequency bins or frequency ranges. 8. Audio decoder (100; 300) according to one of paragraphs. 6, 7, wherein the outputs (336) of the neural network (380; 600) or machine learning based structure represent the scaling values (136; 336; M(k, n)). 9. Audio decoder (100; 300) according to one of paragraphs. 6-8, in which a neural network (380, 600) or machine learning framework is trained to constrain, reduce, or minimize the deviation (MSE _MA ) between a set of scaling target values (494, IRM(k, n)) and a set of values (484, M (k, n)) scaling obtained using a neural network (380; 580; 600) or using a machine learning-based framework. 10. Audio decoder (100; 300) according to one of paragraphs. 6-9, wherein a neural network (380; 600) or machine learning-based framework is trained to constrain, reduce, or minimize the deviation (MSE _SA ) between a target absolute magnitude spectrum (510), a target amplitude spectrum, a target absolute spectrum, or a target normal spectrum (510). ) and an absolute magnitude spectrum (592), an amplitude spectrum, an absolute spectrum, or a normal spectrum obtained using processed spectrum scaling that uses scaling values (584) that are provided by a neural network (380; 580; 600) or by a framework based on machine learning. 11. Audio decoder (100; 300) according to one of paragraphs. 6-10, wherein a neural network (380; 600) or machine learning framework is trained such that scaling to one or more spectral values (132; 326; ) spectral decomposition of the decoded representation (122; 322; ) of the audio signals, or for one or more preprocessed spectral values that are based on the spectral values of the spectral decomposition of the decoded representation of the audio signals, is in the range between 0 and a predetermined maximum value. 12. Audio decoder (100; 300) according to claim 11, in which the maximum value exceeds 1. 13. Audio decoder (100; 300) according to one of paragraphs. 6-12, wherein a neural network (380; 600) or machine learning framework is trained such that it scales to one or more spectral decomposition spectral values of the decoded representation of the audio signals or to one or more preprocessed spectral values that are based on the spectral decomposition spectral values of the decoded representation of audio signals, limited to 2 or limited to 5, or limited to 10, or limited to a predefined value greater than 1. 14. Audio decoder (100; 300) according to one of paragraphs. 6-13, wherein the neural network (380; 600) or machine learning framework is trained such that the scaling values are limited to 2, or limited to 5, or limited to 10, or limited to a predefined value greater than 1. 15. Audio decoder (100; 300) according to one of paragraphs. 6-14, in which the number of input features of the neural network (380; 600) or structure based on machine learning is greater by at least a factor of 2 than the number of output values of the neural network or structure based on machine learning. 16. Audio decoder (100; 300) according to one of paragraphs. 6-15, wherein the filter (130; 360, 370, 380, 338) is configured to normalize the input features of the neural network or machine learning structure to a predetermined mean value and/or to a predetermined variance or standard deviation. 17. Audio decoder (100; 300) according to one of paragraphs. 1-16, in which the neural network (380; 600) contains an input layer (610), one or more hidden layers (612a-612d) and an output layer (614). 18. Audio decoder (100; 300) according to clause 17, wherein one or more hidden layers (612a-612d) use linear rectification units as activation functions. 19. Audio decoder (100; 300) according to one of paragraphs. 17, 18, wherein the output layer (614) uses linear rectification blocks or limited linear rectification blocks or sigmoid functions as activation functions. 20. Audio decoder (100; 300) according to one of paragraphs. 1-19, in which the filter (130; 360, 370, 380, 338) is configured to obtain coefficients ( ) short-time Fourier transforms, which represent spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges. 21. Audio decoder (100; 300) according to one of paragraphs. 1-19, wherein the filter (130; 360, 370, 380, 338) is configured to extract logarithmic magnitude, amplitude, absolute or normal values (372) and determine scaling values (136; 336; M(k, n)) based on the logarithmic absolute value, amplitude, absolute or normal values. 22. Audio decoder (100; 300) according to one of paragraphs. 1-20, wherein the filter (130; 360, 370, 380, 338) is configured to determine a plurality of scaling values (136; 336; M(k, n)) associated with the current frame based on the spectral values (132; 326; ) decoded audio representation (122; 322; ), which are associated with different frequency bins or frequency ranges of the current frame, and based on spectral values (132; 326; ) decoded audio representation (122; 322; ) that are associated with different frequency bins or frequency ranges of one or more frames preceding the current frame. 23. Audio decoder (100; 300) according to one of paragraphs. 1-22, wherein the filter (130; 360, 370, 380, 338) is configured to determine a plurality of scaling values associated with the current frame based on the spectral values (132; 326; ) decoded audio representation (122; 322; ) that are associated with different frequency bins or frequency ranges of one or more frames after the current frame. 24. Apparatus (200; 400; 500) for determining a set of values defining filter characteristics (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) based on the decoded audio representation (122; 322), wherein said device is configured to obtain spectral values (132; 326; ) decoded audio representation (122; 322), which are associated with different frequency bins or frequency bands, and wherein said device is configured to determine a set (382; 412; 512) of values defining the characteristics of the filter (130; 360, 370, 380, 338), such that the scaling values (136; 336; 484; 584) provided by the filter based on the spectral values of the decoded audio representation that are associated with various frequency bins or frequency ranges that approximate the target scaling values (494), or wherein said device is configured to determine a set (382; 412; 512) of values defining the characteristics of the filter (130; 360, 370, 380, 338), such that the spectrum obtained by the filter based on the spectral values (132; 326; ) of the decoded audio representation (122; 322), which are associated with different frequency bins or frequency ranges, and using scaling values (136; 336; 484; 584) derived from the decoded audio representation (122; 322), approximates the target spectrum (510). 25. Device (200; 400) according to clause 24, wherein said device is configured to train a structure (380; 480; 580) based on machine learning, which is part of the filter (130; 360, 370, 380, 338) and which provides the values (136; 336; 484; 584; M (k, n)) scaling to scale the absolute value values (362, 372; 462, 472; 562) of the decoded audio signal (122; 322; 432; 532) or the spectral values (326; 446; 546) of the decoded audio signal to reduce or minimize the deviation (MSE _MA ) between a set of scaling target values (494; IRM(k, n)) and a set of scaling values (136; 336; 484; 584; M(k, n)) obtained using a neural network based spectral values (326; 446; 546) of the decoded audio representation, which are associated with different frequency bins or frequency ranges. 26. Device (200; 500) according to clause 24, wherein said device is configured to train the structure (380; 480; 580) based on machine learning to reduce or minimize the deviation (MSE _SA ) between the target spectrum (510; ) and spectrum (592; ) obtained using processed spectrum scaling (532; 546), which uses scaling values (584) that are provided by a machine learning-based framework. 27. Device (200; 400; 500) according to one of paragraphs. 24-26, wherein said device is configured to train a structure (380; 480; 580) based on machine learning such that scaling for spectral values of the decoded representation of the audio signals or for one or more pre-processed spectral values that are based on the spectral values of the decoded representation of the audio signals, is in the range between 0 and 2 or is in the range between 0 and 5, or is in the range between 0 and 10. 28. Device (200; 400; 500) according to one of paragraphs. 24-27, wherein said device is configured to train a structure (380; 480; 580) based on machine learning such that the absolute value is scaled for spectral values of the decoded representation of the audio signals or for one or more pre-processed spectral values that are based on the spectral values of the decoded representation audio signals is limited to a range between 0 and a predefined maximum value. 29. Audio decoder (200; 400; 500) according to claim 28, in which the maximum value exceeds 1. 30. A method (900) for providing an enhanced audio presentation based on an encoded audio presentation, wherein said method comprises providing (910) a decoded audio representation ( ) coded audio representation, wherein said method comprises obtaining (920) a plurality of scaling values (M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values of the decoded audio representation that are associated with the various bins by frequency or frequency ranges, and wherein said method comprises scaling (930) the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (M(k, n)) scaling values to obtain an enhanced audio representation ( ). 31. A method (1000) for determining a set of values specifying filter characteristics to provide enhanced audio presentation ( ) based on the decoded audio representation, wherein said method comprises the step of obtaining (1010) spectral values ( ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein said method comprises determining (1020) a set of values defining filter characteristics such that scaling values provided by the filter based on spectral values of the decoded audio representation that are associated with different frequency bins or frequency ranges approximate the target values scaling, or wherein said method comprises determining (1030) a set of values defining filter characteristics such that the spectrum obtained by the filter is based on spectral values of the decoded audio representation that are associated with various frequency bins or frequency ranges, and using the values scaling derived from the decoded audio representation approximates the target spectrum. 32. A digital storage medium containing computer-executable instructions stored therein that, when executed by a computer, cause the computer to perform the method of claim 30. 33. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the neural network (380; 600) or machine learning-based structure is trained in such a way that scaling for one or more spectral values (132; 326; ) spectral decomposition of the decoded representation (122; 322; ) of the audio signals, or for one or more preprocessed spectral values that are based on the spectral values of the spectral decomposition of the decoded representation of the audio signals, is in the range between 0 and a predetermined maximum value, the maximum value exceeds 1. 34. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the neural network (380; 600) or machine learning framework is trained such that it scales to one or more spectral decomposition spectral values of the decoded representation of the audio signals or to one or more preprocessed spectral values that are based on the spectral decomposition spectral values of the decoded representation of audio signals, limited to 2 or limited to 5, or limited to 10, or limited to a predefined value greater than 1. 35. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the neural network (380; 600) or machine learning based structure is trained such that the scaling values are limited to 2, or limited to 5, or limited to 10, or limited to a predefined value greater than 1. 36. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the filter (130; 360, 370, 380, 338) is configured to normalize the input features of the neural network or structure based on machine learning to a predetermined mean value and/or to a predetermined variance or standard deviation. 37. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the input signals (372) of the neural network (380; 600) or machine learning structure represent logarithmic absolute values of the spectral values of the decoded audio representation, the spectral values being associated with different frequency bins or frequency ranges. 38. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the neural network (380; 600) contains an input layer (610), one or more hidden layers (612a-612d) and an output layer (614); wherein one or more hidden layers (612a-612d) use linear rectification units as activation functions. 39. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ); wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) comprises a neural network (380; 600) or machine learning-based structure configured to provide scaling values (136; 336; M(k, n)) based on the set spectral values (132; 326; ), describing the decoded audio representation (122; 322; ), wherein the spectral values are associated with different frequency bins or frequency ranges; wherein the neural network (380; 600) contains an input layer (610), one or more hidden layers (612a-612d) and an output layer (614); wherein the output layer (614) uses linear rectification blocks or limited linear rectification blocks or sigmoid functions as activation functions. 40. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) is configured to extract logarithmic absolute value values (372) and determine scaling values (136; 336; M(k, n)) based on the logarithmic absolute value values. 41. Apparatus (200; 400; 500) for determining a set of values defining filter characteristics (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) based on the decoded audio representation (122; 322), wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ), wherein said device is configured to obtain spectral values (132; 326; ) decoded audio representation (122; 322), which are associated with different frequency bins or frequency bands, and wherein said device is configured to determine a set (382; 412; 512) of values defining filter characteristics (130; 360, 370, 380, 338), such that scaling values (136; 336; 484; 584) that are associated with different frequency bins or frequency bands, and which are provided by a filter based on the spectral values of the decoded audio representation that are associated with the different frequency bins or frequency bands that approximate the target scaling values (494), or wherein said device is configured to determine a set (382; 412; 512) of values defining the characteristics of the filter (130; 360, 370, 380, 338), such that the spectrum obtained by the filter based on the spectral values (132; 326; ) of the decoded audio representation (122; 322), which are associated with different frequency bins or frequency ranges, and using scaling values (136; 336; 484; 584) derived from the decoded audio representation (122; 322), approximates the target spectrum (510). 42. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) is configured to obtain scaling values (136; 336; M(k, n)) such that the scaling values cause amplification for one or more spectral values (132; 326 ; ) decoded representation (122; 322; ) audio signals or for one or more pre-processed spectral values that are based on the spectral values (132; 326; ) decoded representation (122; 322; ) audio signals. 43. Audio decoder (100; 300) for providing a decoded audio representation (122; 322; ) based on the encoded audio representation (110; 310), wherein the audio decoder includes a filter (130; 360, 370, 380, 338) to provide improved audio presentation (112; 312; ) decoded audio representation (122; 322; ), wherein the filter is configured to obtain a plurality of scaling values (136; 336; M(k, n)) that are associated with different frequency bins or frequency ranges based on the spectral values (132; 326; ) decoded audio representations that are associated with different frequency bins or frequency ranges, and wherein the filter is configured to scale the spectral values of the decoded representation ( ) audio signals or a pre-processed version thereof using (136; 336; M(k, n)) scaling values to obtain an enhanced audio representation (122; 312; ); wherein the filter (130; 360, 370, 380, 338) is configured to obtain scaling values (136; 336; M(k, n)) such that the scaling values enable both amplification and attenuation by scaling. 44. A digital storage medium containing computer-executable instructions stored therein that, when executed by a computer, cause the computer to perform the method of claim 31.

Images